How do you automate trust?


(F. Randall Farmer) #1

Paidcontent.org released an article today…

http://paidcontent.org/2013/02/06/fixing-online-comments-how-do-you-automate-trust

Let’s discuss it!

The social web has been around for more than a decade now, but even after all that time, no one has quite figured out how to fix online comments. Some bloggers have given up trying and don’t allow comments at all…

Discourse is first and foremost a discussion forum platform - not exactly the same as blog comments. I think it will probably require different features/plugins to be used effectively as a blog-commenting platform - at least as most people think of them today. One task at a time…

In addition to some other innovations, such as links that automatically expand within a comment (in the same way Twitter’s “expanded tweets” do), Atwood says he is trying to build a reputation system that will grant users new abilities based on the level of trust the platform has in them. Although he doesn’t provide a lot of detail, in a comment on a Hacker News discussion thread he suggests that it will be based on behavior such as flagging abusive posts.

Here’s where the lack of specificity leads to some early speculation about how “trust” mechanisms on Discourse will work. For example, the “flagging” reputation score is kept separate from other scores - it simply makes sense that being good at flagging content means that you can be more trusted at flagging content. [As I pointed out already in this thread]

Measuring trust and rewarding good behavior is something online communities have been trying to do for years, with mixed success. Some believe that sites like Slashdot — which has a moderation platform that awards “karma points” for certain behavior and appoints moderators automatically — have a good solution to the usual problems of trolling and flame wars, while others argue that these systems are almost always fatally flawed. Metafilter (which charges users $5 to become members) has many fans, but it is also a relatively small community. Branch is another attempt to reinvent user forums and discussion as invitation-only hosted conversations.

Why no mention of StackOverflow/StackExchange which are successful examples of trust scores used for unlocking features? How is StackOverflow “fatally flawed”? On the Spam/Troll end of the scale, there are many “automatic” and crowd-sourced reputation systems that filter out billions of pieces our junk-mail, filter spammers by IP address, and auto hide postings every single day. We don’t see most of it, exactly because of how effective it usually is.

Atwood says he wants to use a badge system for rewards (something Huffington Post also uses), but Gawker founder Nick Denton said in an interview last year that a similar reward system his sites used was a “terrible mistake,” because it was easily gamed and encouraged the wrong kinds of behavior. Denton has since completely revamped Gawker’s commenting system in an attempt to make reader comments the centerpiece, as well as a potential business model.

Bad badging systems work badly. It’s true. Here’s a story of gamification that actually killed a site. There are many of these stories:

All I can say is some reputation systems accomplish their goals, and some don’t. Most of the ones that failed that I’ve looked at were not well thought-through, designed or tested. As long as I’m helping Discourse, I pledge to do my best to work to provide incentive systems (badges, points, whatever) as options to forum operators that we have reason to believe will reinforce the behaviors desired: Timely suppression of the worst content, and recognition of the best.

Veteran blogger Anil Dash pointed out in an insightful post in 2011 that one of the only ways to maintain and encourage a healthy conversation — regardless of what platform you use — is to be involved in those discussions yourself as much as possible (a point Bora Zivkovic of Scientific American also made recently). Unfortunately for publishers looking for a quick or inexpensive fix, that kind of engagement is almost impossible to automate.

I agree deeply with the engagement requirement - how could anyone disagree (except someone that thinks online community is somehow a no-effort marketing-channel)? Discourse’s tools aren’t offering some magical solution to content moderation - we’re only trying to make the moderation task easier by offloading much of the work the the community, who by definition has more time-per-post than any moderation staff could ever have.

Just as “this is spam” helps mail providers get filter junk-mail, and StackOverflow uses upvotes to identity best content (and content authors) - This platform uses reputation systems to improve online discourse.

Help us identify the behaviors that you as a forum operator want to encourage and discourage so we provide you with the tools to make that happen.


(Philip Durbin) #2

Maybe. That’s the hope anyway. It’s a hard thing to measure.

I have a pretty good understanding of the repuation system in Stack Exchange (I’ve listened to every podcast about it). For the most part, it works quite well (though Stack Overflow has “big city” problems).

I’m quite a bit more fuzzy about how the reputation system in Discourse works. Maybe a screen shot of what a moderator sees would help.


(F. Randall Farmer) #3

It is possible to measure the “crap-end” of the quality scale pretty easily: Ratio of Flags to Posts, for example.

Not a lot to share there yet. Only the most rudimentary mechanisms are in place today - the team is will be rolling more out as the platform goes through teething pains and gradually matures…

I know we’d be very interested in any feedback/suggestions experienced forum operators would have about user/contributor behaviors you’d like to see encouraged or extinguished. And also, any mechanisms you’ve seen used effectively to accomplish these goals. We’re not looking to “throw the baby out with the bathwater…” and are looking far and wide for stuff that works fine (or might work better in Discourse.)

P.S. - Hmm. That’s an archaic expression. What’d be the modern equivalent?


(Philip Durbin) #4

I can’t argue with that.

I guess I could spin up a Vagrant image to look at the admin interface but that sounds like work.


(Philip Durbin) #5

Ok, I’m running vagrant up per https://github.com/discourse/discourse/blob/master/DEVELOPMENT.md

I hope it just works and I get something at http://localhost:4000


(Philip Durbin) #6

[quote=“pdurbin, post:5, topic:1785”]Ok, I’m running vagrant up per https://github.com/discourse/discourse/blob/master/DEVELOPMENT.md1

I hope it just works and I get something at http://localhost:4000[/quote]

vagrant up (and subsequent commands) worked fine and while the VM is quite slow on modern hardware I can log in as an admin (eviltrout) and see the following on a post when I click the wrench:

Under http://localhost:4000/admin/site_settings I’m seeing interesting things like:

  • cooldown_minutes_after_hiding_posts: How many minutes must a user wait till they can edit thier post after it is hidden due to flagging
  • ninja_edit_window: how quickly you can make an edit without it saving a new version

Lot of little dials to turn…

I’m not sure any of this gets directly to the heart of the claim “This platform uses reputation systems to improve online discourse” … but it’s interesting. :slight_smile:


(Shawn Holmes) #7

Automating trust is going to come by looking at multiple layers of behaviors. I’m only going to touch on one to start (since it is the easiest). This first metric is tracking, over time, a user’s accuracy and consistency at flagging off-topic posts.

It begins with a seeder: the Administrator, or user who initially set up the site. They are automatically at full trust.

As the community begins, various off-topic posts will arrive. Astute users will flag these right away. When the Administrator sees these flagged posts, and confirms that they are off-topic, closing them in the process, they implicitly close a trust loop with the original flagger. “You flagged this, I’m already trusted, and I agree with your assessment, therefore, you’ve gained trust.”

As this behavior continues, eventually, the user that flags off-topic posts grows to a trust level in which they themselves have the ability to close the topics. Eventually, they will be vetting other people’s flags, and in turn, causing those users to become trusted.

The inverse is also true. Flagging a post that isn’t actually off-topic may, in fact, have no effect at their trust (if it is low) or negatively impact their trust level (if they’re already well-trusted); this latter behavior is an example of the system saying, “hey, you’re a moderator now, you should know better than to flag something off-topic that doesn’t warrant the flag.”

This type of trust seems to be a good start, but again, only comprises one layer of automating trust. It gets a bit more complex from here on out. :smile:


(Jeff Lunt) #8

That was a great video - totally worth watching.


(Nicholas Perry) #9

This seems to be common trend where incentives are not used smartly: When Economic Incentives Backfire