On the Necessity of Up/Down Vote-Based Reputation Systems on Boards Involving Subjective Discussion


(Alex) #1

I am using Discourse for a moderately-sized political discussion board.

The software is very good, and meets all of my needs except one: there is no up/down vote-based reputation system.

What I would like to see is a very simple system, where up/down votes are 1-to-1. The poster starts out with zero rep, and each vote by the community adds or subtracts a single point (better would possibly be that a poster with more rep points’ vote would be worth more than a poster with low rep points, but that wouldn’t be necessary). The reputation level would then be visible (somewhere) on each post. It would b very similar to the system Reddit uses.

Beyond this - and I know some people won’t like this idea - I would, if I could, implement a feature where if a person receives a certain amount of down-votes within a certain time period, they are automatically suspended for a temporary period. This would serve to fully democratize the moderation process, which is the goal of my proposed feature.

I have read all of the existing threads on this board discussing the concept of up/down voting and reputation points, and understand the position of the developers. Because this discussion has been largely philosophical in nature, rather than practical, I will address the philosophical along with the practical elements which create a situation where disapproving judgements by the community are needed to discourage negative forum behavior.

In the previous threads, the development team has presented an argument very similar to the arguments for purely positive-reinforcement-based parenting. This is an interesting topic which I’ve read quite a bit about. Though I find the concept somewhat utopian, many experts genuinely believe in it.

However, while this method may or may not work with children at home, what has been proven is that in a school or daycare setting, negative-reinforcement is necessary in order to maintain order. Even universities, obviously, have systems of negative-reinforcement to deal with anti-social or otherwise problematic/disruptive behavior. And what I deal with on my forum is a similar situation to that of a school, where personalities are prone to conflicting with one another and a system is needed to dissuade people from engaging in problematic behavior.

I understand that on support forums, such as this one or the Ubuntu board, there isn’t a need for negative reinforcement, as the communities have very specific purposes, and personalities and emotions don’t really enter into the scheme of interpersonal interaction. A positive-only reputation system, based on the existing badge system, works perfect in this environment, as the quality of a poster is based on their knowledge of the subject matter, rather than something more personal/nuanced.

However, on a large board where many emotional (and even personal) issues are discussed, I have found that chaos ensues if there is not a mechanism of negative reinforcement to discourage negative forum behavior.

Presently, the only means I have of using negative-reinforcement are manual and unilateral: I can temporarily suspend accounts or ban people outright. This strikes me as primitive, and often gratuitous, as well as non-democratic/authoritarian.

Within the context of political discussion, there is significantly more room for negative posting because you are dealing with the subjective, rather than the objective (the latter being what you are dealing with on a support forum, obviously). The emotional nature of subjective discussions leads to all different types of negative behavior, including insults/name-calling, the purposeful (or not) spreading of wrong information and stating positions on subjects that one knows nothing about.

A reputation system would serve two main purposes:

  1. To let the poster him/herself know that people do not like their posts, which will in some (hopefully most) cases cause them to reflect on what they are posting. Without this feedback, they may post indefinitely without being aware of the fact that people dislike their posts. Positive reinforcement can't work here, because the poster who doesn't receive many likes (or a badge for all the likes they've received) will likely be unaware they haven't received this positive feedback.
  2. To let other posters know that the poster in question is considered by the community to be a low-quality poster, which will make them much less likely to engage in debate/discussion with someone who the community considers to not be worth the time/effort. This is certainly more of an issue the larger the board becomes, as a poster is less likely to be familiar with all of the other posters on the board.

The importance of this to a large community where emotional discussion takes place cannot be overstated.

Beyond this, the current system of no reputation makes certain that everyone in a discussion is on the same footing at all times. I can see how in a Marxist/egalitarian sense that could be considered positive, however, on a forum where emotionally-triggering issues are discussed, people will often create an account solely for the purpose of arguing with others. Presently, there is very little ability to distinguish between quality posters who are integrated members of the community and people who have showed up simply to be contrarian.

Again, the option of using banning as a means to deal with posters who disagree with others is very primitive. It also results, in my experience, in a slippery slope, where anyone can potentially be banned for anything, as there is no way to scientifically define “flaming” or “trolling.” When people see others being banned for contrarian posting, they then fear making any statement disagreeing with another poster, and many will ultimately abandon the community due to a feeling of being arbitrarily restricted.

As a totally separate point (which is not especially important to me, but is worth mentioning), a reputation system encourages posters to work hard to provide good content. With the present system of badges, that incentive is much less prominent.

I am not myself a coder, but I would think that the system I am arguing for would be relatively simple to implement. A poster’s total “likes” are already calculated, so I would imagine you could just convert that system to an up/down system, where likes could be removed, then insert that number underneath posters’ usernames.

Finally, to wrap-up my pitch here, I will say that there is an ongoing battle between Discourse and Nodebb. Overall, it appears that Discourse is much better software. The problem I see people having, and the biggest danger to Discourse in this most important time of competition, is the lack of customizations leading to a less satisfying user experience. The up/down vote and rep system goes a long way toward enhancing the user experience.

More importantly, of course, is the ability to mitigate conflict on larger boards dealing with non-objective subject matter.

Looking at the directory, I see that the main use of this software presently is for support forums and closely-knit niche discussion boards, where conflict between users is generally virtually non-existent and thus there is little need for the democratization of quality control.

Adding the option of up/down votes would open up a whole new range of usability to Discourse, both on the level of the user experience and on the practical level of managing the community in a democratic fashion.

Clearly, due to the philosophical leanings of the development team, the positive-reinforcement only model should remain the default. However, options are always a good thing, and I am certain that I am not the only board admin who understands that the natural climate of my community requires negative-reinforcement in order to mitigate conflict between users and create an environment where quality posting is encouraged.

I’m very interested to get other people’s thoughts on this.


(Jeff Atwood) #2

You can view several talks I’ve given on this to understand the differences.


TL;DR:

  • systems of fact, data, and science can benefit from downvotes, because statements can be scientifically proven to a reasonable degree.

  • systems of opinion do not benefit from downvotes and are in fact materially and seriously harmed by them, because nobody can prove an opinion. Empathy is the order of the day in opinion systems.

On top of that be very aware that Q&A is not discussion, in fact Q&A are discussion are rather opposites. One is a classroom, the other is a restaurant.


What are "Likes"?
My thoughts about ‘reputation’ or ‘karma’
#3

In principal I like the idea of up/down voting whatever the subject matter, given I feel the like mechanism is shallow at best. However I tend to feel up/down voting is ideally suited to peer review (science, politcal science etc), although unless voting is weighted, takes no account of the qualification of the person pressing the button. I suppose the underlying seniority or trust level of the user could be used in the weighting, but that’s another discussion.

One area I have been looking at is scored quotes whereby a post is reviewed and awarded a score as opposed to a simple up or down vote. So for example a post may be quoted and scored in 3 places by another member with the aggregate of the scores counting towards the total up/down vote: +1 -2 +3 giving a final score of +2 or two up votes.

As for sin binning members based on feedback, I think that’s a potential issue. Given the nature of political discussions, I can see factions ganging up against each other - in effect taking part in a race to see who can kill off the other side first.


(Alex) #4

Just clicked on the first one and saw you start out with a comment about parenting. Good we’re thinking along the same lines as far as the sociology involved.

I’m going to watch these tomorrow and get back with you, because I find it very interesting. I’ve considered writing a book on the social dynamics of online communities.

I won’t comment on the fact v opinion issue until I view your talks.

Okay, one comment: your position there pre-supposes that consensus-building is negative in every context of opinion-based subject matter.

Regardless, this ultimately does go back to a practical issue, which I don’t presently see a solution to other than a democratic system of community validation of individual users.

As a totally separate issue, opinion-oriented discussions lead to emotional states which very often result in negative posting behavior. Obviously, there is no way to separate downvoting of poor posts which contain insults or inaccuracies and downvoting of posts based on a disagreement of opinion. Even if it is ideal to not have people downvoting opinions they disagree with, it is in my view necessary to allow downvoting in order to address anti-social or otherwise disruptive behavior.

The only other option is to use an authoritarian/unilateral system of doling out punishments.

While the community may or may not be good at building a consensus on opinion, I believe they are much better than mods at deciding who is behaving inappropriately.

Along with this, the other practical reality is that on a large forum having enough mods to regulate every discussion is in itself problematic. The flagging system helps, of course.


(Jeff Atwood) #5

First, watch both talks, or at least the brighttalk one. Please.

After that focus on real, concrete examples, “stuff that actually happened” versus hypotheticals that only exist in our heads.


(Sam Saffron) #6

This is not accurate at all. We have a very rich flagging system. We would certainly consider a system for optionally automatically suspending users based on flagged post count (eg: if you posted 5 posts that got flagged to oblivion its likely we should block your account.)

That said, I know newrelic have a plugin for up/down votes. It’s just not something I would ever see in core.

Using downvotes to build consensus sound like a recipe for disaster. Discouraging all dissenting opinions is not a way to promote healthy discussion.


(Alex) #7

What I have experienced is that people will flag posts of people they are in an argument with. I’ve even had to reduce the daily flag-limit due to flag-abuse.

Some flagged posts are obviously inappropriate and the need to delete or even temp-ban a poster is clear. But many flagged posts put the moderator in a position where they have to make a dictatorial decision on whether or not the poster deserves to be punished. My goal is to put the ability to approve or disapprove of a post back into the hands of the community.

I am unable to locate this.


(Jeff Atwood) #8

Again:

Based on my experience, I have never – literally, as in “has never happened” – seen flagging abused as a method of disagreement on any actively used Discourse I’ve looked at. And I have looked at hundreds.

Perhaps your community is truly unique, it is possible, but generally speaking this is just not a problem in any Discourse community I’m aware of.


(Alex) #9

I would suspect this particular issue is unique to my community or at least rare, yes. I am not trying to suggest this is a flaw. But it does happen in my particular situation.

The context is that of “NO U” style flame wars, where people call one another names and so on, then flag every post in the exchange as “trolling.”

However, I realize this is not especially relevant to the larger Discourse user-base, and perhaps the entire concept of this thread is as well.

Maybe it would be best if I moved over to the marketplace and see if I can solicit someone to build what I’m looking for as a plugin.


(Sam Saffron) #10

Newrelic have a “voting” plugin ask @nikdavis for details I don’t know where the code is. Regardless, if you want a marketplace topic you are going to need big :dollar:s to get something built.

How much do you want this? It would probably take a few months to build properly so 20-30k dollars seems like a pretty reasonable budget for it.

Do you feel 20 thousand dollars passionately that this feature is missing from Discourse?

In general when I see stuff find its way into #marketplace people are very passionate about “wanting” something but they are only willing to put 100-200 bucks behind this desire.


(Alex) #11

I was thinking more than $200 and less than $20,000.

I was also thinking there would be other users who are interested in this I could perhaps partner with. Maybe they’ll show up in this thread.


(Nik Davis) #12

We did commission an up/downvote points plugin. You can see it on our forum, https://discuss.newrelic.com below the user’s avatar.

There had been talks about open sourcing it, but we don’t have the assets to maintain it for the community right now.


Upvoting on New Relic Discourse, widely available?
(bob tang) #13

Is this possible for the plugin to be shared without support?


#14

I’d like to second this…


(Alex Reguyal) #15

Thanks @codinghorror for this post. I have pretty much decided to develop a community website using Discourse, but have been agonizing at the thought of employing a monumental effort to incorporate a true Q&A feature (a separate category) in the website. Now I feel that it makes more sense not to introduce the upvote/downvote feature at all.

I probably just need to do a little bit of customization for the topic page in the Q&A category:

  • The Reply button for the topic/question itself will be labeled as “Answer this question”
  • For all replies to the topic/question, the name of the poster will be preceded by “Answer by:”
  • The “Summarize topic” button will be available for all topics in the Q&A category if there are 10 or more replies. The top 5 Answers (i.e. replies to the topic itself) sorted by number of likes will be presented first, followed by all other replies.

I will really appreciate it if you could provide more insights and suggestions.