Reputation Spec: Proposal and Critique


(Adam Capriola) #1

I was once an active member of an online community that employed a reputation system – which comprised of liking/disliking individual posts and gaining/losing reputation points with weight given by seniority of the caster – and I found it to be a fantastic incentive to contribute positively to the message board. Each user’s rep was clearly visible so I really wanted to build a positive reputation and I put a lot of effort into making my posts insightful. (“PLEASE LIKE ME!! I WANT TO BE COOL TOO!!”)

We have trust levels in Discourse, which sort of convey stature, but to me trust levels indicate activity level more than anything. Semi-active users will sit at trust level 2 and the most avid users will hit the pinnacle: trust level 3. :confetti_ball:

But trust levels don’t say much about the quality of a user’s posts. There’s got to be a better way to score each user on the merit of their contributions – and then also display their accrued reputation. I concur with @JJames that it would be helpful to differentiate users in some way.

Originally I was going to propose adding dislikes to penalize users for poor content (because I’ve seen this be effective) and factoring in likes to tally a reputation score, but I had trouble coming up with a good mockup and I imagine it would be intensive to assimilate dislikes into the backend.

Instead here’s a simpler idea:

Likes-to-Posts Ratio × Total Likes Ratio = Reputation

Likes-to-Posts Ratio = Your Total Likes Received / Your Total Posts
Total Likes Ratio = Your Total Likes Received / (Global Total Likes Received - Your Likes Given)

*Excluding PMs and specified categories.
*Flagged posts, mentions, and linked posts could factor in as well.

And here are a few example users (assume no likes given by them for this):

Global Total Likes Received = 1,000

  • User 1: 50 likes, 50 posts (0.05 reputation)
  • User 2: 10 likes, 2 posts (0.05 reputation)
  • User 3: 7 likes, 1 post (0.049 reputation)
  • User 4: 20 likes, 100 posts (0.004 reputation)
  • User 5: 10 likes, 50 posts (0.002 reputation)
  • User 6: 3 likes, 100 posts (0.00009 reputation)

I think that formula works pretty well for giving a relative reputation. Users 1, 2 and 3 all have the essentially the same reputation with varying like and post counts. User 4 has twice as much reputation as User 5 with the same like-to-post ratio but double the sample size. User 6 is not very reputable.

This should hold up over time too because the Global Total Likes Received will keep increasing, so there is incentive to keep posting otherwise your reputation share will naturally decrease. New users won’t be able to jump in with an initially high like-to-post ratio and become more reputable than someone who’s been around longer either. (There could also be a minimum post threshold to be met before a user’s reputation is calculated.)

The only problem is I don’t know how to depict the reputation score in a meaningful way. I haven’t taken statistics in a while, but maybe standard deviation could be used to place each user in a percentile which could then be represented with a bar under each user’s avatar:

– 50th percentile (average)

– 100th percentile (above average)

*Note darker green after 50% to more clearly show who is above average.

Hovering over the bar would give a tooltip with the percentile (and/or maybe user-defined terms for ranges of percentiles). You’d immediately be able to tell if the user in the conversation is generally a knowledgeable and helpful person.

SHORTCOMINGS

  1. This could discourage users from posting unless they think they’ll get likes. In some instances that would be a positive; in others it would be a negative. A good conversation doesn’t always need to involve likes.

  2. Users may also be less inclined to like each other’s posts. In the proposed formula a user’s reputation isn’t affected by themselves giving likes, but the act of giving a like then encourages others to like that post too (which will then detract from the user’s reputation). This needs to be addressed but I’m not sure how. Maybe compare the user’s likes given/likes received ratio to the global likes given/likes received ratio and deduct for a negative disparity?

  3. I’m oblivious to any obvious performance issues that this proposition would cause. Enlighten me!

  4. My formula or standard deviation idea might be awful using actual numbers. (Any statistics majors in the house?)

That’s all I’ve got … I wanted to at least throw this out there and see if anyone can improve upon the idea. Thanks for reading!


My thoughts about ‘reputation’ or ‘karma’
Thoughts on a plugin which turns discourse into stackoverflow
My thoughts about ‘reputation’ or ‘karma’
My thoughts about ‘reputation’ or ‘karma’
(Mittineague) #2

Maybe instead of a “dislike” you could put “flagged and agreed” into the algo somehow?
Suspensions, Warnings?


(Jeff Atwood) #3

I can only see this as a plugin. I think this would have some possibly severe repercussions to community.


What is the reason to not use reputation points?
(James) #4

How do you mean? In what way?


(Mittineague) #5

Does it bother you at all seeing where you are here in terms of Likes Received?
Can you imagine that for some members it might, especially if that information was made more easily obtainable?
https://meta.discourse.org/users


(James) #6

Fantastic. I didn’t even know about that link.

No, it doesn’t bother me at all. I think it’s a good way to motivate people to participate in the community.

I’m not familiar with Discourse’s plugin arch. How hard would it to build a plugin that displays something like this next to the user’s name?


(Mittineague) #7

Not all that hard I guess, IMHO the biggest problem would be “squeezing it” into the available space.

Info could be got from
https://meta.discourse.org/users/JJames.json

and put into “avatar bottom”


(Göktuğ Yılmaz) #8

Can you elaborate please?


What is the reason to not use reputation points?
(Erlend Sogge Heggen) #9

What is the reason to not use reputation points?
(OG) #10

I’d like to follow up on this subject as I see there is clearly demand for possibility of adding something to user profile.

I’ve gone through plugin development tutorials and I have plugin skeleton working, but I’d really like some guidance before I dive more into it…

I need some visually basic stuff… e.g.

For each post in topic I need to get poster data, do some basic calculation e.g.

Likes-to-Posts Ratio × Total Likes Ratio = Reputation

And with this I’d like to populate poster-avatar-extra div (as plugin outlet under avatar is now phased out) and user-card-post-names plugin outlet.

What is roughly needed to get this work… is it better to develop it as plugin or widget?

What is proper way to get and process poster data efficiently?

Are there any examples?


(OG) #11

I know there is possibly strong opposition against customizations “cluttering” profiles… But still…

Anybody? Thx in advance…


#12

Likes-to-Post Ratio is simple but I think that it will have a negative impact on discourse in many forums.

Many users will be reluctant to comment if they think that their comments will not receive likes.

Here’s two specific examples of negative consequences from using the Likes-To-Post Ratio.

Example 1 - Software support forum
When I report a bug with the software and the support staff agree that it is a bug I usually get a like. If they find my bug report incredible and nobody else has reported the bug then I don’t get a like. As often happens, other users will report the same bug at a later date. The support staff see that it is a “real” bug so the confirmation reports get a like but my original bug report doesn’t receive a belated like.

If a reputation system used Likes-to-Posts Ratio then I would be “losing” reputation for reporting bugs early. I could get a better reputation by only confirming bugs that had already been reported by someone else.

Example 2 - User who don’t give likes
There are many users who are stingy with their likes.

I looked for a ratio of Likes received: Likes given greater than 10:1 and found that at 41 likes and below there are many users who are that stingy. At a glance, I’d say 25% (someone else could generate a report that gave the exact results). Above that level of likes received there are only a few users who are that stingy and they appear to be developers.
185:2 Plugin developer
63:3 “great contributor” contributed 25 accepted pull requests
61:1 "I work at Apple as a senior web application developer"
53:5

A ratio of 10:1 is, in my view, quite extreme. Such users user will improve their relative reputation at the expense of other users by not giving any likes.

I don’t think that stinginess in giving likes should improve your reputation in comparison to other users.


(OG) #13

Hello, I would like to get purely technical advice regarding custom user reputation and profile fields… Should I post it in another forum?