Proposal: Consolidated Review Queue

Feature
18

I haven’t updated in a couple weeks, but I’ve made some good progress on migrating flags over to the review queue. I have been distracted this week with some family emergencies but those should be sorted out soon.

The refactor here is quite major and involves many changes. I am unsure how other team members will be able to review the PR eventually because it will be giant, but we’ll do our best!

I have reached the point in the refactor where I want to implement scoring properly. This means removing a bunch of settings we had for flags (min_flags_staff_visibility, for example) and replace them with score based equivalents. I wanted to jot down my idea for flag scores here and see what people thought before I go too far implementing:

  • a ReviewableFlaggedPost has a score

  • The score is the sum of the ReviewableScore records for that flagged post. Each ReviewableScore record represents a flag a user has applied to the post. The ReviewableScore score is calculated as user_flag_score + flag_type_score_bonus + take_action_bonus.

  • flag_type_score_bonus would be configurable by flag type. For example you could set spam be higher than inappropriate if you desired.

  • take_action_bonus: a value (0.0 or 5.0) depending on whether a staff member “took action”

  • user_flag_score is calculated per user, and is: 1.0 + trust_level + accuracy_bonus

  • trust_level is the user’s trust level (0.0 - 5.0)

  • accuracy_bonus is the percentage of the user’s previous flags that were agreed with * 5, for a value of (0.0 - 5.0). A minimum of 5 flags is required.

So for example, if a post was flagged by two users. One (u0) is TL1 who has never flagged before, and the other (u1) is TL3 whose flags are agreed with 50% of the time. Both are flags whose flag_type_score_bonus are set to 1.5:

# 1.0 + trust_level + (5.0 * agree_percentage)
u0_flag_score = 1.0 + 1.0 + (5.0 * 0.0) = 2.0
u1_flag_score = 1.0 + 3.0 + (5.0 * 0.5) = 6.5

# user_flag_score + default_for_flag_type + take_action_bonus
reviewable_score_u0 = 2.0 + 1.5 + 0.0 = 3.5
reviewable_score_u1 = 6.5 + 1.5 + 0.0 = 8.0

# sum of reviewable scores
flagged_post_score = 3.5 + 8.0 = 11.5

Reviewable items are sorted by reverse score, so this particular post with a score of 11.5 would show up before a post with a value of 9.3 and so on.

12 Likes