Feedback on plugin idea: sub-categories as "Discourse Arenas"

(Robbie Straw) #1

Continuing the discussion from Here’s my Categories as Sub-Reddit / Sub-Discourse Idea:

At Fatal(Syntax)™ we aim to facilitate the exchange of ideas. – Specifically we’re looking to use Discourse as a way to collaborate on designing several RPG campaigns.

Right now we’re testing this idea among a small group of close friends. As we start adding more projects and more users, we’re going to hit a wall in the not-so-distant-future.

As it is: we can’t segment off multiple projects without category explosion happening.

My goal is to implement the “sub-discourse” idea as a plugin, however I’m going to bolt on some interesting changes to the way permissions [trust] is applied across sub-discourse’s.

This includes: abilities to read and write to a sub-discourse, ability to password protect a sub-discourse, and the ability to scope moderation to one or more sub-discourses.

The status quo

  • Discourse offers categories as the only means to effectively separate discussions.

  • Many moderation features are site-wide, and they cannot be “scoped in” to a
    particular area of discussion.

  • You cannot “wall off” areas of the site.

  • Public/Private is a site-wide choice

  • All categories have the potential to be ranked and displayed on the front page.

A proposed solution:

Introducing: Discourse Arenas

Discourse Arena’s are a proposed sub-category/sub-discourse system that aims to fix the problems above:

  • An arena has simple, orthogonal dual-mode “read &write” permissions.

  • You can require a minimum trust level for viewing the arena.

  • You can require a minimum trust level for posting/replying to a topic in the arena.

  • You can also simply mimic the default settings of the main arena!

  • All other features [such as image limits, etc.] remain the same as the main arena.

  • An arena can be password protected and optionally marked as invisible.

  • An arena can optionally be excluded from the main arena’s topic listing.

Rather than an “all or nothing” approach, arena’s provide a safe way to segment discussion and provide a configurable level of independence from the main arena. The plugin will require you to have one arena, the main arena. This arena will be non-optional, and non-removable.

The main arena will be displayed as the site’s name [wherever arena’s are displayed] and it will be an overview of the discussion for your entire site. If you installed this [non-existent] plugin today, by default, all your posts and categories would be in the main arena.

All arenas will be shown on a separate menu, whose position will be based on available screen-real estate.

By default it will showup to the right of the topic listing.

If the width of the viewport is too narrow to support this, we will put an expandable menu up in the “user tray” as a fourth button to the right of the categories menu.

If an arena is configured to display its topics in the main arena, the ranking of those topics will compete with the topics of all other arenas. (In other words: one big popularity contest.)

Arena’s can [optionally] have a weight attached to them, to float or sink their post(s) as a whole. (For e.g: perhaps you’d want the Politics and Religion arena to be sunk to the bottom so it doesn’t garner too much attention from the front-page traffic / incoming search traffic.)

Apart from this, there is no way to influence the ranking of an arena’s post(s) in the main arena’s topic list.

If an arena is set to display in the main arena, but requires some minimum trust level, it will effectively render all posts as though the “invisible” flag were set to true. Writing permissions would disable the reply/new topic functionality (along with their corresponding UI features) if a user does not meet the minimum trust level.

Moderation of arena’s

This would be the most fundamental change to Discourse which is why I have left it for the end.

For my use case: it makes sense, in the context of multiple projects, that moderator(s) should have limited control over other forums.

Moderator-trolling is the worst trolling. – In the context of multiple [potentially competing] projects, I don’t think moderator’s should necessarily be trusted w/ the arenas of other projects. – At least, not initially.

At a minimum: there should be an intermediate trust step between sub-arena moderators and main-arena moderators.

As it is today: moderators can only be assigned from the /admin/users panel of Discourse. Working with the UI as it is, I think the functionality would work as follows:

Next to the “Grant Moderation” button there will be a drop-down which selects from the available arenas.

By default this will select the main-arena (e.g: apply site-wide).

Alternatively there will be main-arena-ONLY (where the moderator would not have access to moderate sub-arenas) – as well as a full listing of sub-arenas.

Upon clicking the “Grant Moderation” button, the [sub-]arena will be (A) removed from the drop-down menu, and (B) added to a listing of arenas they can moderate.

Next to their bullet in the listing, there will be a button to remove that
arena from their listing.

If main-arena is in their list of moderated arenas, then no further arenas
are applied to their permissions context.

Further arenas will still be available for selection, even if the user has been assigned to the main-arena. This way, if the main-arena context is removed, the user simply goes back to moderating their previous set of arenas.

The major point of contention [that I can see] is the changes to the moderation system.

As such: I hope to develop three features, in parallel, as separate plugins.

arena, arena-ranking, and arena-permissions

The latter two will, of course, depend on arena.
Aside from that: I aim for there to be no dependencies between the three plugins.

arena will provide you with the ability to add/remove arenas.
This will act as a convenient “sub-discourse” option for people who desire it.
However all moderation/trust will remain AS-IS.

This WILL include the ability to password protect an entire arena.
This WILL NOT include the trust-based read&write permissions.

arena-ranking will give you some basic abilities to adjust
the ranking of sub-arenas in the main-arena topic listing.

arena-permissions will include all permissions and moderation
modifications which will further segment sub-arenas from the main-arena.

Thanks for reading this far, I welcome questions, comments, and criticisms.

I hope to put up a public fork on github in the near future. – I’ll be starting w/ the base arena plugin, as I think it’s applicable and useful for a large number of Discourse installations.

(Jeff Atwood) #2

Interesting – good writeup. You may want to include screenshots so it’s a little clearer what you are proposing, and more people can visualize it.


I find that this is a dangerous way to develop software, to imagine the problems you might have before you have them and try to “fix” them in advance. It is always better to wait until you actually have the problem before attempting to fix it, otherwise you’re basically guessing. I like to call this imagineering.

A few of these things already exist, or are about to exist.

  1. Trust level 3 will include a “frequent flyers’ lounge” category that is visible only to them. Sam is doing the necessary groups work on this now.

  2. We do have a “hotness” level for each category which eventually will be used to determine how many of those topics in that category appear on the hot page, which will (eventually) be the new default.

(Robbie Straw) #3

I’ll definitely try to do some mockups and add them ASAP.

I tend to agree. Though it seems like the “sub-discourse” idea is a pretty hot one, so I wanted to try and tackle it. Most of the over-engineering would be specific to my [imagined] use-case, which is why I tried to split the problem space into three distinct chunks.

Then I think I’ll start by focusing on a solid arena implementation.

Since it sounds like the trust system is still being fleshed out, there is no point in “imagineering” that aspect of the system to my use-case. Not yet, anyways.

I’ll revisit the moderation issues when those features are a bit more ‘frozen’, and when I have the problems to back them up.

Thanks for the feedback!

(Robbie Straw) #4

Two photos to get started.


We’d make a slight change to the existing categories menu.
Above the categories we would list Arenas. The arenas would be sorted in the same order as categories.

There’s no visual cue here in regards to which arena / category is currently selected. – This would probably be in the body above the topic listing.

Next up would be the topic listing. – The homepage [the main arena] would list posts from all arena’s on the site.


In this picture you see three arenas. – The main arena (named after the title variable in site preferences) named Discourse, an arena for technology news, and an arena where users can talk about plugins.

If you were to select an arena, you would be taken to a topic listing including only topics from the selected arena.

I have some ideas for slight changes to the /categories page. I also think adding a dropdown for choosing an arena on the post composer would be desirable.

(Robbie Straw) #5

Just a quick update: I’ve been hacking some of this up over at GitHub - drbawb/discourse: A platform for community discussion. Free, open, simple. on the features/arenas branch.

I’m new to Ember.js so progress is moving slow but steady. I’ve also lost two 500GB hard drives that host a bunch of VMs, so my workstation is in crisis mode at the moment.

I did add some models w/o migrations, I’ll be adding those ASAP; but I don’t actually have any business logic yet so this shouldn’t be a problem.

Right now I’m hacking this into the actual Discourse source, mainly because I think it’s going to require some database changes [and it doesn’t seem like plugins are allowed to define their own migrations / models?]

Here are a few quick screenshots from my development box:

What you’re seeing here is the arena column added to the main topic listing. I plan to have the main arena just shown as a blank space [similar to how uncategorized topics are currently listed.

I’ve also added an arenas submenu to the topic list/categories drop down.

I’d like to start by getting these tied to actual models. After that I’m going to work on an arenas page that lists top (n) topics from various arenas, just like the current categories page does.

I’m hoping to spend some time this weekend wiring this up to an actual model.

@codinghorror since I’m hacking on the discourse core so to speak, do you think my repository structure is appropriate for issuing pull requests in the future?

I basically have the arena commits on a topic branch that is regularly merged with the upstream master. I’m not sure if these merge commits will interfere with a future pull request?

(Sam Saffron) #6

@drbawb I very much appreciate the work you are doing but worry about the current incarnation and naming.

@codinghorror is comfortable adding support for sub-categories (only one level), thing is we have not seen a need for it yet in any of the communities we run so have been less enthused about adding it.

I worry that this current design introduces a new concept (arena) that is not needed when category / sub-category is way easier to explain. I worry a lot about the clutter added on the front page, people have criticized Discourse front page of being too loud, this mock takes loud to the next level.

As to where the code should live, I am comfortable to accept a patch that adds backend support for sub-categories. However, if you are going to stick with arenas and the current UI mocks this belongs in a plugin. Plugins need to be able to run migrations, its a pre-requirement for us offering a great platform.

On a side note I noticed that page took 18 seconds to load in dev, how are you set up? It seems excessive even for a first request.

(Sam Saffron) #7

Also note, I have most of the backend for group support already implemented. It allows you to wall off categories.

(Robbie Straw) #8

Not sure about the load times: that’s on my MacBook pro which is current [i7, 8GiB RAM] but it’s usually a slow machine. (No SSD or anything.)

I’ll look into it and report back.

As for naming it sub-categories instead: that’s a fairly trivial change in terms of the UI I was aming for, and I was actually thinking about reusing the categories table instead of creating a new one for arenas.

As I look at this more: I’m just duplicating the categories table anyways, so I think it’d be easier to add something like a parent_category_id column [or just parent_id] to the categories table and leverage the discourse core a little bit more.

I actually saw this, I’ve been playing with it on my personal machine. (Obviously there’s no UI so I had to add the records by hand.)

It works really well, the /categories page didn’t filter correctly though last time I tested it. (e.g: it showed categories and topics I shouldn’t have been able to see; but clicking the topics yielded an access denied page) – the topic list filtered properly, however.

(Sam Saffron) #9

Yeah missed that, need to add some tests.

Lets focus on backend support for this stuff first, its way less bike-sheddy

(Robbie Straw) #10

As an aside my justification for adding sub-categories is that I want to use discourse to organize separate “projects” (something like github organizations I suppose, except for discussion, not source control).

I think one extra level will suffice for my purposes: I don’t really want to encourage going much deeper than that for a few reasons:

  • It’s hard to show that kind of hierarchy “at a glance” on the main topic page; and I still want all the site’s content to show up on the home page.
  • You shouldn’t have to go digging to find content on the site
  • My specific needs only require some way for many different projects to have [potentially] overlapping categories without confusion.

(Robbie Straw) #11

I’ve been reproducing about 18 second initial load times consistently.
Most page hits [even without code changes] take at least 900ms to load.

I wouldn’t worry about it: I can almost guarantee the bottle neck is at my database.

I’m going over the network [VPN] to a box that’s dedicated to PostgreSQL.

The db server is decently performant: my T1 is not. – Queries that take 0.5ms to complete in production can take 50-100ms to complete on my dev box; just because of latency and available bandwidth.

(Kiran Patil) #12

Since we are considering discourse as our discussion and training platform. We need subcategories at least one level.

Our use case would look like as follows,

Main Category: Software Testing
sub Category: Manual testing, Automated Testing, Selenium tool, Cucumber, BDD

Main Category: GNU/Linux
sub Category: Administration, Networking, Shell Scripting

Main Category: Database
Sub Category: Postgresql administration, PL/pgSQL - SQL Procedural Language

(Sam Saffron) #13

We do plan to get to it, but its not queued for the near future unless we get a strong request from one of our partners.

I would be happy to work with the community to design and get this feature into core if there is a bigger rush.

(Lowell Heddings) #14

I’m curious what a strong request would entail. I thought the plan was to avoid this entirely until the Hot tab was figured out… speaking of which, whatever happened to the Hot tab?

(Jeff Atwood) #15

We had to put the hot tab… on ice… for now.

Mostly we just didn’t need it yet with the current partners. It’s unlikely we’ll need it with the next one either, but we’ll see.

(Jason May) #16

This sounds a lot like category-specific moderation:

(Mittineague) #17

This topic is a year and a half old so I imagine most everything has been worked out or abandoned by now, But maybe not.

Hmmm. I’m not seeing that in this topic, what did I miss?