An idea for Discourse AI: automatic management of mega-threads

We are experimenting moving from mailing lists to Discourse as part of the Fedora Changes Process. This is the way by which developers propose possible major changes to the operating system, for coordination and communication. It happened that the first such change is somewhat controversial, and got hundreds of replies in the first few hours. (Here, if you like.)

As everyone probably knows, gigantic discussions are difficult no matter what the platform. Discourse has an opinionated design, where there is internal tracking of replies (and links within the page) but everything is presented linearly. The basic idea[1] is that gigantic megathreads are terrible and nested hierarchical presentation doesn’t really help. I’m a convert to this school of thought — it’s better to have “break-out” linked topics. Ideally, people would 1) create their own if needed, and 2) follow those breakouts with new posts. But, creating linked replies is kind of a stealth feature (poor discovery), and people just naturally don’t self-categorize well, so mods have to spend effort keeping things where they’re supposed to be.

Most importantly, when something big happens quickly, human moderators can’t respond quickly enough. Enter AI. I’m really skeptical about AI for content generation,[2] but it’s demonstrably good at classification, and decent at summarization, too. I’d love a feature like this:

  1. Enabled per-category or per-tag.
  2. When a topic meets a threshold[3] :
    • at least 5 unique posters
    • 50 replies in 12 hours
  3. or when manually selected by a moderator
  4. First post (considered separately!) and replies are used to sort these replies into 3–5[4] break-out topics.
  5. AI generates titles for each of these, and creates new linked topics for each
    • first post includes a pre-written per-category template, with space for a generated paragraph describing the break out
  6. Optionally, also creates a “Misc/Other/General” breakout for things which do not fit cleanly.
  7. AI moves existing replies into the appropriate break-out.[5].
  8. AI creates a new reply to the main topic[6], listing links to all of the breakouts and directing discussion there.
  9. Main topic is closed to new replies. [7]

That could be the end of it. If break-outs themselves meet the threshold, the same could apply with more specificity. Or…

  1. AI monitors breakouts and if there are posts that belong in the other break-outs, moves them.[8]
  2. The “Misc/General” thread (or original topic, if there is no misc/general and the original isn’t closed) is monitored for new major themes and the AI creates new break-outs as those emerge, linked from the main topic[9]
  3. If a break-out subsequently has replies which drift to a new topic that matches the top level more than being a “sub-breakout”, that gets added as a linked topic from the main topic.
  4. If a break-out itself exceeds the original trigger threshold for this feature, rather than the thing happening all the way down, the split should only happen if there is a strong bifurcation[10] — 200 replies all really about the same thing should stay in the one break-out. [11]

  1. there’s a topic about this somewhere around here — I’ll link it if someone can find it for me!) ↩︎

  2. no matter the appeal of the chatgpt party trick ↩︎

  3. all numbers should be configurable — but I’ll use specific examples rather than repeating that ↩︎

  4. this should be configured as a range, and the AI should decide on more or fewer depending on the content ↩︎

  5. super-bonus — posts which cover more than one subtopic but which can be cleanly split should be split and each part moved into the relevant break-out ↩︎

  6. More prominent than a “mini-post”! ↩︎

  7. Or, left open, but new replies automatically moved? I think it’s better to close. But, if there is no “misc” topic, probably best to leave open instead. ↩︎

  8. or, flags them to be moved by moderators? ↩︎

  9. with a reply to the main topic linking prominently ↩︎

  10. currently defined in a hand-wavy way, but I hope someone can put his more concretely! ↩︎

  11. I know it’s tempting to say “if someone doesn’t have anything new to add, they should use Likes or Reactions instead”, but for hot topics people like to get their voice in, so this gives a place for it. ↩︎

5 Likes

Any progress, added thoughts, FYIs, etc. with this idea?

Are you quietly getting support and/or push back?