When we founded Discourse in 2013, I had the idea that we’d support topics of any length or size, from 10 replies to 100 to 10,000 to 100,000 or even more!
A few years ago we ran into a technical limitation where we send down a list of all the post IDs in the topic when you enter the topic. This starts to cause problems on the client – particularly older smartphones with less memory and CPU power – at around ~10,000 replies, so we created a site setting that automatically closes topics at 10,000 replies and defaulted it to on. It’s certainly possible to turn this setting down or off, but we believe in “safe by default” at Discourse, and extremely large topics were no longer safe, at least for some clients.
This was meant to be a temporary solution as it was a significant amount of engineering effort to create a solution where we sent down only a partial list of topic ids based on what you are viewing. We planned to revisit this later.
However, as we started to investigate MEGATOPICS, it became clear that they are
not that common
often quite problematic
… and now after looking at this for quite some time, with many live examples, I believe that MEGATOPICS should, in fact, be permanently discouraged. Here’s why:
Megatopics are impractical to read. By the time you reach 10,000 replies, who in their right mind is going to sit down and read through all 10,000 of those replies? So how does anyone catch up with the current state of the topic – maybe by reading the first post and the first 50 replies? Reading only the last 100 posts? Even using our handy “summarize this topic” function, you’re looking at reading 1,000 of the most liked replies to understand what went on in that topic. Your best bet is for someone to read the whole thing and edit a summary into the first post, which would require a staff member, or the first post to be a wiki.
Megatopics are impractical to navigate. How long did it take this topic to get to 10,000 replies? A week? A month? A few years? While you can easily jump to the beginning and end of any topic using the handy timeline feature in Discourse, you are on your own figuring out what exactly happened in the vast middle of that topic. We’ve discussed adding breakpoints or chapters to topics at some point, but there were certainly important “events” in any long running topic you’ll need to dig through the dates and posts to figure out yourself.
Megatopics are genuinely rare “in the wild”. Right now I’m looking at the archives of a forum we migrated to Discourse in 2017 that dates back to 2003, and sorting by replies, I count 7 topics out of 87,602 that are over 10k replies. Seven! That’s about 1 megatopic for every ten thousand topics. There’s also a fairly steep dropoff in topic length at that point; only 9 additional topics are over 8k replies.
Megatopics tend to lose control of the actual topic. It is normal for a topic to veer and meander, but the longer the topic, the more likely it is that gentle conversational curves morph into radical changes. Furthermore, the longer the topic goes on, the more the world changes around the original opening post that kicked things off – if it is about a then-current news event, for example, other more recent news events may have rendered it moot or significantly altered its meaning and trajectory by now.
Megatopics are often users expressing a desire for a chatroom. There’s nothing wrong with the desire for a chatroom, but chat works best in actual chat software, which is designed to host impermanent, rambling, neverending conversations about anything – and where users can create their own channels at will. For example, here’s a topic titled Ask me [warhammer] lore questions which currently has 58,283 replies… and counting.
So here are my recommendations:
Use natural breakpoints in topic titles: years, seasons, months. A topic titled “Best Horror Movies” will be hard to find and hard to navigate. But a topic titled “Best Modern Horror Movies” or “Best Horror Movies of 2018” will fare much better.
If something significant has changed, start a new topic, and link to the old one. Instead of adding to the “Diablo III” topic, create a new “Diablo III – Reaper of Souls” topic to cover the game’s latest expansion and patches with the changes to the core game as a result. In discussing the Houston Oilers after their 1999 move to Nashville, create a new “Tennessee Titans” topic.
For big timeline events, consider creating a new topic. Big events are often worthy of their own topics. A huge release like Smash Brothers certainly deserves a new “Smash Brothers on Switch” topic where people interested in that can easier find it, versus being buried deep in the general Nintendo Switch catch-all topic. This can be linked both to and from the general Switch topic (or the general Smash Brothers topic) as needed, of course, and now it can be found easier via search as a standalone topic.
If your users are implicitly telling you they want chat, maybe set one up? It’s perfectly fine for Discourse to live side by side with ephemeral chat, and it’s exactly how we use Discourse internally! Chat doesn’t compete with the longer form conversations on Discourse, it is highly complementary to those conversations. There are plenty of great free chat options out there, like Slack and Discord.
I agree - on a forum there is limited use for a 10,000+ post Topic.
I would just have it so that when it reaches 10K, it’s automatically closed and the system posts a continuation Topic and then links the two. (Perhaps with a mod flag to bypass the restriction).
There’s also the use case of megatopics as landfills. The admins want you to stop trashing the forum with a bunch of garbage, so they’ve given you a landfill to dump it all in.
If off-topic chatter is becoming a problem in a community, instead of relegating it to a single topic within an existing category maybe create an entire “off-topic” category or sub-category. This at the very least grants the content some ability to have structure.
I honestly see no reason as to why this shouldn’t be left in the hands of those who are in charge of the forums. If they want topics to be over 10k then they should have topics over 10k.
Especially if it is a forum where such a thing is quite common.
I guess people can just make “[topic title] part 2” and so on but at that point you gotta wonder why not have it be 1 thing.
With this change we now have a topic sitting at 61k that is locked. Ho boy would that be a lot of parts.
At the end of the day it is up to you, but this topic is about ensuring you understand the ramifications of the choice you are making for the members of your community.
While Discourse is just a part of your life, we live and breathe it and it would be remiss of us if we didn’t ensure that everyone is armed with the knowledge to create the best experience possible for their audience.
For our hosted customers this is suppressed unless you are an enterprise customer. Allowing megatopics in our multisite cluster causes performance problems so we just restricted this. Note, we do plan to change it so you can increase the PM limit up to 10k.
I’ve used this in the past. In my case, I also muted that category for all members and posted an announcement explaining how you can un-mute it if you’re interested in off-topic discussion.
Agreed. I think the following additional things should happen when a megatopic gets automatically closed:
A continuation topic is automatically created. E.g. “What did you have for lunch today?” gets automatically superseded by “What did you have for lunch today? (2)”. This would work exactly like our Linked Topic creation, along with a closing notice linking to the new linked topic.
Chances are if a Topic reaches 10,000 posts - 10,000 is not the natural limit. By closing it you are shutting down something that members, more than likely, want to continue.
Maybe just add a mod setting in addition to what @erlend_sh has said “do not continue topic if reaches threshold”. I would also add an admin setting too (I’d probably want to keep the max to what may be more optimal for the server - so maybe half that at 5K?).
Having said that, if simply auto-closing it is much easier to do - do that, as there’s plenty of other (more useful) things that your resources could go on
The SitePoint vBulletin forums had a few “mega threads”. But they were not what I’d call discussions. A few were “game” threads which were more post-to-post where there was no presumption that anyone would read every post from first to last. And there was a catch-all Introductions intended to consolidate multiple “Hi, I’m new here” threads. There were also some “opinion” threads which were little more than a series of “I like __ best” with a sprinkling of substance here and there.
I guess whether or not a community wants non-discussion topics in its forum is up to the community. But in my experience and in my opinion any discussion that starts to get anywhere near “mega” has become full of tangential and repititous posts that make reading from first to last such a major task that it is highly doubtful anyone would do so.
The limit is already a site setting (and has been for quite a while). See auto close topics post count and auto close messages post count. If you are self-hosted you are free to change these settings as much as you wish. On discourse.org hosting we’ve enforced the 10,000 post count setting globally (as Sam mentioned above), so that cannot be changed.
Revisiting this after a year, and after some experience with the philosophy of decentralized social networks, I have another way to frame the problem:
Discourse didn’t want to fix the performance problem because megatopics are trying to scale the software past the human limits.
10,000 replies is clearly arbitrary. But it’s also clearly way too many posts for a single topic. It’s just like the follower / following performance limits on Twitter-like platforms: if you actually need 100k followers, consider a blog instead
In quanto persona relativamente nuova su College Confidential e genitore che cerca di aiutare un figlio al terzo anno di scuola superiore nel processo di ammissione all’università, trovo questi [megatemi] schiaccianti. Sono sopraffatto dalla generosità di persone come doschicos nel fornire informazioni pertinenti. Sono anche sopraffatto dalla lunghezza e dalla complessità di questi [temi]. Li ho trovati difficili da leggere e ancora più difficili da analizzare ai fini della moderazione.
I megatopic sono problematici se hai un forum orientato a compiti discreti con fini definiti. Le discussioni comunitarie aperte possono facilmente raggiungere i 10k di messaggi in breve tempo rimanendo su un unico argomento ristretto.
Co-amministro un forum con meno di 500 utenti registrati che discute di notizie e attualità e raggiungiamo spesso il limite. Siamo già a metà del nostro terzo topic dedicato al COVID-19, con diversi topic collaterali. Il secondo è stato chiuso dopo soli 21 giorni.
Detto questo, anche per noi non è un grosso problema aprirne uno nuovo. Anche se teoricamente preferirei avere topic teoricamente infiniti disponibili, è qualcosa a cui possiamo rinunciare se implementarli richiede uno sforzo enorme.
Tuttavia, non credo che i megatopic debbano essere considerati un caso limite raro, né si dovrebbe assumere che siano sempre una cosa negativa. Alcune comunità consolidate vanno addirittura orgogliose di avere singoli topic sia enormi che di lunga durata.
Lo stesso vale per il nostro argomento sul COVID-19. Abbiamo dovuto chiudere il primo dopo 2 mesi, poi ne abbiamo iniziato un secondo che chiuderemo alla fine del primo mese. Ne inizieremo uno nuovo ogni mese, dato che il numero di post sembra crescere molto rapidamente. Per onor di cronaca, loro (noi) rimaniamo per lo più in tema, con solo pochi post che si discostano verso la politica, ma ormai questo è strettamente intrecciato con la politica. Abbiamo anche un argomento sul COVID-19 esclusivamente per la ricerca in corso e un argomento solo per tracciare la diffusione del virus, il numero di casi/decessi, ecc.
Manteniamo i nostri argomenti a un massimo di 1.000 post. Siamo un piccolo forum privato, il che ci salva dal dilemma che altri affrontano.
