I’m considering using Discourse as the backend for my website and app, but I haven’t seen any documentation on its scaling limits.
What’s the maximum number of categories we can have? Is it 1000? 10,000? 100,000?
I plan to heavily use the API and create a custom frontend for both web and mobile. Part of this is programmatically creating a new Category whenever a user wants to. So it’s more of a backend systems question than the UI.
For example, will the /categories.json endpoint scale? It doesn’t seem paginated.
Thanks @sam! Follow up question – what would the approximate limits be for:
the number of tags
the number of topics in a category
For example, can I have a few million unique tags, as well as a few million topics within a single category?
The tags, topic_tags, and topics tables don’t appear to be sharded, which should work for a few million rows, but I haven’t looked into the queries themselves, nor any UI that could potentially break.
Even with optimal indexing, query speeds will at some point take “too long”. What that is will vary from person to person.
Then there’s Ruby, Ember, bandwidth and browser limits. Feel free to research, but I wouldn’t worry about those.
I think that long before any software limits are reached “sane” limits will have already been passed. And what that would be will vary from person to person and how they interact with the information.
tl;dr You will break your users long before you break any code.
Personally I never subscribe to a site that has 2500 categories.
Virtually you can create all the categories you want but your site will only be confusing and not navigable.
Note that an official answer has already been given to your question and also an alternative to using categories.
The problem of categories in the first place is the problem of transparency and navigation for users. Working previously on the catalogue dmoz.org and making categories for Russian directories, we are faced with the fact that with a large number of categories, problems arise in users. A large number of categories, it’s like there are no more categories. They become unnecessary, impossible to use. Nobody understands the structure of the site, including those who did it. I think that’s the problem.
Navigation should be navigation, and if it loses its functions, it is sad.
UI considerations aside, I’m curious what happens when you get up into the 1000+ categories range. Would responses just take a little bit longer on average, or are there catastrophic consequences? Is this something that can be solved by throwing better hardware at the problem?
Like the OP, I’m working on a project that could easily have 1000s of categories. It uses group based permissions heavily and an individual user may only be able to see a handful (this is why I can’t use tags). So I’m not worried as much about the UI being overwhelming, but would like to know what the bottleneck is at that higher range.
Honestly probably nothing terrible, unless you are admin and have access to 1000+ categories. At that point we would ship you every single category json in your first page, the categories page would get slow both server side and client side.
Relating to this:
If I have 100s to 1000s of categories, is there an option for me to limit the number of search results?
My problem: Due to a large number of categories, clicking on the dropdown when creating a new topic, or searching for categories when subscribing to them, has a long delay (10s of seconds).
I know we have an option to restrict the number of results in such scenarios to tags. Do we have/can we have the same for categories?
I have a site with over 500 categories (if I recall correctly). I think it was doing ok on an $80/month cpu optimized droplet with about 250k page views per month.
Interested to know how posting new topics works with so many categories?
The New Topic button on the homepages (Categories, Latest, New) requires you to select a category, unless you allow uncategorised posts. Selecting a category in the drop down seems impractical with so many in the list. How do you deal with that aspect?
I have 300+ (with a lot of sub cat though), but the admin is the only one to see that much.
You can search within the drop down, but you have to know beforehand the name of the category you want to post in.
The main “issue” for me would be to scroll to a far down category on the homepage (easily fixed by using the search feature)
I’m interested in this as well. My community is amateur radio operators, and there are 1000s of small local clubs. I would like to offer each local club their own private group to discuss club matters potentially.
It would probably be a long time before I hit any limits. I imagine I would probably hit a maximum of about 500 groups at full capacity, each with its own category.
These groups would only be displayed to users who were added to the club via API.
So (mostly) all categories are private and available only to members of the group? I’m not sure, but I think that solves much of the performance issues and if you think you’ll stay at about 500 you might be on even if I’m wrong. I think about 200 is what’s the recommended max, but I have a site with 200 or 300. And you’ll have warning so you could split them off into multiple instances of your numbers get too big.
