Subcategories view problem for site larger than 1,000 categories

AI-generated summary

A user reports a bug where subcategories in a category view are limited to 20 when using ‘Boxes’ layout, despite the full list being visible in the dropdown. This occurs on a site with over 1,000 categories, and the issue stems from Discourse’s handling of large category lists. av203 suspects the problem lies in the CategoryList logic and view rendering. HAWK confirms the limitation is intentional due to performance concerns, but acknowledges potential future improvements. av203 requests configurability for MAX_UNOPTIMIZED_CATEGORIES or CATEGORIES_PER_PAGE, noting that monkey patching is fragile. sam agrees that monkey patching is undesirable and suggests increasing CATEGORIES_PER_PAGE to 100. nbianca confirms this is a bug and has already fixed it in a PR, explaining that the issue arose from incorrectly applying lazy loading to subcategories on large sites.

I’m hosting a site that has more than 1,000 categories. The community it is running for prefers using categories for semantic separation over, for example, tags.

This is working reasonably well, however, I encountered an issue recently (and I’m not sure when it started):

Browsing to a category view that has more than 20 sub-categories and uses “Boxes” to render the subcategory link elements, renders only (the first?) 20 categories in the list.
Clicking on the details dropdown select element does show the full list.

From my initial investigation, I encountered discourse/app/models/category_list.rb at main · discourse/discourse · GitHub and how it has some behavior around handling sites that have more than 1,000 categories.

I suspect that together with how the view discourse/app/views/categories/index.html.erb at main · discourse/discourse · GitHub handles the category list it receives, and does not paginate, causes the view to show only 20 subcategories.

Is this the intended behavior?

Yup, I think you’re bang on with your suspicion. Discourse was not designed to handle thousands of categories, for lots of reasons. We have been doing some work on category loading so that may change in the future.

3 Likes

Thanks for clarifying.

Would you consider at least making CategoryList::CATEGORIES_PER_PAGE or CategoryList::MAX_UNOPTIMIZED_CATEGORIES configurable in some manner? even a hidden site setting would work for my case.

Writing plugin code to replace these is somewhat ugly as I can’t find any other way than monkey patching CategoryList which feel extremely fragile.

Entering the world of monkey patching is certainly not ideal.

That said I would be worried about fiddling with CategoryList::MAX_UNOPTIMIZED_CATEGORIES , we picked that cause stuff gets problematic at higher numbers. CategoryList::CATEGORIES_PER_PAGE though could probably comfortably push up to 100.

@nbianca / @hugh any thoughts here?

3 Likes

This is a bug that I have just fixed here:

https://github.com/discourse/discourse/pull/31507

We have introduced this feature because sites with over 1,000 categories had an unusable “/categories” page. Unfortunately, this introduced a bug because if a site has more than 1,000 categories, it will try to force lazy loading (or page by page) of subcategories too even when it is not the case.

4 Likes