Just how bad is it to have 750 categories

I’m importing a Ning community about oil rights. The belief is that those who own rights in one county don’t care a whit about what’s happening in other counties.

image

I’ve read It’s Time We Talked About Tags and paid attention to, I think, every topic about the issue over the past ten months.

There are about 10 general interest categories, and then about 20 state categories and each state has 5-50 counties for a whopping total of something like 750. I’ve tried to figure a way around it, but have failed to find a better solution.

One thing I’ve considered is using groups to limit access to categories so that people would see only the categories they care about then they hit /latest or /top (which I can still do).

The problem with that scenario is that it’s important that search engines be able to see all of the categories.

It seems OK on the test site (4GB DO droplet). /categories loads in 1012.8ms and /latest loads in 726.

Is this a horrible mistake?

5 Likes

It’s not a disaster if you can keep it under 1000, but I am not sure users will be able to comprehend that many categories. I dunno. Try it, see how it works, and report back in 6 months?

It’s good that you checked the page load times, though, that’s certainly advisable.

So why not dump them into their tag view and have that be their homepage? I’m not sure why that would be so problematic? Do they really need security here?

2 Likes

Your willingness to edit people’s comments never ceases to amaze me, but that’s not important now.

That’s something of a relief.

I’d think the same, but they’ve been comprehending them on Ning, which I find hard to . . . comprehend.

Well, partly because I’m not that familiar with tag view, perhaps I should do some re-reading.

No. They don’t need security; using groups to limit access was my attempt at giving people just the trickle they want from the fire hose. They need to be able to see only the counties that they care about. If you’ve got minerals in Oklahoma, you don’t care about the goings on anywhere else.

And when the people in Whosit County Kentucky generate 200 posts a month about their dirt, there is a bunch of noise for most people.

Is it possible to mute all tags by default and then un-mute the ones you care about? That might solve the problem.

Just don’t ever visit the homepage; come in at your preferred county. If this is the case, the only time you’d ever need to see the homepage is on your first visit, yes?

Heck a static HTML page with a map that people clicked, linking to the appropriate tag for that county, would be the best homepage in this case, yes?

So maybe the feature request is to make a tag route your preferred homepage, or even a category cc @sam we already allow people to select latest, categories, or top as their preferred homepage… why not a category or tag (or group!), at that point? why not?

7 Likes

That almost does it.

I have a topic that is a list of very county category, so that’s a start, but you might care about 10 county categories plus the 10 top-level general interest categories.

Yeah, this is how I plan to deal with this issue on a similar setup. Direct people straight to their ‘home’ category.

If I hit performance problems with 1k categories, I’ll also restrict the loading of categories in the site model somehow. It loads all categories by default whenever you load the site.

@pfaffman I’m very curious to hear how this turns out actually.

1 Like

It looks like you have a category per county. With state code as a “comma code” at the end.

Is it too late to rethink hierarchy? For me, having parent categories would help. If not by state, some other geopolitical boundary. I might miss out on something “hot” happening in a far county. But would I be all that interested in keeping up with three time zones away?

Mostly. We have State parent categories, so there are only about 30 top-level categories.

A hornet’s nest is the only thing that comes to mind when i think of managing 750 categories. Hell, Craigslist likely covers the entire country with less than 750 categories.

Could you create one category per state and create sticky topics in each category to cover the counties? You’d also have the freedom to use the category for ‘general state talj’ etc.

1 Like

I might try country region as parents with states as subs, then tags for the counties. As tags, leaving the state code in the name would a good idea.

1 Like

If that’s true, it’s because they skip over a whole bunch of places. 750 is just counties that have mineral rights. And about 30 states don’t have any.

Again, they managed 750 “groups” on Ning, so that’s what they’re used to (which doesn’t really make it any less crazy, but that’s another matter).

4 Likes

I have similar issue:
Lots of main categories (about 800) with about 5 subcategories. Common users would only have access to about 10 main categories.

The question will be:

  • would common users have performance problems even if most of the main categories are not accessible they because of access right?
  • what about admins who have access to all main categories? Will the system just explode because of the amount of categories?

Discourse is a great tool to have a ecosystem for discussions . I wonder, if I’m nearly the only one who has that big installation which uses groups to restrict the access to categories that much.

1 Like

I’ve got a site with half a million monthly monthly views with several hundred categories (I think it’s only about 300? EDIT: it’s 814). It’s on a reasonably fast system with 2 cores {4 threads} and 16 gb ram. Performance is ok. I think that it was on a digital ocean 8gb cpu optimized instance.

I think that all you can do is to try. If those groups have little interaction across each other, you could make it a multisite instance which would split them across multiple databases. They could share a single site as the SSO server.

Median performance is pretty good, but 99th percentile is pretty bad:

4 Likes