Balancing Google Indexing with Group-Based Search Restrictions

Support
1

We’ve structured our Discourse forum so that category visibility is strictly controlled by groups. Our goal is to create an organized, personalized experience for logged-in users while ensuring that public content remains accessible for SEO purposes. Our current setup includes:

:check_mark: Group-Based Access – Users must join a group to see certain categories.
:check_mark: Tracked Categories – Upon joining a group, users automatically track its associated categories.
:check_mark: Muted Categories Behavior – Muted categories should be collapsed by default.
:check_mark: Search Permissions – Logged-in users should only see search results from the categories they have access to.
:check_mark: Public Visibility for SEO – Non-logged-in visitors (including Google crawlers) should be able to see all categories and index them.
:check_mark: Onboarding Requirement – New users must select a group before participating.

Challenges We’re Facing

:small_blue_diamond: 1. Group-Based Category Visibility Not Working as Expected

  • When a user joins a group, the expected behavior is that the group’s categories should appear on the homepage and category page.
  • However, this does not happen immediately—sometimes the categories don’t show at all or only appear after scrolling down.
  • Categories created earlier (e.g., Phlebotomists) behave correctly, while newly created ones (e.g., Bailiff) take longer to display or don’t appear until triggered manually.
  • Users often only see muted categories instead of the expected tracked/watched categories.
  • Temporary fix: If a user unmutes at least one category manually, other joined categories start appearing correctly.

:small_blue_diamond: 2. Search Results Showing Unrelated Categories

  • Logged-in users are seeing search results from categories they do not have access to.
  • Expected behavior: Discourse should limit search results to only the categories assigned to a user’s group.
  • Observed behavior: Search is still returning results from all categories, which means users can see snippets of content they technically shouldn’t have access to.
  • Related discussion: A conversation on ask[.]discourse[.]com confirms that Discourse should restrict search results based on group permissions.
    Screenshot 2025-03-12 at 6.30.10 PM
    Screenshot 2025-03-12 at 6.30.10 PM1814×2318 405 KB

:small_blue_diamond: 3. SEO & Google Crawling vs. Restricted Access

  • We want all categories to be indexed by Google so that content can be discovered in search results.
  • However, we don’t want logged-in users to see all categories by default—only the ones tied to their selected group.
  • Current issue: If we hide a category from logged-in users, it also disappears from Google, making it impossible to index.
  • Ideal solution: Google should be able to crawl and index all categories, but search results for logged-in users should still be restricted based on group permissions.

Questions for the Community

:one: How can we ensure that joined categories appear immediately and above muted categories?
:two: How do we fully enforce search result restrictions for logged-in users while keeping content visible for SEO?
:three: Is there a way to allow Google to index all public categories while still hiding them from logged-in users until they join a group?
:four: What’s the best approach to enforce group selection at signup without blocking public readers from discovering content?

Related Discussions & References

:pushpin: Group-Based Category Access Not Working as Expected – Discussion about category visibility delays, muted categories overriding tracking settings, and search behavior issues.

:pushpin: [Search Results Limited by Group Permissions (ask[.]discourse[.]com)] – Confirms that Discourse restricts search results based on group permissions, but we need a way to balance this with SEO.

:pushpin: Video Demonstration of the Issue:

3

Muting isn’t a restriction. Because you must use everyone to show content to randoms and Google, then every your users have access to all those categories, mute or not, and they’ll see right search results.

If you really are using group restrictions, and at same time keeping non-logged visitors and Google out from those restricted categories, then there is a bug. But I doubt that because it works here (and on my forum).

So my bet is you are using everyone for reading.

4

Thanks @Jagster for the reply, I am working with @emonunix on this.

Here is what we are trying to accomplish.

  • We want to give users access to discrete group/groups with access to categories
  • We want to hide all categories that are not in these group/groups
  • We want google to index everything (all categories)

Right now we are using ‘muting’ as a way to hide categories from users when they come to the site. The have to join a group to access categories.

Now the funny part is, they only are aware of a set of categories in their group…they cannot ‘see’ any other categories (because they are muted).

They then start to search, in theory for stuff in the categories they have access to and suddenly they see a bunch of stuff from categories they had no idea existed…that’s confusing them.

Is there anyway to accomplish this?

1 Like
5

And at same time every single user has access to all those categories, and see them in search results.

Perhaps you can break that ground security rule using plugins, or something, but out-of-the-box allowing something to everybody meaning at same time that something is not for everybody won’t happen.

6

Right @Jagster exactly why we are asking. We don’t know if there was another way.

Ideally there would be a way to limit logged in users to viewing and searching on X while logged out users (aka Google) can view and access Y.

Or maybe it’s a simple as making muted categories not accessible in the search tool…or maybe that’s just a design oversight? It seems odd that something is muted from your view, but yet you can find it via search…it’s like a back door.

7

Muted means it shut up and don’t pop up in /latest. It is not blocked. To hide something you have to use groups, and then you’ll keep Google out. As should, because at that point that content is not free anymore.

You would be in funny position when your registrate user seach and found something from Google, clicks it and arrives to your forum — but in restricted category. What would you do at that point:

  • force to make subscription
  • force to log out and read it as a stranger
  • show error 403

I don’t still understand your plans to block things for logged in user and yet show everything to randoms. It is like you would advertize what you have, but if that user doesn’t choose right, they don’t get it — or they must log out. Most of sites do opposite :smirking_face:

But no. Muting is not blocking.