Allow google access to logged in categories

Support
1

So making going into /admin/customize/robots
Using sitemap plugin, I noticed this at the bottom. If I wanted to allow access for google to all posts that require login to view how would this part be changed?

Or would I just use Allow statements for all the categories?

I’m strictly looking at SEO as I changed the visibility of some categories recently and would like google to crawl the pages still.

<%- if SiteSetting.sitemap_enabled? %>
<%- unless SiteSetting.login_required? %>
Sitemap: <%= request.protocol %><%= request.host_with_port %>/sitemap.xml
<% end %>
<% end %>

1 Like
2

If either a category or the whole forum aren’t public, there is no way to allow Google to crawl it.

2 Likes
3

The forum is public. Some categories are not. What I am trying to accomplish is people being forced to login to view the categories but allow google to see the pages and crawl it.

I was reading online with robots file that could be accomplished with Allow: but the example only showed how to allow a specific page and not a whole category.

1 Like
4

It is not possible. Non-public categories require a login to be access. Google’s web-crawler does not have a login (it’s anonymous). Therefore, Google cannot see the category, or any of its content.

An aside, if Google was able to access the content without a login, so could technically-knowledgeable users.

4 Likes
5

Gotcha. Is there anyway to do the a forced popup to show the category to login on a per category basis? In sense of hiring someone to do a plugin for this?

1 Like
6

I’m not aware of an existing plugin that allows that, but there’s definitely been discussion about this before here on Meta. A quick search should bring it up.

1 Like
7

Google will ban you for this behavior. Do some research.

8

Thanks. I saw it on an SEO blog but isn’t worth the risk. I’m just going to hide it with CSS off the main categories page.

1 Like