I want Google to index the content in the link format domain/u/username, what should I do?
I have removed the Disallow: /u part to let the link format domain.com/u be indexed by Google, but what should I do if I want the slug format domain/u/username to be indexed by Google? Is there any other configuration?
I have not logged in and cannot access the slug format: domain/u/username, so I see that it is a problem for Google to index this slug format (attached image).
If I remember right that limit comes a bit deeper than robots.txt. Sure, it is quite fast test allowing /u in robots.txt. But if it comes deeper from the core you may need a plugin.
Dig here. That was a thing perhaps a year ago, came along with SEO-stuff, and there was strong reason why it is not allowed.
I know the url is Noindex but I don’t know where to edit it in the source discourse. In the robots.txt file, the Disallow: /u part has been removed. I have attached a photo.