toanvoc wants Google to index user profiles in the format domain/u/username, but they are currently being blocked from indexing. Don suggested disabling the “hide user profiles from public” site setting, which toanvoc tried but was still unable to submit the link for indexing.
Jagster mentioned that the limitation might come from the Discourse core and suggested using a plugin. toanvoc had already tried adding the link to robots.txt, but it didn’t work. j127 discovered that user profiles send an x-robots-tag: noindex HTTP header, which prevents Google from indexing the URLs.
toanvoc is unsure where to edit this setting in the Discourse source code. Jagster questioned the need to index user profiles, suggesting it wouldn’t improve SEO and that allowed URLs are already reachable to Googlebot. toanvoc is looking for a local community development model to make user profiles public and indexable.
I want Google to index the content in the link format domain/u/username, what should I do?
I have removed the Disallow: /u part to let the link format domain.com/u be indexed by Google, but what should I do if I want the slug format domain/u/username to be indexed by Google? Is there any other configuration?
I have not logged in and cannot access the slug format: domain/u/username, so I see that it is a problem for Google to index this slug format (attached image).
If I remember right that limit comes a bit deeper than robots.txt. Sure, it is quite fast test allowing /u in robots.txt. But if it comes deeper from the core you may need a plugin.
Dig here. That was a thing perhaps a year ago, came along with SEO-stuff, and there was strong reason why it is not allowed.
I know the url is Noindex but I don’t know where to edit it in the source discourse. In the robots.txt file, the Disallow: /u part has been removed. I have attached a photo.
I want to open index for this slug domain like this example: https://domain.com/u/toannguyen/activity (example link), what do I need to do in the discourse source code?
Currently, when I submit this type of link in Google Search Console, it says it is blocked from indexing. Hope to get help?
I don’t know, but why would you like to do that? It isn’t something that someone would search for. It doesn’t improve your SEO, and all allowed URLs are already reachable to Googlebot.
