# Canonical structure for /u/\* causing many urls to be indexed
**URL:** https://meta.discourse.org/t/canonical-structure-for-u-causing-many-urls-to-be-indexed/226237
**Category:** Marketplace
**Created:** [May 5, 2022, 3:58pm UTC](https://meta.discourse.org/t/canonical-structure-for-u-causing-many-urls-to-be-indexed/226237 "2022-05-05T15:58:00Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@stance455](https://meta.discourse.org/u/stance455)
#### Post date: [May 5, 2022, 3:58pm UTC](https://meta.discourse.org/t/canonical-structure-for-u-causing-many-urls-to-be-indexed/226237/1 "2022-05-05T15:58:00Z")
Initially, I wrote this for the “Bug” category …
… but don’t let me stop you if anyone feels this should warrant a PR or commit 🤞
I’ve been studying the Discourse canonical structure pretty hard the past 2 months and overall it’s great.
Weird URLs with things like `no_definitions=true` or `/search?q=` give the correct canonical.
This all works when serving the JS version of the site to users and crawlers alike.
But it seems `/u/*` urls were overlooked - they have canonicals, but to URLs that are causing thousands of extra urls to be indexed by google.
**The job:**
I’d like all the urls after the users name to canonical to the main user profile page
So `/u/FlyNumber` is the main profile page
The following would have a canonical to the above ( instead of what happens now, which is a canonical to itself)
```plaintext
/u/FlyNumber/summary
/u/FlyNumber/activity
/u/FlyNumber/activity/topics
/u/FlyNumber/activity/replies
/u/FlyNumber/activity/likes-given
/u/FlyNumber/badges
```
---
### Author:  [@justin](https://meta.discourse.org/u/justin)
#### Post date: [May 5, 2022, 5:14pm UTC](https://meta.discourse.org/t/canonical-structure-for-u-causing-many-urls-to-be-indexed/226237/2 "2022-05-05T17:14:40Z")
Is this for your custom indexing setup discussed here?
> [@Need "window.prerenderReady" implemented on my instance \[PAID GIG\]](https://meta.discourse.org/t/need-window-prerenderready-implemented-on-my-instance-paid-gig/225232/):
>
> I’m using [prerender.io](https://prerender.io) to serve the “application/JS” version of the site to crawlers ( my instance serves crawlers the JS version via a hidden setting ) It’s working well but seems discourse might [fall under this category](https://docs.prerender.io/docs/25-empty-or-partially-rendered-pages) … but some web pages use some custom loading flows or constant polling that may trick Prerender’s logic; thus, it fails to make a decision on the ready-ness of the page. The first time prerender accesses any discourse URL, it will timeout ( 20 seconds set by prerender ) T…
If so, it might be helpful to note this so those looking at the job know what they’re getting into.
By default, `/u/` URLs are not indexed, which is set both in robots.txt and passed in the initial header request on the page.
---
### Author:  [@stance455](https://meta.discourse.org/u/stance455)
#### Post date: [May 5, 2022, 5:36pm UTC](https://meta.discourse.org/t/canonical-structure-for-u-causing-many-urls-to-be-indexed/226237/3 "2022-05-05T17:36:26Z")
> [@justin](#):
>
> Is this for your custom indexing setup discussed here?
No, as I’ve dropped using prerender - it couldn’t render the main menu , login button etc.
Google bot is getting the “app” directly
I have the discourse hidden site setting set to serve crawlers the JS version. Google seems to handle it well. (More updates on that soon.)
> [@justin](#):
>
> By default, `/u/` URLs are not indexed
Good point, `/badges` as well…
> [@justin](#):
>
> set both in robots.txt
I’m using a custom edited robots file
> [@justin](#):
>
> and passed in the initial header request on the page.
>
> 
I’m using Cloudflare workers to [alter the header](https://developers.cloudflare.com/workers/examples/alter-headers/) to `'index'`
---
### Author:  [@stance455](https://meta.discourse.org/u/stance455)
#### Post date: [May 6, 2022, 6:22pm UTC](https://meta.discourse.org/t/canonical-structure-for-u-causing-many-urls-to-be-indexed/226237/4 "2022-05-06T18:22:52Z")
> [@justin](#):
>
> By default, `/u/` URLs are not indexed, which is set both in robots.txt and passed in the initial header request on the page.
>
> 
```plaintext
/u/FlyNumber/summary
/u/FlyNumber/activity
/u/FlyNumber/activity/topics
/u/FlyNumber/activity/replies
/u/FlyNumber/activity/likes-given
/u/FlyNumber/badges
```
I’d like to also point out that removing the canonicals altogether for those urls would be a step in the right direction. (IMHO my method is better for SEO)
As stated above, it’s set to `noindex` so not sure why Discourse generates the canonical at all.
---
### Author:  [@stance455](https://meta.discourse.org/u/stance455)
#### Post date: [May 15, 2022, 5:14pm UTC](https://meta.discourse.org/t/canonical-structure-for-u-causing-many-urls-to-be-indexed/226237/5 "2022-05-15T17:14:14Z")
Perhaps someone knows a clever way of doing this with JS and cloudflare workers? This way I avoid messing with the Discourse code.
I can setup a “trigger” on ~~/u/\*/summary~~ - (I can only trigger on `/u/*`) - and execute something like this:
```plaintext
const canonical = document.querySelector('link[rel="canonical"]');
if (canonical !== null) {
canonical.href = 'NEW_HREF_GOES_HERE';
}
```
What can I do here to pass along the username to `NEW_HREF_GOES_HERE` - and the canonical would end up being /u/\* instead of /u/\*/summary.
Any help here is very much appreciated.
**Edit**
Perhaps someone can point to the relevant github page(s) - I’ll take my chances editing the code.
---
### Author:  [@system](https://meta.discourse.org/u/system)
#### Post date: [June 14, 2022, 5:14pm UTC](https://meta.discourse.org/t/canonical-structure-for-u-causing-many-urls-to-be-indexed/226237/6 "2022-06-14T17:14:21Z")
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.