# Canonical Meta Data Does Not Change Correctly in Discourse App when not loaded by a webcrawler
**URL:** https://meta.discourse.org/t/canonical-meta-data-does-not-change-correctly-in-discourse-app-when-not-loaded-by-a-webcrawler/159435
**Category:** Feature
**Created:** [31 juli 2020 om 17:00 UTC](https://meta.discourse.org/t/canonical-meta-data-does-not-change-correctly-in-discourse-app-when-not-loaded-by-a-webcrawler/159435 "2020-07-31T17:00:39Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@neounix](https://meta.discourse.org/u/neounix)
#### Post date: [31 juli 2020 om 17:00 UTC](https://meta.discourse.org/t/canonical-meta-data-does-not-change-correctly-in-discourse-app-when-not-loaded-by-a-webcrawler/159435/1 "2020-07-31T17:00:39Z")
There is a bug in the Discourse app related to updating the `` meta data element in the `` section of the Discourse DOM.
Basically, when a browser client enters the application, and the application is first loaded, the `` element will set according to that initial page load; but then when a user clicks in the app (normal user behavior), without reloading the page manually, the `` link will not update.
I have tested this bug and reproduced it on the meta site:
> Fig 1. Enter meta from the home page, the canonical link is correct, as it the title element.
> Fig 2. Visit a topic. The title element changes correctly, but the canonical link is not correct (does not update as it should).
> Fig 3. Visit another topic. The title element changes correctly, but the canonical link is not correct (does not update as it should).
> Implications for SEO
This bug could adversely effect SEO because when Google indexes the page, if Googlebot is not “hard reloading” every page, the canonical information will be incorrect for each page (as in the image sequences above).
> Reproducibility
I have reproduced this bug consistently on both the meta site and our site.
> Notes
I have seen these kind of `node.js` (SPA) lifecycle issues before with other web frameworks (not only Ember) where DOM elements are not updated, based on (Ember and other SPA framework) lifecycle hooks within the web application framework.
---
### Author:  [@Falco](https://meta.discourse.org/u/Falco)
#### Post date: [31 juli 2020 om 17:04 UTC](https://meta.discourse.org/t/canonical-meta-data-does-not-change-correctly-in-discourse-app-when-not-loaded-by-a-webcrawler/159435/2 "2020-07-31T17:04:35Z")
> [@neounix](#):
>
> This bug could adversely effect SEO because when Google indexes the page, if Googlebot is not “hard reloading” every page
This won’t ever happen as we don’t serve the SPA for Googlebot. You can set your User Agent to GoogleBot UA too see how it works.
---
### Author:  [@neounix](https://meta.discourse.org/u/neounix)
#### Post date: [31 juli 2020 om 17:16 UTC](https://meta.discourse.org/t/canonical-meta-data-does-not-change-correctly-in-discourse-app-when-not-loaded-by-a-webcrawler/159435/3 "2020-07-31T17:16:42Z")
Hi @Falco
Thanks for your reply.
Yes, it’s not a problem when the UA is set to GoogleBot (just confirmed).
I agree that may not be a problem with SEO since you do not serve the SPA to GoogleBot, but it is none-the-less a bug in the SPA.
Also, I need to think about the implications of not serving the SPA to GoogleBot.
Thanks for letting me know that “small fact”…🙂
(Note: I had no idea that “Suggested Topics” were not served to GoogleBot, but now I know … thanks for informing).
---
### Author:  [@Falco](https://meta.discourse.org/u/Falco)
#### Post date: [31 juli 2020 om 17:51 UTC](https://meta.discourse.org/t/canonical-meta-data-does-not-change-correctly-in-discourse-app-when-not-loaded-by-a-webcrawler/159435/4 "2020-07-31T17:51:08Z")
We serve a completely different document for crawlers as not every crawler can run javascript and we want discourse to be accessible for those clients too, even if they receive reduced functionality they can consume all content.
---
### Author:  [@neounix](https://meta.discourse.org/u/neounix)
#### Post date: [31 juli 2020 om 18:02 UTC](https://meta.discourse.org/t/canonical-meta-data-does-not-change-correctly-in-discourse-app-when-not-loaded-by-a-webcrawler/159435/5 "2020-07-31T18:02:57Z")
> [@Falco](#):
>
> We serve a completely different document for crawlers
Thank you so much for letting me know.
Now, I realize that some earlier discussions about SPA, “infinite scroll” and other SEO related issues were completely wrong, since the SPA is not served to GoogleBot.
This changes my approach to some custom code I wrote recently; and now I know to check using the GoogleBot UA in the console.
Thanks so much for that, @Falco ! Much appreciated.
Question:
What is the best way to add a single custom javascript file to the HTML which is rendered to GoogleBot?
Is there a “standard way” to modify the HTML served to bots?
The reason I ask is that we have some custom code which was created in a plugin I wrote (meant for bots); but I checked using the GoogleBot UA in the console (thanks again for telling me that I need to do that), and none of that custom plugin code is consumed by GoogleBot.
---
### Author:  [@neounix](https://meta.discourse.org/u/neounix)
#### Post date: [1 augustus 2020 om 05:20 UTC](https://meta.discourse.org/t/canonical-meta-data-does-not-change-correctly-in-discourse-app-when-not-loaded-by-a-webcrawler/159435/6 "2020-08-01T05:20:13Z")
In the interim, since I cannot accomplish what I want in a (handlebars-based) plugin for HTML served to crawlers, we have decided to simply strip out the canonical tags from Discourse, which is a partial solution for now until I can figure out how to modify the canonical tag with some Javascript for web crawlers.
Discourse provides a nice mechanism for these kinds of changes in the container `yml` files, so that is what I have done today.
> **[Removing Canonical Link Tags (canonical\_link\_tag) from Discourse](https://community.unix.com/t/removing-canonical-link-tags-canonical-link-tag-from-discourse/379427)**
>
> Recently I learned that Discourse serves a totally different application to crawlers, like GoogleBot for example. I learned this when writing a Discourse plugin code to alter the meta tag based on if the topic was imported from the legacy forum...
I am very grateful to Discourse meta for pointing out that the Discourse app served to (identified) crawlers is not the same as the pages served to users.
Please note that I am not recommending this “interim solution” to other Discourse sys admins. I am simply sharing what I have decided to do, at this time, and how I did it (until we come up with a more interesting solution).