Sitemap.xml for Google Webmaster


(Anton) #21

What about the same answer in the Stack Overflow article, where it says this:

Granted, for really small, static, easily crawlable sites, using Sitemaps may be unnecessary from Google’s point of view once the site has been crawled and indexed. For anything else, I’d really recommend using them.

Discourse isn’t a really small, static website, is it?


(Jeff Atwood) #22

It actually is. View it with JavaScript disabled to see what I mean. That’s what is presented to crawlers.


(Michael - DiscourseHosting.com) #23

In this context, static means the content doesn’t change a lot i.e. not many pages are added.
And small means there are not a lot of pages.

So that does not cover Discourse forums although they are easily crawlable indeed.


(Sunny) #24

I have seen instances where just adding a sitemap on a simple Wordpress website, boosted traffic. Even if good to have, I would prefer to create and submit it to Google - specially during the first few months of my discourse forum being indexed. (For now until 500 pages, I may just use a free tool to create sitemap and manually submit to Google).

[Haven’t researched yet for alternative]: I noticed that a category page with lots of topics by default has a infinite scroll. I generally prefer having a pagination for SEO purpose. From my experience of getting over 100M users from Google (there are plenty of times when Google recommends something, but the solution doesn’t work in practice).


(Sam Saffron) #25

No need for a “free tool” when you have this “free tool” :wink:


#26

tried it with the latest discourse version, got 502 refused to connect error.


(Sam Saffron) #27

If Discourse Sitemap Plugin is broken for you please report on that topic so we can address it.


#28

someone has reported a similar problem in github, I’m waiting for the fix.


(Richard - DiscourseHosting.com) #29

It seems like the caching mechanism is broken in Rails 5. Looking into it later today.


(Kenny DuBose) #30

@RGJ Any findings on this?


(Vinoth Kannan) #31

It is fixed already.


(Dmitry Demenchuk) #32

If Google can’t successfully crawl your site to find a link, but is able to find it in the sitemap it gives the sitemap link no weight and will not index it!

Regarding a link you provided above a guy from Google is saying the opposite:

In addition to using Sitemaps extensively for “non-web-index” content (images, videos, News, etc.) we use information from URLs included in Sitemaps files for these main purposes:

  • Discovering new and updated content (I guess this is the obvious one, and yes, we do pick up and index otherwise unlinked URLs from there too)