The Sitemap plugin is available on our Business and Enterprise plans as well.
Probably nothing. For some reason Google seems to give the “How do I do X” a higher priority than the actual topic with that exact title. Why? I’m not sure. It might just be an AI making that decision based on unknown patterns.
Well that’s a big issue then for us. We’ll look into this, but it sorta defeats the whole purpose then, coz we can’t expect people to “go through” every topic in the community. In practice, most people will google for their issue (even if they’re a member of the community) to arrive at the answer.
It’s really hard to tell what’s going on with Google in your case. I took the liberty of taking a closer look at your crawler stats and on first glance it looks like the Google crawler isn’t visiting your community very often. Did you already try to gain insights from the Google Search Console? Maybe there’s some information there that could help in getting to the bottom of this.
I’m seeing the same behavior for a small portion of recently created topics here on Meta. I’ll discuss this with our team in order to find out if there’s something we can do or if it’s simply Google being Google. I’ll keep you updated.
This seems to be an issue then. Coz we definitely can’t continue if google doesn’t index individual posts coz it sorta defeats the entire aim of having the community (since most folks will discover content from google - including our current users, instead of going through the categories on discourse).
And this doesn’t seem to be an issue just on our discourse, but even this forum (meta.discourse.org).
To showcase this, I googled the title of this post, and this is the result I got:
Basically for folks who end up using the discourse community as a knowledge base (which ofcourse many companies do), this becomes a big issue.
We’ve followed pretty much most of the guidelines given in the 2 blogs on SEO that were shared above, our content is rich, detailed, highly technical. Yet when people google for it, it doesn’t show up.
So genuine questions guys: Given our use case (which is creating a knowledgebase via support queries, which can also help in SEO), and seeing the issue at hand, would you recommend that we just start planning on finding an alternative?
A shot in the dark: Googles preference for one sub-domain over the orther might be influenced by “Core Web Vitals” Largest Contentful Paint (LCP)
The simulated report on https://pagespeed.web.dev/ is “bugged” by Discourse. Better check the Google Search Console report “Experinece” → “Core Web Vitals” for your domain.
I am not sure this is related to sitemaps or cloud hosting. Meta is hosted on AWS which is a completely different place to where we host many of our other customers and we started seeing very uneven results for meta lately and quite a few sites across various hosting options.
I have been trying to tune a few things to see if anything helps.
We no longer follow links to .rss which saves google from scanning /1/2 etc variants of a topic that all share a canonical.
We explicitly tell Google not to follow links inside the .rss feed in case it gets an rss feed.
Because Meta, meaning OPs here, doesn’t use ”why” as Google wants? I did some expriences and even the context stays same, using different sentences changes search results. As expected.
And… personal search history is big and not so good thing. My results are quite often pointing somewhere else than to Meta, because I’m doing searches here, not via Google.
Yes. Still Discourse may have some issues with Google, and that may or may not come from ”forum status” where Google isn’t reacting to forums same way as to ordinary website. Or there is some technically issues — even that is the reason quite rarely.
It’s the same thing I did, after an initial period of improvement then it returns to being ignored.
After two months now even the new threads in the sitemap are not being indexed, it is a situation that has been going on for months and we have lost a lot of views.
We want to try this for one month and if it doesn’t improve we will change software