Sudden drop in traffic

That’s right, all the discussions in my case were reworked by google and disappeared from the search.

I have no idea, for now on my sites I still have:
Not enough usage data in the last 90 days for this device type.

The decline continues despite there is always new content and updated discussions.
No errors are reported, it is clear that something has happened!

2 Likes

Are you on the latest version of Discourse?

3 Likes

We’ve got 2.8.0.beta11 installed now, it looks like we need to install 2.9.0.beta4.

3 Likes

.beta6 now… reminds me I gotta upgrade! :wink:

2 Likes

Never had sitemap using plugin, now that it’s native and plus various fixes on latest version for about a month noticeable change in indexing, some was close to instant, not sure it’s helping traffic but some indexing activity has to be welcome.

2 Likes

Since we did the upgrade from very early 2.x version to latest 2.9, this is how the coverage looks now, it took about 12 days for that uptick to start. “2” denotes approx when the latest version was achieved.

The historical trend was a declining indexing of valid links, but you can see the trend is reversed, i.e. it has had an effect of increasing indexing activity overall, at least.

There is approx. a net increase of 15K valid links.

While it is not totally clear in the above graphic, the invalid links went up a little post upgrade, but have since started to trend back down.

Slowly, and the pages invalid with a redirect, were the main and largest invalid group and it is now showing a slow but downward trend for the last month.

Less crawling and more is now directly sourced via the sitemap, a very clear before vs after sitemap effect:

One more before and after, breadcrumbs (and sitelinks had similar take up)

Tbh I will give it more time, at least 3 months to see how it all pans out because it was a big jump from version and going from no sitemap after a couple foyers to a site map - in the meantime I will try keep pace with the upgrade.

Some obvious improvement here too, things went weird for a bit current state looks improved overall for more vitals:


The only technical improvements that might be helping speed (on top of any programatic improvement with the PG13 DB and Discourse codebase) is that the HD would be nVME, and a very updated docker too - definitely noticing a big improvement in performance overall and speed tests look very good now using google pagespeed.

Overall the general concerns on negative trends in google console appear to be turning around with version 2.9.beta4-6, albeit slowly.

4 Likes

Also note that we recently added

which will help with this :+1:

6 Likes

It’s likely that the person clicking on the search term link on the Google page didn’t find what they wanted in the discussion on the Discourse forum we are analyzing :slight_smile:

I dunno what changed. Maybe update, but around start of October, suddenly Google is now indexing GOOD Urls for desktop.

Discovery suddenly began October 1-3 and the sudden uptick around 25th October in the curve coincides with the flip above, I dunno is this google seeing things properly or an update improving indexability.

It’s still not indexing for mobile.

This deployment is only a few commits away from totally up to date - 2.9.0.beta10 [f7a4fd1f49]

1 Like

Yes it is. Poor URLs is totally different thing than not indexing.

1 Like

This happened: Introducing Discourse Splash - A visual preloader displayed while site assets load

That’s a good point. @agemo is using “indexing” and Google’s scoring of “good/medium/bad” Web Vitals interchangeably, when they are absolutely not the same thing, even though the later may affect ranking.

3 Likes

I’ve got zero “good URLs”. Should I be concerned? haha :upside_down_face:

1 Like

Well it’s funny you should pedantically point that out, and it’s a technical point, but it’s missing the substantive overall point, cause sometimes language - this actually coincides with an uptick in indexing activity, that I didn’t post. So I’m waxing on the derivative, as it’s all derivative of the indexing service…

1 Like

Well see I don’t get that because that was around July and so it’s 3 months later before any GOOD urls were being seen by google as per above screenshots, I would have hoped sooner on this deployment as it has had the splash screen commit within days of it’s initial release.

I suspect when I removed some very simple but old custom mod for a banner (CSS/HTML) and reverted to using versatile banner component instead, then things clicked into place correctly, though it would not explain the GOOD urls appearing start of october, but it might explain the october surge.

I was aiming to reduce CLS issues, and so I am guessing the pages render and appear better. Maybe I’ve solved a deeper issue here. IIRC versatile banner is not being used on mobile views.

1 Like

Another thing that has also happened, and is why I started to look again closely in the first instance, is that non-indexed pages discovery has started to increase with a drop in index too, hmmm…all around the last week or so in October, when all these things seem to kick in at once.

I’ve looked at the non-index / 404’s, they seem to be a main source of the recent non-index rise, and are now close to 10,000, they appear as below for e.g.:

site.com/topic-name/47168?page=137
site.com/topic-name/470?page=491

If I remove the ?page=123 the url resolves correctly, so the page is there - where is google fishing this incorrect link formatting? :thinking:

I was thinking it was an error from phpBB import or some other permalink artefact/setup-incorrectly, or something else.

2 Likes

Page experience for Desktop is working fine, and search console is spitting out stats.

Page experience for Mobile nada, zilch, zero impressions.

What’s going on?

1 Like

I’m in the same situation too, it all started in september and I didn’t make any changes or install plugins.
I only updated Discourse when needed.

The reason I believe is a problem with the CLS

I also deleted all the AdSense banners and the floating Google GDPR consent banner, but after a long time nothing has changed.

1 Like

CLS has absolut nothing to do with search results.

I just can’t understand. Why are you looking at totally secondary metrics that has nothing to do with search engine results?

1 Like

Because I’m trying to understand this

But obviously I’m wrong and the two things are not related.

2 Likes

There are discussions on meta like this one Google May 4th Core Update impact on Discourse forums that talk about how Core Web Vitals impact google rankings. The primary reason for the loading splash screen Introducing Discourse Splash - A visual preloader displayed while site assets load was to work around the poor FCP measure on the initial loading of a Discourse site (thanks to the Discourse team for that update btw)

So much about search ranking is obscured but it does seem clear that Vitals like CLS are indeed important. CLS was the problem I was having above Sudden drop in traffic - #35 by scottmastro in this same topic that I have since fixed.

A barebones Discourse site does not appear to have CLS issues. You can confirm that with the meta site too https://pagespeed.web.dev/report?url=https%3A%2F%2Fmeta.discourse.org%2F This pagespeed site is useful for showing real user statistics from your site but the “Diagnose performance issues” section that does a live view of the site is not useful since it uses the crawler version of the webpage. I found this site useful when investigating my CLS issue: https://gtmetrix.com/

It also shows no CLS issues for meta.

My issue was a custom banner plugin I wrote that would appear at the top of every page and didn’t specify the height in advance, causing consistent layout shifts (but would happen so fast you wouldn’t even notice it). I solved it by just specifying the height in advance so the browser can accommodate the height before the image is fetched.

The one insight I have is that it takes a really long time for the fix to be acknowledged. It’s based on statistics gathered by real user visits. It took over a month for me. You can do the “validate fix” option on the search console and it will notify you within 28 days. It’s a bit frustrating since there’s basically a month gap between testing a fix and getting the feedback.

Anyway, not sure if any of this is helpful but I thought I’d share my experience.

6 Likes