Google May 4th Core Update impact on Discourse forums

It is certainly possible to make this change (all anonymous get HTML view) but it would heavily impact usability for anonymous users. Yes they would see content faster but an enormous amount of features that work for anon would not work plus the site would not “look right” for anonymous.

We could possibly make this some sort of site setting so you could experiment, but stuff like “infinite loading” would stop working for anonymous, there are some very high costs here. We would also need to invest at least some engineering in giving a /login route a bypass so people could actually register or login.

9 Likes

Would it maybe be possible to serve a HTML view as the FIRST page seen by anonymous users coming in, but then do it with all the features if they continue to browse ? That would seem to be a good solution (I don’t know if it would be acceptable to search engines, though)

That does sound less than ideal. Is there any way to serve up the static copy and then “saturate” it with the dynamic parts? That’s probably a large architecture overhaul, so maybe not in the cards. Principally, we’re looking at 49k LCP errors on our site starting in May and search traffic hit at the same time. Our current LCP score averaging 5.3s. I’m looking for ideas on how to bring that number down.

Maybe adding or removing some plugins? Increasing or reducing the number of categories? Putting static assets on a CDN? We tried to get Cloudflare working last winter without success, but we could try that again. I don’t know the Discourse architecture very well, so I’m looking for leads.

3 Likes

That’s exactly what we found, we are experimenting with removing every plugin and even ads (now our websites have no ads, optimized images…etc) and we managed to bring down the LCP but barely to the yellow zone now it’s not an error but more of a warning which still affects our website, we did notice a slight increase since then but need more time to confirm this…

tbh I am very tempted to start an open-source Nuxt+Vuejs clone of Discourse or a wrapper on top of it, it seems like the only reasonable choice at the moment!

2 Likes

Yeah there’s no way to reduce that initial load without some significant engineering, because you’re downloading the entire Discourse app.

Some extra salt on the wound is that JS performance on Android is generally worse than iPhones… and apparently Google only counts Android devices in their “real world” mobile metrics. On Meta iOS accounts for roughly 40% of mobile traffic.

12 Likes

All I can say here is that we are aware of the slow FCP and LCP and have long term plans to improve them.

Specifically @eviltrout is upgrading us to ember CLI. Once that is done we can start thinking about and possibly experimenting with code splitting and other tricks.

There are no easy tricks here, we use a CDN, we are very deliberate about how we load stuff we spent countless hours optimising here but fundamentally we use JavaScript to render our pages and delivering/parsing/running the JavaScript takes time on the first load.

26 Likes

Sorry for bringing back up this kind of old topic again, but now have some more data after conducting some testing for the last couple of months…

So here are the two websites I’ve been testing before, one migrated from Discourse (EmberJs) to a Vue with Nuxt custom built front-end website.

While the second is Discourse, with removing ads, custom fonts and absolutely anything that can be removed to make it as lightweight as possible (which worked to get the LCP errors from Error level to Warning).

1. Discourse Forum (removed custom Fonts, Ads, Plugins…etc)

As you can see on May after the update we lost 50% of keywords rankings, on October we started the changes and that helped get us a little spike for a short while but then went down! just like if there is some resistance (in other words, there is a penalty from Google)


As you can see on the image above, The changes I made (removing all extras) helped get the URLs to be moved from Poor URLs, to URLs that need improvement but even that didn’t help!

2. Custom Vue/Nuxt front-end with Discourse backend

On this website as noticed over a month ago, it’s been growing back to pre–May 4th highs.

Conclusion:

YES, GOOGLE CARES ABOUT LCP!

Hope it’s taken more seriously by the Discourse team now, perhaps moving away from Ember might be worth it, I had to do that myself on a large project, yes it was so expensive to move but it was so worth it.

11 Likes

I also agree that the LCP is a related penalty. I’ve been watching this thread for a long time. There is no concrete recommendation yet on this issue.

3 Likes

Thanks for reporting back! I think this still applies:

The ember CLI upgrade is still underway and making progress, but If you’re waiting on us to move away from ember entirely you might want to consider another platform and maybe check back on where we are with LCP a year from now.

14 Likes

Well I’m not sure it will be worth it for Discoure to upgrade to Ember CLI but who knows as we’ve had the same experience on another project and we had to dump it entirely. Upgrading to Ember CLI takes almost as much effort as upgrading to Vue or something else.

Regardless, My research was just to point out the issue and to come up with a conclusion as in the early days, almost everyone was debunking the fact that LCP has anything to do with ranking.

6 Likes

We’re probably about 90% done with that, which was on our long term roadmap anyway, since it brings a ton of developer convenience and keeps us up to date with Ember. @eviltrout can advise with specifics as he is in charge of the effort.

Yes, but this does not mean that every site will now switch to static HTML rendering so they can dominate the web with their magic SEO fast page load superpowers. Turns out, the actual content on the page happens to be rather important to the ranking as well :wink:

You can look up the history of Google AMP to see where this kind of exaggerated focus on one metric can lead to quite a bit of trauma and misguided engineering work.

15 Likes

Well that’s exactly what I’m trying to debunk on my post, Google has so much content already of decent quality, so if they had to decide based on user satisfaction I believe LCP is the minimal thing they would base their judgment on. After all Google has been warning about it few months before the update.

To be honest, I’ve had so much experience with Ember CLI and it’s as bad as before. plus the upgrade effort not sure if it will be worth it. But we will see how that goes hope @eviltrout has any input on whether they’ve seen any speed improvements.

Sadly from my research above! Google in fact does focus on user experience and LCP a lot, we’ve tried every single other thing. And, as you can see from the second website we did absolutely nothing other than getting rid of the LCP error and that put us on track to get back all our rankings (in fact as of now we already just did).

Hope that helps,

5 Likes

Can you be specific about what it is that you don’t like about Ember CLI? Provide examples.

3 Likes

We started using Ember CLI in our startup and one of the reasons was we saw that it was being used in discourse (that brought it to our attention) we tested it, it was easy to start with and easy to work with. but it was so bloated (aside from other reasons).

Ember CLI has introduced a recent update that would make any app written in versions pre-3 to be rewritten that’s when we decided to get rid of it entirely.

Yes Ember CLI has lazy loading but it’s not efficient at all (at least during the testing we’ve done), and most of the libraries out there written for Ember CLI were either outdated or so buggy that we had to write most of the things ourselves, or clone old repos and maintain them ourselves.

Ember CLI or not always has a bad render time as well (which wouldn’t help with LCP issue we discussing here).

Other, than that the way Ember works make it easy to end-up with a bloated app.

I wish I still had the old analytics we were doing before deciding to jump-ships, we just recently finished migrating from Ember to Vue a couple of months ago and I can’t be any happier with the performance of our apps and the speed of development.

PS. I haven’t had the chance to check Discourse repo, but upgrading to Ember CLI could bring more trouble as you will have to upgrade again to Ember Octane (which isn’t even stable yet) and uses totally different syntax…etc it’s a mess least to say tbh. I’m not sure if the arguments that were used before to pick Ember are still viable at the moment @Jeff.

Hope that makes sense.

4 Likes

What does “taking seriously” mean? Torching our entire ecosystem and starting from scratch?

Discourse is a growing project, we are very much aware of this issue and are considering mitigations such as fastbood, more aggressive code splitting and so on. All of this is pending our ember cli upgrade.

I am curious to see this alternative front end? Can you PM me a link. Fundamentally doing HTML only that is not customizable is trivial, we ship an html only view, you can see the LCP on samsaffron.com is very good, that is simply a discourse plugin that renders HTML.

7 Likes

Dear Yassine,

In general, I agree with you about LCP and Google SEO and greatly appreciate your analysis and insight.

Could you explain to me why if Google is using LCP to the extent you are promoting, why are two topics I wrote on our Discourse forum, which has very poor LCP according to Google, ranks both #1 and #2 out of 3,580,000 entries?

See:

If seems to me if the LCP issue with the Discourse SPA was as serious as you are advocating, and I’m not being adversarial in the least, only curious based on your expertise; why is a slow site like ours which does not use any CDN and has very poor LCP able to claim the top two spots in a topic posted only 11 and 13 days out, where these two topics are #1 and #2 out of nearly 3.5 million other posts?

I’m honestly curious how, if Google’s LCP is so influential as you are presenting, how our very poor LCP performing site ends up with such great SERP results.

Thanks!

4 Likes

According to your example, the answer seems pretty obvious: You’ve searched for quite specific terms where there isn’t really any competition with a better LCP. Being the “best” is easy when you’re the only one. As per the above posts, content is still the most important factor, but when there is a lot of content available for your search, the other factors are becoming important. You may even prove his point rather than the opposite.

I know this has already been mentioned above, but couldn’t a fast, static HTML only version of the forum be generated, and this used to give to search engines? (forbid them to crawl the actual forum where registered people navigate and post).

You say there is a plugin to generate a static view? Is this available for everybody to use?

2 Likes

This seems to still be a “guess” and is not yet proven to be a fact, isn’t this correct?

According to Google and others LCP is not yet a factor used in rankings and will not be used as a ranking signal until May 2021, isn’t that right?

Screen Shot 2020-11-23 at 5.48.14 PM

It does seem a bit overboard to me to be pushing the Discourse team to make major changes to their eco-system based on analysis and graphs from a very small number of people who claim LCP is effecting SEO now, when Google claims this signal is not yet active.

Is the LCP signal active or not?

Google says that LCP is not yet being used as a signal for SEO.

Just to let you know, I am definitely not an EmberJS fan and I agree that LCP is important. I am simply seeking facts based on proof and hard evidence.

My only “point” is that when I read this thread, it seems that folks are really pushing hard on Discourse meta to make major structural changes based on something that, according to Google and other SEO experts, is not yet being used as an SEO signal.

You guys are saying that Google is not being honest with the public?

FYI, it is highly unlikely that Google, a publicly traded company, would deceive the public. This would open up Google to a huge potential financial liability.

5 Likes

Fair enough. I myself don’t know much about LCP. I admit that. I was only basing myself on what is said on this topic, and you’re right, I don’t know if it’s accurate at all (except the evidences presented here). So, please read my post, as “if the LCP thing is correct”.

2 Likes

Your conclusion (which I think is that LCP is not used by Google to determine search rankings) may well be right but you don’t don’t get to it by the route you set out.

  • It’s such a unique search term that Google made misspelling suggestions. There isn’t much to choose from.

  • You would need to carry out a lot of searches to conclude anything. If I search for “+discourse +gon” your site doesn’t appear at all and the top result is The Discourse Encouragement Fund

  • Also I think Google personalises search results. The site you likely frequent the most appeared at the top for you but might not for others. For me, the top result is plugin - Discourse Meta. I use DuckDuckGo normally so maybe this result is not personalised at all.

None of this says or proves anything about LCP. This has been an interesting topic and I look forward to it continuing. Personally I’m happy with how fast Discourse is.

3 Likes