There are discussions on meta like this one Google May 4th Core Update impact on Discourse forums that talk about how Core Web Vitals impact google rankings. The primary reason for the loading splash screen Introducing Discourse Splash - A visual preloader displayed while site assets load was to work around the poor FCP measure on the initial loading of a Discourse site (thanks to the Discourse team for that update btw)
So much about search ranking is obscured but it does seem clear that Vitals like CLS are indeed important. CLS was the problem I was having above Sudden drop in traffic - #35 by scottmastro in this same topic that I have since fixed.
A barebones Discourse site does not appear to have CLS issues. You can confirm that with the meta site too https://pagespeed.web.dev/report?url=https%3A%2F%2Fmeta.discourse.org%2F This pagespeed site is useful for showing real user statistics from your site but the “Diagnose performance issues” section that does a live view of the site is not useful since it uses the crawler version of the webpage. I found this site useful when investigating my CLS issue: https://gtmetrix.com/
It also shows no CLS issues for meta.
My issue was a custom banner plugin I wrote that would appear at the top of every page and didn’t specify the height in advance, causing consistent layout shifts (but would happen so fast you wouldn’t even notice it). I solved it by just specifying the height in advance so the browser can accommodate the height before the image is fetched.
The one insight I have is that it takes a really long time for the fix to be acknowledged. It’s based on statistics gathered by real user visits. It took over a month for me. You can do the “validate fix” option on the search console and it will notify you within 28 days. It’s a bit frustrating since there’s basically a month gap between testing a fix and getting the feedback.
Anyway, not sure if any of this is helpful but I thought I’d share my experience.