What an amazing write-up!
I started looking at this, in my testing I got 50x improvement perf wise with:
https://github.com/discourse/discourse/pull/39398
We probably need a bunch more testing, but the upside is that a tiny bug here will just mean counts are a bit off, we have to shed load, no choice at all.