I was looking at our forum in ahrefs which reports on “AI Citations” and saw this:
I checked in my settings and found this:
Does anyone in here have experience with this? Does throttling the bots mean they don’t index the forum at all? Has anyone removed throttling that can speak to the impact on resources and AI citations? Thanks.
No, it doesn’t mean. Throttling stops, depending on how it has been done, because robots.txt is toothless, bots to act as mini DDOS attacks.
There are three reasons why you don’t get any AI citations:
Ahrefs is one of those companies whose bot should be banned in the first place. That is one reason among many why the assumption ”Ahrefs is wrong” is one of those three.
And as far as I know, they are blocked by default in Discourse:
So yeah, the ahrefs report is probably unreliable.
Why they choose not to tell you that they weren’t able to get any data, but instead erronously report “0”.. now that’s a question for them.
You can unblock their bot, but be prepared for the occasional mini-ddos, increased pageviews and server load.
The ahrefs bot doesn’t need to be able to index our forum for this to work as it’s only reporting on what it sees on Google.
It would be very unusual if that were the case as our forum gets a couple million hits a year … which is why I’m wondering about active AI bot blocking. Or maybe the framework is otherwise technically unfriendly to the AI scrapers, like say the js rendering etc.
Can you get the bots to actually cite your forums?
If so, the problem is ahrefs. This remains my guess, the AI Citations feature is fairly new and might not work yet.
If you cannot, something else might be amiss. In that case, try a few other Discourse forums. If they do show up, then the problem isn’t the framework, but something specific to your site.
For what it’s worth, here’s what ChatGPT gives me for an example query about Meta:
No it isn’t. And that isn’t that many. I got 3,5 million a year, if your hits means search hits from Google. If it means user visits, it isn’t that much either. If it means bot visits, it is even low. Plus hits doensn’t mean any of AI would tell your forum is defenetly used as a cited source. It depends what type content you have there.
But yet, it isn’t such thing what you can ask here. Only Ahrefs knows how they are counting theirs metric.
Plus Ahrefs doesn’t see via Google how many times an AI has cited
