Consolidated Pageviews: exclude uptime crawler from report without blocking?

So I’ve set up an instance of Uptime Kuma to monitor my forum, and now, of course, the crawler graph has skyrocketed. I know I can toggle crawlers entirely from the display by clicking “Crawlers” in the graph key, but I’d like to just exclude the “Uptime-Kuma” user agent behind the scenes and still watch for other spikes.

(@pfaffmanyou’ve brought this up before – did you happen to come up with anything?)

You can hit /srv/status and that doesn’t get counted. You might also be able to hit something like /t/-/123.json and then look for title or something like that. I think that won’t count as a pageview.

1 Like

Thank you Jay! I’ve added that to the tracking URL, and it appears to have stopped adding to the crawler count.

I forgot to mention that prior to this I’ve had Uptime Robot in place for a long time, just tracking the base URL.

Interestingly, YTD Discourse has reported just 1 hit from Mozilla/5.0+(compatible; UptimeRobot/2.0; http://www.uptimerobot.com/).

I wonder if some uptime crawlers are automatically excluded from reporting…? :thinking:

1 Like

Glad to hear it.

There are some cases where /srv/status can say ‘ok’ when it’s not so OK. I think there’s a communiteq plugin that catches some more cases, but I couldn’t find it in 10 seconds.

I don’t see UptimeRobot in the source. And I’m pretty sure that I remember UptimeRobot hits counting at some time in the past, but I’ve been using /srv/status for a long time.

1 Like

I forgot about that !!! And I had to look for it for over 10 minutes as well.

It’s here GitHub - communiteq/discourse-betterstatus and TBH I have no idea if it still works. If it does, it will perform a very rudimentary check to see if Redis and Postgres are working.

1 Like