Greeting, So I noticed something funny between the 15th to the 16th of February 2022 in our self-hosted DigitalOcean Discourse instance. As the picture below shows, the number of anonymous users per day jumped from roughly 1 000 per day to averaging 10 000 per day. I have tried to understand the origin of this but to no avail. [image] I can not seem to match it either with the views on our content or with the stats from Google Search Console or Google analytics. We also tried checking the logs, but nothing much. [image] Does anybody have an idea what can be the cause of this? Our community: https://community.world-like-home.com/ Thanks very much

[image] efuetatem1: Does anybody have an idea what can be the cause of this? That would be near impossible if not tricky to answer in a way that it speaks to your specific site. What you can do though to start to figure it out is to look at the crawler report in your dashboard to see if it is caused by crawlers. Also, I edited your topic title to make it more descriptive :wink:

Thanks @osioke But if it was the crawler, I was hoping discourse will label it explicitly as such? Hope I am not missing something ? @Jagster Is it possible for discourse to count bots as anonymous users? [image]

[image] efuetatem1: Is it possible for discourse to count bots as anonymous users? The entity doing the request is the one who identifies itself as either a “normal” user or a bot. It’s a honor based system, with all ups and downs from that. Most bad actors in the bot ecosystem won’t identify as such and will issue requests disguised as “normal” users, and there is not much Discourse can do in those cases. If you are comfortable with the command line you long into your server and use the following to track where most requests are coming from: cd /var/discourse/shared/standalone/log/var-log/nginx/ grep " 200 " access.log | awk '{print $4}' | sort | uniq -c | sort -r

[image] efuetatem1: Is it possible for discourse to count bots as anonymous users Of course, nothing else is needed that a bot is identified itself as an user. Changing an user agent is really trivial thing — even your browser can it. And Discourse knows only those bots that is using… well, known UA :wink: Sure those can be real users too if somewhere more high traffic site is a link to you.

My guess is that the PDF uploaded there is something that got linked elsewhere and lots of people are downloading it directly? Is that PDF something that got uploaded by a bad actor and is getting lots of traffic for some reason?

Thanks, @pfaffman but there’s no problem with the pdf, I actually uploaded it myself. I was just showing the picture to indicate that there’s no correlation with the thousands of anonymous users discourse is showing. [image] Jagster: Of course, nothing else is needed that a bot is identified itself as an user. Changing an user agent is really trivial thing — even your browser can it. And Discourse knows only those bots that is using… well, known UA ok, thanks. [image] Falco: If you are comfortable with the command line you long into your server and use the following to track where most requests are coming from: Thanks @Falco The command line you provided has helped us trace the IPs responsible for the jump. For now, we are going to continue our observation before deciding if we want to block the crawlers.

[image] Falco: grep " 200 " access.log | awk '{print $4}' | sort | uniq -c | sort -r Just to note, in my case the great majority of accesses are POSTs to a message-bus endpoint. In other words, probably user’s browsers. In one case every minute and in another case much more often. "POST /message-bus/ /poll?dlp=t HTTP/1.1"

Those are most of the requests in any Discourse site indeed, but they aren’t counted as pageviews, so won’t be reflected on the “Consolidated Pageviews” graph on the dashboard, making this a bit off-topic.

Fair point. Perhaps you could add grep -v POST into your pipe?

How can I figure out why I have a big jump in anonymous pageviews

Community Building Data & reporting

Jagster (Jakke Flemming) March 8, 2022, 2:44pm 3

You are getting a lot bots.

Topic		Replies	Views
Pageviews from Anonymous Users have exploded but Google Analytics showed no traffic growth. How to find about where the increase come from? Data & reporting	23	2711	January 5, 2021
Help me understand the Anonymous Users statistics Data & reporting	8	1332	March 6, 2019
Anonymous views suddenly very high Data & reporting	48	2434	December 10, 2025
MegaIndex bot did about 4,000 pageviews on one day Community Building	40	5112	December 2, 2023
View IP address of guests / anonymous visitors? Data & reporting	13	1702	January 13, 2022

How can I figure out why I have a big jump in anonymous pageviews

Related topics