# Unidentified Crawler with High Amounts of Pageviews
**URL:** https://meta.discourse.org/t/unidentified-crawler-with-high-amounts-of-pageviews/137738
**Category:** Data & reporting
**Created:** [January 4, 2020, 8:16am UTC](https://meta.discourse.org/t/unidentified-crawler-with-high-amounts-of-pageviews/137738 "2020-01-04T08:16:55Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@anthonylavado](https://meta.discourse.org/u/anthonylavado)
#### Post date: [January 4, 2020, 8:16am UTC](https://meta.discourse.org/t/unidentified-crawler-with-high-amounts-of-pageviews/137738/1 "2020-01-04T08:16:55Z")
Hi all!
When we look at our crawler page view count, there seems to be an unidentified entry that is racking up over 500k page views in a month:
Is there any easy way to find out what this could be? It seems to average some 10,000-15,000+ hits a day.
---
### Author:  [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [January 4, 2020, 5:31pm UTC](https://meta.discourse.org/t/unidentified-crawler-with-high-amounts-of-pageviews/137738/2 "2020-01-04T17:31:31Z")
If you are on our hosting email support and we can handle it for you.
---
### Author:  [@anthonylavado](https://meta.discourse.org/u/anthonylavado)
#### Post date: [January 4, 2020, 6:05pm UTC](https://meta.discourse.org/t/unidentified-crawler-with-high-amounts-of-pageviews/137738/3 "2020-01-04T18:05:34Z")
We’re a self hosted FLOSS project, so I suppose that’s out of the question 🙂
I know I could put some more filtering on there and get our infra guy to look at more logs - I guess I was just wondering if anyone else had seen this before.
---
### Author:  [@Falco](https://meta.discourse.org/u/Falco)
#### Post date: [January 4, 2020, 8:24pm UTC](https://meta.discourse.org/t/unidentified-crawler-with-high-amounts-of-pageviews/137738/4 "2020-01-04T20:24:26Z")
Check the nginx logs (`access.log`) for requests coming with that user agent and the respective IP.
---
### Author:  [@anthonylavado](https://meta.discourse.org/u/anthonylavado)
#### Post date: [January 4, 2020, 11:20pm UTC](https://meta.discourse.org/t/unidentified-crawler-with-high-amounts-of-pageviews/137738/5 "2020-01-04T23:20:02Z")
I just met with our lead SysAdmin in person and figured it out. It’s HAProxy doing a health check every 5 seconds 😂
---
### Author:  [@Falco](https://meta.discourse.org/u/Falco)
#### Post date: [January 5, 2020, 1:42am UTC](https://meta.discourse.org/t/unidentified-crawler-with-high-amounts-of-pageviews/137738/6 "2020-01-05T01:42:00Z")
Health checks should set a proper User Agent, so this doesn’t happen again. Also, you can use the `/srv/status` route for health checking.
---
### Author:  [@anthonylavado](https://meta.discourse.org/u/anthonylavado)
#### Post date: [January 5, 2020, 1:42am UTC](https://meta.discourse.org/t/unidentified-crawler-with-high-amounts-of-pageviews/137738/7 "2020-01-05T01:42:43Z")
> “Yeah, it should - but I’m lazy.” - SysAdmin
I’ll see what I can do. Thanks!
---
### Author:  [@system](https://meta.discourse.org/u/system)
#### Post date: [February 4, 2020, 1:42am UTC](https://meta.discourse.org/t/unidentified-crawler-with-high-amounts-of-pageviews/137738/8 "2020-02-04T01:42:47Z")
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.