# New googlebot detection?

**URL:** https://meta.discourse.org/t/new-googlebot-detection/134306
**Category:** Support
**Created:** [November 24, 2019, 12:15pm UTC](https://meta.discourse.org/t/new-googlebot-detection/134306 "2019-11-24T12:15:45Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![mcdanlj](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/mcdanlj/32/131829_2.png) [@mcdanlj](https://meta.discourse.org/u/mcdanlj)
#### Post date: [November 24, 2019, 12:15pm UTC](https://meta.discourse.org/t/new-googlebot-detection/134306/1 "2019-11-24T12:15:45Z")

</div>

We noticed a spike in anonymous (not bot) page views at forum.makerforums.info and discovered that they were associated with `crawl-$IP.googlebot.com` hostnames in the 66.249/16 netblock. I found that Google recently announced that they are going to change their user agent strings:

[https://webmasters.googleblog.com/2019/10/updating-user-agent-of-googlebot.html](https://webmasters.googleblog.com/2019/10/updating-user-agent-of-googlebot.html)

Two concerns, related:

- Reporting (minor): The reporting was just wrong, since the graph didn’t show it as “Crawlers” activity but as “Anonymous”
- Indexing (important): Does the present-the-whole-page turn-off-infinite-scrolling code work with the new user agents? (If not, we might just have been badly un-indexed…)

---

<div class="post-metadata">

### Author: ![gerhard](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/gerhard/32/119479_2.png) [@gerhard](https://meta.discourse.org/u/gerhard)
#### Post date: [November 24, 2019, 12:38pm UTC](https://meta.discourse.org/t/new-googlebot-detection/134306/2 "2019-11-24T12:38:49Z")

</div>

I just tested the crawler detection with the user agents mentioned in the blog post and everything looks fine. You can test it yourself by using a browser plugin of your choice, setting one of the user agents and visiting your site.  
The reporting uses data from the `RequestTracker` which relies on the same crawler detection, so I’m not sure why you are seeing bots counted as anonymous traffic.

---

<div class="post-metadata">

### Author: ![mcdanlj](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/mcdanlj/32/131829_2.png) [@mcdanlj](https://meta.discourse.org/u/mcdanlj)
#### Post date: [November 24, 2019, 12:53pm UTC](https://meta.discourse.org/t/new-googlebot-detection/134306/3 "2019-11-24T12:53:57Z")

</div>

Aha! While I thought I saw the new user agents, I was wrong. Bad log search. The new Chrome version strings I saw came from `google-proxy-$IP.google.com` and were real traffic going through google proxy.

I’m very sorry, I should have dug deeper. But I guess now there is a record here that the new user agent strings are OK for the next confused person who comes to search! ☺

😊

---

<div class="post-metadata">

### Author: ![gerhard](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/gerhard/32/119479_2.png) [@gerhard](https://meta.discourse.org/u/gerhard)
#### Post date: [November 24, 2019, 11:36pm UTC](https://meta.discourse.org/t/new-googlebot-detection/134306/4 "2019-11-24T23:36:02Z")

</div>


