I think the value is too high. Is there something wrong? [ภาพ]

There has been a lot of churn around crawling rules lately due to Google changing their robots.txt behavior so this may be normal.

You can see who it is: /admin/reports/web_crawlers And add them here: admin/site_settings/category/all_results?filter=crawler user agents [image] How to block all crawlers but Google's feature About 1/3 of our traffic is from crawlers (about 250K last month). Is there …

[image] Stranik: /admin/reports/web_crawlers [ภาพ]

Except as your data shows, there is a rogue crawler! Block it via your site settings.

Les crawlers sont très élevés ?

Soutien

Stranik (Evgeny) Mai 13, 2020, 3:36 2

Vous pouvez voir qui c’est :

/admin/reports/web_crawlers

Et les ajouter ici :

admin/site_settings/category/all_results?filter=crawler user agents

Sujet		Réponses	Vues
How to block all crawlers but Google's Feature	1	4025	Juillet 21, 2019
Web Crawlers Data & reporting	12	1268	Juillet 31, 2023
Smarter handling of random crawler traffic Feature	2	3550	Mars 29, 2018
Crawler-mania - what happened there? Support	3	536	Février 17, 2021
Too many Crawlers, is that a problem? Data & reporting	6	2546	Juin 25, 2020

Les crawlers sont très élevés ?

Sujets connexes