您可以查看具体是哪些爬虫:
/admin/reports/web_crawlers
并将它们添加到此页面:
admin/site_settings/category/all_results?filter=crawler user agents