# 网络爬虫

**URL:** https://meta.discourse.org/t/web-crawlers/271767
**Category:** Data & reporting
**Created:** [2023年七月16日 06:41 UTC](https://meta.discourse.org/t/web-crawlers/271767 "2023-07-16T06:41:00Z")
**Posts on this page:** 1
**Showing post:** 7

<div class="post-metadata">

### Author: ![JammyDodger](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/jammydodger/32/254611_2.png) [@JammyDodger](https://meta.discourse.org/u/JammyDodger)
#### Post date: [2023年七月17日 07:20 UTC](https://meta.discourse.org/t/web-crawlers/271767/7 "2023-07-17T07:20:51Z")

</div>

这里还有一个您可以参考的指南：

> [@Controlling Web Crawlers For a Site](https://meta.discourse.org/t/controlling-web-crawlers-for-a-site/244925):
>
> bookmark This guide explains how to manage web crawlers on your Discourse site. person_raising_hand Required user level: Administrator Web crawlers can significantly impact your site’s performance by increasing pageviews and server load. When a site notices a spike in their pageviews it’s important to check how web crawlers fit into the mix. Checking for crawler activity To see if crawlers are affecting your site, navigate to the Consolidated Pageviews report (/admin/reports/consolid…

---

_[View the full topic](https://meta.discourse.org/t/web-crawlers/271767)._