# Hoe gaan self-hosted gebruikers hier om met slechte crawlers?

**URL:** https://meta.discourse.org/t/how-self-hosted-here-are-dealing-with-bad-crawlers/361280
**Category:** Community Building
**Created:** [10 april 2025 om 16:02 UTC](https://meta.discourse.org/t/how-self-hosted-here-are-dealing-with-bad-crawlers/361280 "2025-04-10T16:02:49Z")
**Posts on this page:** 1
**Showing post:** 8

<div class="post-metadata">

### Author: ![darkpixlz](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/darkpixlz/32/549896_2.png) [@darkpixlz](https://meta.discourse.org/u/darkpixlz)
#### Post date: [21 maart 2026 om 09:20 UTC](https://meta.discourse.org/t/how-self-hosted-here-are-dealing-with-bad-crawlers/361280/8 "2026-03-21T09:20:57Z")

</div>

Cloudflare has always been nice to me and I’ve never had to pay for anti-bot services. That and their newer stuff like anti-AI is great and what keeps me a customer and shill for them I guess. Don’t want AI scrapers stealing your data? Just use one of their managed rules (given it’s entirely possible using just normal robotstxt like I do on my site)

 ![IMG_1788](https://global.discourse-cdn.com/meta/original/4X/d/e/8/de8443d01cb54c18f9d72a9690ab58c71c861301.jpeg)

.. vs a generic managed one, way better..

 ![IMG_1789](https://global.discourse-cdn.com/meta/original/4X/4/6/3/46302f36410c19e512c50acfd620697d25837ab4.jpeg)

Whether or not these startups actually listen to and respect the file is another story, but good on them for trying at least. None of my sites have issues with bots in the past and I’m still repeatedly happy with the ability to block common wordpress exploits directly there after reading my logs.

---

_[View the full topic](https://meta.discourse.org/t/how-self-hosted-here-are-dealing-with-bad-crawlers/361280)._