I’m an admin on a free instance for open source kindly provided by the lovely people at Discourse.
I see part of being a good person is not abusing this kindness by using resources beyond the conditions of the free service.
Sadly, spiders have been taking the majority of our allocation of views and only seem to be getting worse. (Does anyone else suffer from this?)
Can anyone make any suggestions to help mitigate this issue?
Best regards
Stephen
You can ban most of bots. It helps. But Discourse doesn’t care when a WordPress would be on its knees.
How to catch humans… well, tell to all of us when you figure out it.
Thanks for the appreciation; with the increase in scraping going on this has been a concern for many of our customers.
You can check your bot report and block the most egregious crawlers from your site, that’d be a good start.
We’re looking at developing better tools for managing this, but for the next while unless it becomes a DRASTIC problem, don’t worry about it. 
The only issue with that is UI. Adding those Discourse way is kind of pain somewhere, because there is quite many of those useless SEO-bots and other content stealers. If we had some way to send list or something 
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.