Is there a way to prevent web scraping

We are sharing study material on our forum (we solve the question and public detailed answer) and someone is trying to scraping our forum content to publish on theirs website. Is there a way to prevent this behavior?

If the website is public, there is no way to stop them . Because there will always be workarounds.

It is up to the operator of the website to protect the areas that should not be generally accessible. I would work with the user rights & groups for Discourse. However, this requires a well thought-out site structure.

You can also block the IP address of unwanted visitors. But no matter what you intend to do, it will only be a emergency solution.

If you don’t want to share your knowledge, you only have three options:

  1. closed community / by invitation only
  2. take legal action against the person(s).
  3. offer paid content and block areas.
3 Likes

You could also modify category permissions to require TL0 and above to view, which will disallow anonymous/crawlers to see those categories and associated topics.

5 Likes