# Prevent scraping script running on my discourse platform

**URL:** https://meta.discourse.org/t/prevent-scraping-script-running-on-my-discourse-platform/85067
**Category:** Marketplace
**Created:** [April 11, 2018, 7:58am UTC](https://meta.discourse.org/t/prevent-scraping-script-running-on-my-discourse-platform/85067 "2018-04-11T07:58:10Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![AnnaNaumova416](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/annanaumova416/32/89198_2.png) [@AnnaNaumova416](https://meta.discourse.org/u/AnnaNaumova416)
#### Post date: [April 11, 2018, 7:58am UTC](https://meta.discourse.org/t/prevent-scraping-script-running-on-my-discourse-platform/85067/1 "2018-04-11T07:58:10Z")

</div>

Hello. My name’s Anna and I want to have my own server running Discourse platform.

However, I have installed discourse platform on my server and it’s working fine.

One concern is that if the user access to the site using scraping script, for example, using PHP or Python, then I want to prevent sending content to those IP.

Idea is that if the user doesn’t use the browser then doesn’t send him the content.

I want this to be done by a plugin but I am open to your suggestion.

I am also open to any budget suggestion from you. Thanks in advance. Anna.

---

<div class="post-metadata">

### Author: ![PabloC](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/pabloc/32/49633_2.png) [@PabloC](https://meta.discourse.org/u/PabloC)
#### Post date: [April 11, 2018, 10:16am UTC](https://meta.discourse.org/t/prevent-scraping-script-running-on-my-discourse-platform/85067/2 "2018-04-11T10:16:52Z")

</div>

Its not simple to detect scrappers because toda headless browsers are used to scrape.

A simple first step would be to block all the ips you know are scrappers.

Also, how much harm are scrapers doing? What kind of content do you have?

---

<div class="post-metadata">

### Author: ![AnnaNaumova416](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/annanaumova416/32/89198_2.png) [@AnnaNaumova416](https://meta.discourse.org/u/AnnaNaumova416)
#### Post date: [April 11, 2018, 10:19am UTC](https://meta.discourse.org/t/prevent-scraping-script-running-on-my-discourse-platform/85067/3 "2018-04-11T10:19:00Z")

</div>

Hello @PabloC.

Thanks for your reply. For now, I am just opening the site so there is no content at all~

But I just want to detect if the user is using a browser or not.

If he is not using a browser then block sending content at the server side. Is it possible?

---

<div class="post-metadata">

### Author: ![mbcahyono](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/mbcahyono/32/97721_2.png) [@mbcahyono](https://meta.discourse.org/u/mbcahyono)
#### Post date: [April 11, 2018, 10:44am UTC](https://meta.discourse.org/t/prevent-scraping-script-running-on-my-discourse-platform/85067/4 "2018-04-11T10:44:01Z")

</div>

Something like [Cloudflare’s browser integrity check?](https://support.cloudflare.com/hc/en-us/articles/200170086-What-does-the-Browser-Integrity-Check-do-)

---

<div class="post-metadata">

### Author: ![AnnaNaumova416](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/annanaumova416/32/89198_2.png) [@AnnaNaumova416](https://meta.discourse.org/u/AnnaNaumova416)
#### Post date: [April 11, 2018, 10:45am UTC](https://meta.discourse.org/t/prevent-scraping-script-running-on-my-discourse-platform/85067/5 "2018-04-11T10:45:35Z")

</div>

Thanks, @mbcahyono for your reply.

Sure very similar to that.

Is there anyway to block content to non-browser users?

---

<div class="post-metadata">

### Author: ![fefrei](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/fefrei/32/119538_2.png) [@fefrei](https://meta.discourse.org/u/fefrei)
#### Post date: [April 11, 2018, 12:53pm UTC](https://meta.discourse.org/t/prevent-scraping-script-running-on-my-discourse-platform/85067/6 "2018-04-11T12:53:41Z")

</div>

Currently, there isn’t. Something like this would have to be a plugin, and wouldn’t be effective – as @PabloC mentioned, modern scrapers _are_ browsers.

---

<div class="post-metadata">

### Author: ![AnnaNaumova416](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/annanaumova416/32/89198_2.png) [@AnnaNaumova416](https://meta.discourse.org/u/AnnaNaumova416)
#### Post date: [April 11, 2018, 12:56pm UTC](https://meta.discourse.org/t/prevent-scraping-script-running-on-my-discourse-platform/85067/7 "2018-04-11T12:56:05Z")

</div>

Thanks, @fefrei for your reply.

I am going to give up making that kind of plugin.

Because there is not much benefit having that, even though I can have one!

I have posted many posts to get help, but I finally give up… However thanks.

---

<div class="post-metadata">

### Author: ![system](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/system/32/443519_2.png) [@system](https://meta.discourse.org/u/system)
#### Post date: [May 11, 2018, 12:56pm UTC](https://meta.discourse.org/t/prevent-scraping-script-running-on-my-discourse-platform/85067/8 "2018-05-11T12:56:06Z")

</div>

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.
