# Allow list of internal hosts for scanning
**URL:** https://meta.discourse.org/t/allow-list-of-internal-hosts-for-scanning/64442
**Category:** Feature
**Created:** [6월 13, 2017, 5:06오후 UTC](https://meta.discourse.org/t/allow-list-of-internal-hosts-for-scanning/64442 "2017-06-13T17:06:19Z")
**Posts on this page:** 7
**Page:** 1
### Author:  [@eviltrout](https://meta.discourse.org/u/eviltrout)
#### Post date: [6월 13, 2017, 5:06오후 UTC](https://meta.discourse.org/t/allow-list-of-internal-hosts-for-scanning/64442/1 "2017-06-13T17:06:19Z")
In the latest release of Discourse we added extra protection to prevent [SSRF](https://www.acunetix.com/blog/articles/server-side-request-forgery-vulnerability/) attacks. This new code ensures that links are only crawled if they are not on private networks, so if your server replies with an internal address for any host it won’t be crawled.
However, this is not always ideal, for example if you are running a couple of Discourses on a private network, they wouldn’t be able to onebox each other or crawl links to fetch topic titles and such.
To fix this, I’ve [added a new site setting](https://github.com/discourse/discourse/commit/009f0921dc4afa6b4b104e0cd72bf1850406b929) to whitelist internal hosts for link crawling and oneboxing.
Simply add your hosts to the `allowed internal hosts` site setting and they will be crawled even if they are internal.
You should be _absolutely sure_ these hosts are safe to crawl before you do this: we won’t crawl on any ports except 443 and 80, but if you are running other web services on the same host it’s possible an attacker could create a onebox or link crawling request that would hit those services and change data.
---
### Author:  [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [6월 13, 2017, 11:11오후 UTC](https://meta.discourse.org/t/allow-list-of-internal-hosts-for-scanning/64442/2 "2017-06-13T23:11:00Z")
> [@eviltrout](#):
>
> hit those services and change data.
Note that this is _only_ possible if the HTTP onebox request triggers some kind of action on GET or HEAD. That is, you have a page on your intranet like this …
`https://internalsite.example.com/delete-all-our-data`
… where simply visiting that URL (issuing a HTTP GET to it) would.. delete all your data. Correct @eviltrout?
---
### Author:  [@fefrei](https://meta.discourse.org/u/fefrei)
#### Post date: [6월 14, 2017, 9:01오전 UTC](https://meta.discourse.org/t/allow-list-of-internal-hosts-for-scanning/64442/3 "2017-06-14T09:01:01Z")
There is also a (theoretical) information disclosure risk: If `https://internalsite.example.com/show-super-secret-data` returns a page that will onebox and where the onebox contains sensitive data, this data will be leaked. (This is pretty unlikely, because most sensitive internal sites probably won’t onebox at all, or without sensitive data in the onebox itself.)
---
### Author:  [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [6월 14, 2017, 9:21오전 UTC](https://meta.discourse.org/t/allow-list-of-internal-hosts-for-scanning/64442/4 "2017-06-14T09:21:07Z")
It’s _exceedingly_ unlikely that an internal page would onebox, don’t you think?
---
### Author:  [@fefrei](https://meta.discourse.org/u/fefrei)
#### Post date: [6월 14, 2017, 9:29오전 UTC](https://meta.discourse.org/t/allow-list-of-internal-hosts-for-scanning/64442/5 "2017-06-14T09:29:03Z")
Yeah, it is. But not impossible, and I refuse to not celebrate that Discourse has taken precautions against a possible issue just because it’s not very likely 🙂
An albeit unlikely, I think it’s fair to warn sysadmins about the risk before they whitelist a host.
---
### Author:  [@eviltrout](https://meta.discourse.org/u/eviltrout)
#### Post date: [6월 14, 2017, 2:37오후 UTC](https://meta.discourse.org/t/allow-list-of-internal-hosts-for-scanning/64442/6 "2017-06-14T14:37:12Z")
> [@codinghorror](#):
>
> [https://internalsite.example.com/delete-all-our-data](https://internalsite.example.com/delete-all-our-data)
>
> … where simply visiting that URL (issuing a HTTP GET to it) would.. delete all your data. Correct @eviltrout?
Yes, onebox will _never_ POST or PUT data, so if your internal app is built properly and does CSRF protection and all that you are good. It’s more of a danger for those legacy PHP apps where GETs are mutating data.
I hope those are rare, but I could also understand a company saying “hey we know this 15-year old app sucks but it only runs on our internal network so who cares”
---
### Author:  [@ssvenn](https://meta.discourse.org/u/ssvenn)
#### Post date: [8월 9, 2018, 7:46오후 UTC](https://meta.discourse.org/t/allow-list-of-internal-hosts-for-scanning/64442/7 "2018-08-09T19:46:55Z")
This is a nice security precaution feature, but it would be great if the rails production log had some more debug text when it comes to why oneboxes are failing.. something like “host X is on a private network but not whitelisted” or “opengraph meta tags missing” and so on.
I’ve been scratching my head about why internal oneboxes didn’t work all day until I found this explanation of the whitelisting setting. It wasn’t immediately obvious to me that the internal host whitelist required _just_ the hostnames, without any http:// or url paths around it.
At least I learned something new about HTTP teapots after searching for that generic 418 code in the production.log 😆
[https://meta.discourse.org/t/oneboxing-failing-with-418/78629/2](https://meta.discourse.org/t/oneboxing-failing-with-418/78629/2)