In the latest release of Discourse we added extra protection to prevent SSRF attacks. This new code ensures that links are only crawled if they are not on private networks, so if your server replies with an internal address for any host it won’t be crawled.
However, this is not always ideal, for example if you are running a couple of Discourses on a private network, they wouldn’t be able to onebox each other or crawl links to fetch topic titles and such.
To fix this, I’ve added a new site setting to whitelist internal hosts for link crawling and oneboxing.
Simply add your hosts to the
allowed internal hosts site setting and they will be crawled even if they are internal.
You should be absolutely sure these hosts are safe to crawl before you do this: we won’t crawl on any ports except 443 and 80, but if you are running other web services on the same host it’s possible an attacker could create a onebox or link crawling request that would hit those services and change data.