This is a perennial problem faced by all discourse sites at one time or another depending on what sites they are attempting to crawl.
From their help:
I quote: “ We allow hotlinking on forums, but these images cannot be used as content for a website, including blog posts, avatars, site elements, and advertising. You can read our terms of service for more info. If you believe your site has been mistakenly blocked, please contact us for assistance.”
It might be they are not classifying Discourse sites correctly.
So options:
-
Try contacting them to see if hey are blocking you for some reason. You may be able to remove any specific IP block. Possibly try getting the Discourse User Agent to be whitelisted as it is a forum system. They might not be recognising it as such at the moment. You’d be doing the community a favour?
-
Another option might be to move IP address, but if your site is a heavy user you may end up back in the same place as they start blocking that address after so many links.
-
A final option is to bite the bullet and use the proxy crawl service whilst monetising the site further to get your users to pay for it.