The discussion revolves around overcoming high request blocks from Imgur when downloading remote images in Discourse. piffy explains that their forum’s imported content contains many hotlinked images from Imgur, which are not being downloaded due to rate limits triggered by their server IP. Despite having the download setting enabled, the backlog persists because Imgur blocks requests from their IP. pfaffman suggests using a plugin that leverages the Imgur API to bypass rate limits, possibly by creating a job to process imgur images using the API. This approach avoids direct server requests and could circumvent the block. piffy confirms that the issue is with existing imported content, not new uploads, and agrees that an API-based plugin is the most viable solution. pfaffman concludes that the decision hinges on whether the effort to build the plugin is worth the benefit.
As I slowly realize the value that images have to seo, I’ve noticed that many of my images are still being hotlinked from Imgur. That is, my old forum didn’t have any upload function so people would use Imgur to share an image. Since moving to Discourse almost 2 years ago it seems that most of the images are still undownloaded and are being hosted by Imgur (despite having the setting to download remote images checked )
I’ve come to realize this is because the number of requests to Imgur from my server IP is too high and it gets (temporarily?) blocked. Any tips on how to work around this limit?
Maybe a watched word to just disallow imgur images and/or some othe campain to train people to just drag and drop the images straight to Discourse?
Are they displaying properly? Maybe you don’t care?
I think the thing you might do would be to have some proxy that made the imgur request from some other IP. It’s perhaps slighly shady, and likely against their terms of service.
Oh! But wait. Maybe a plugin that used the API would solve your problem.
I think a plugin could do that. I think the code in question is in lib/file_helper.rb starting with def self.download. Or maybe make another job like pull_hotlinked_images that processed only imgur files and didn’t bother with overriding anything. Maybe once your backlog got processed you’d not get hit by rate limits on new posts.
It seems improbable that it would get moved to core, but I don’t make such decisions so don’t blame me or team if I’m wrong.
Thanks for the tips! Just it clarify, my users aren’t uploading from imgur anymore, there is just a large backlog from imported posts that is not being downloaded and replaced (or is being processed at such a slow rate I can’t tell). Our previous platform didn’t allow for image uploads.
In that case, my idea of a plugin that created a job that used the imgur API to download them should do the trick. Jus thow tricky it is, I’m not quite sure.
)
