Download remote images from older posts?

Product Support

AI-generated summary

The discussion revolves around downloading remote images from older posts in a Discourse forum. Canapin seeks a solution to download remote images from about 150,000 older posts after importing a phpbb forum.

vinothkannans provides a step-by-step solution:

  • Enable “download remote images to local” site setting
  • Set “download remote images max days old” to a suitable value (e.g., 3650 for 10 years)
  • Increase “rebake old posts count” to 300
  • Adjust the global rate limiter setting if necessary
  • Set “baked_version” of old posts to NULL in the rails console using Post.update_all(baked_version: nil)

However, Canapin questions the necessity of updating the “baked_version” if “download remote images max days old” is set to a high value. Overgrow explains that this initiates a slow rebake on all posts, but Canapin discovers that old images can be automatically downloaded even without updating the “baked_version”.

Wildweasel expresses concern about the potential performance impact on the server after applying the solution.

1

Hi.

I’ve imported a phpbb forum a while ago.
I think I didn’t set the option to download the remote images during the importation since older posts have embedded images with external links.

I would like to download all these images from older posts.

Is there a command to do this? I have about 150000 posts to be parsed.

2

Any advice here @vinothkannans or @gerhard?

3
  • Enable “download remote images to local” site setting.
  • Set “download remote images max days old” site setting’s value to 3650 (for 10 years).
  • Increase the value of “rebake old posts count” site setting to “300”. To increase it more than 300 you should adjust the global rate limiter setting “max_old_rebakes_per_15_minutes” too.
  • And set the “baked_version” of old posts to “NULL” in rails console.
    ./launcher enter app
rails c
> Post.update_all(baked_version: nil)   # or Post.created_since(N.days.ago).update_all(baked_version: nil)

Now all the remote images in old posts will get downloaded automatically in the speed of 300 posts per 15 minutes.

4

Quick question: why is it necessary to do this:

Post.update_all(baked_version: nil)

If I set download remote images max days old to 20 years?

5

That will initiate slow rebake on all posts because system will detect it needs to rebake posts to new version..

6

But I saw a post from 2010 which contained a remote image, I didn’t change anything but download remote images max days old setting, and the remote image was automatically downloaded after a few minutes.

image
image1099×108 7.98 KB

The very old post was indeed edited by system in order to download the remote image.

edit : over the night, other old images have been automatically downloaded, which is nice, but I definitively don’t undestand the update_all(baked_version: nil) part then.

7

Hi,
I applied that script but I’m if there is a major impact on performance and serveur room ?