# هل يفشل تحميل الصور البعيدة محليًا بصمت؟

**URL:** <https://meta.discourse.org/t/download-remote-images-to-local-silently-failing/105473>\
**Category:** Self-hosting\
**Created:** [2 يناير 2019، 1:52م UTC](https://meta.discourse.org/t/download-remote-images-to-local-silently-failing/105473 "2019-01-02T13:52:46Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![marcozambi](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/marcozambi/32/119505_2.png) [@marcozambi](https://meta.discourse.org/u/marcozambi)\
**Post date:** [2 يناير 2019، 1:52م UTC](https://meta.discourse.org/t/download-remote-images-to-local-silently-failing/105473/1 "2019-01-02T13:52:46Z")

</div>

Hi everybody!  
Back asking support about the download of remote images after this

> [@Rebaking posts does not fix broken locally downloaded linked images](https://meta.discourse.org/t/rebaking-posts-does-not-fix-broken-locally-downloaded-linked-images/98470):
>
> Not entirely sure this is a bug, still… My situation is the following: I imported an old SMF2 website in Discourse. Some of the posts had a lot of images linked, and in order to avoid loosing them in the future if links are broken, I enabled the “download remote images to local” feature in Discourse settings. A few downloads of such images failed, as the NASA server were not available at the moment of the image retrieval, leaving the baked version of that post with broken links to images (i.e…

Let me outline my problem first. I have to make sure that images linked in about 1400 posts in my foru are correctly downloaded and imported in Discourse. This is because the website where these images are hosted is going to close soon and I don’t want, obviously, to loose the embedded images in my posts.  
The website that hosts the images is owned by me and I can read the Apache logs to spot whether the images are really getting pulled from the server or not by Discourse (spoiler, they are, by process Redis).

Now, the feature of downloading remote images normally **works** , but somehow the download of this specific subset of images, all coming from the same website, never worked and still somehow fails silently whenever I try to rebake the posts (more below).

No errors are generated in the Discourse logs and no record for “broken\_images” is created in table post\_custom\_fields.

Before writing I also searched around in meta without finding any particular hint I could use to understand what goes wrong. I found very useful though this post from @vinothkannans, and went carefully through his checklist.

> [@What are the limitations of download remote images](https://meta.discourse.org/t/what-are-the-limitations-of-download-remote-images/64356/2):
>
> If Download remote images to local SiteSetting enabled whenever a post created then all the external images will be downloaded locally (In all 3 above cases). But the downloading process will start only after editing\_grace\_period SiteSetting (default 10 minutes) from the creation of post. Also it will not be triggered for existing old posts when you enable the Download remote images to local SiteSetting. For that you have to recook all old posts. Also downloading remote images will not work in …

So to answer the questions:

1. SiteSetting `download_remote_images_to_local` is disabled. [**ENABLED**]
2. Enough disk space not available to download. (In this case you will receive notification about the problem). Also look at the `download remote images threshold` SiteSetting. [**I have 50% of free disk space left**]
3. If the post date is before `download_remote_images_max_days_old` SiteSetting. [**Set to 7200 days, while the images are from posts between 2015 and 2018 - Imported from SMF2**]
4. If the image is from one of the domains of `disabled image download domains` SiteSetting. [**No disabled domains**]

Here are my settings.

 ![image](https://global.discourse-cdn.com/meta/original/3X/0/e/0ed2fb57b5aeace9b9efabf9610c501bc17485ec.png)

In order to trigger the download of the pictures I launch the rake task

```plaintext
rake posts:rebake_match['attachments_tapatalk','string',10]

```

where “attachments\_tapatalk” is part of the URL of all the images links in the posts.

Once I launch the rebake (with 10 seconds of extra waiting time between posts to not overload the image hosting server) I can see that Redis is requesting the images to the hosting server, but then nothing gets updated in the Discourse posts, i.e. no image seems to get imported as Discourse attachment.

In Sidekiq I can see the number of “Processed” tasks increasing by the expected amount (about 1400).

This is what I see in the logs of the server hosting the images

 ![image](https://global.discourse-cdn.com/meta/original/3X/9/0/902c609a4a3b7685872963e931e9ead37e48f8e5.png)

The second and third images in this topic are, for example, coming from that server and do work. They do work (i.e. they are reachable and available) even linking them here!

> **[Atlas V - SBIRS GEO Flight 4](https://www.forumastronautico.it/t/atlas-v-sbirs-geo-flight-4/24110/15)**
>
> Prime foto del lancio by ULA.

```plaintext
http://astronautica.eu/attachments_tapatalk/20180120/2016f256f61b2995a9f73b1f99ddc32b.jpg
http://astronautica.eu/attachments_tapatalk/20180120/d5ab809851818bd2077daab3989b2ab3.jpg

```

So, I have these unsolved questions:

- why I can see the linked images with no problems in the posts body, but the local download fails?
- where shall I look for more information about what the PullHotlinkedImages/PostsProcess jobs do and what is the exit status/logging for each of them?
- is it possible to manually launch a PullHotlinkedImages process somehow, to trigger/force the download of the linked images in a specific topic?

Thanks in advance!

---

<div class="post-metadata">

**Author:** ![vinothkannans](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/vinothkannans/32/86465_2.png) [@vinothkannans](https://meta.discourse.org/u/vinothkannans)\
**Post date:** [2 يناير 2019، 2:45م UTC](https://meta.discourse.org/t/download-remote-images-to-local-silently-failing/105473/2 "2019-01-02T14:45:30Z")

</div>

> [@marcozambi](#):
>
> is it possible to manually launch a PullHotlinkedImages process somehow, to trigger/force the download of the linked images in a specific topic?

Both “rebake” and “Rebuild HTML” actions will trigger it. To debug and get more details about this issue I’ll recommend you to do it for a single affected post at once.

> [@marcozambi](#):
>
> where shall I look for more information about what the PullHotlinkedImages/PostsProcess jobs do and what is the exit status/logging for each of them?

Code. [pull\_hotlinked\_images.rb](https://github.com/discourse/discourse/blob/master/app/jobs/regular/pull_hotlinked_images.rb) and [process\_post.rb](https://github.com/discourse/discourse/blob/master/app/jobs/regular/process_post.rb)

> [@marcozambi](#):
>
> why I can see the linked images with no problems in the posts body, but the local download fails?

Could you reproduce this issue on try?

---

<div class="post-metadata">

**Author:** ![marcozambi](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/marcozambi/32/119505_2.png) [@marcozambi](https://meta.discourse.org/u/marcozambi)\
**Post date:** [2 يناير 2019، 3:08م UTC](https://meta.discourse.org/t/download-remote-images-to-local-silently-failing/105473/3 "2019-01-02T15:08:31Z")

</div>

> [@vinothkannans](#):
>
> Could you reproduce this issue on try?

Well, yes, meaning that I can restart the rebake\_match at will, ad it always… fails to download the attachments. Do you have some useful test/log file inspection to suggest that I can perform?

One thing I noticed is that seemingly there is no PullHotlinkedImages process visible in Sidekiq, just the number of “Processed” tasks increasing by 1 every ten seconds (as expected, since I pass the waiting time parameter when calling the rebake\_match task)

---

<div class="post-metadata">

**Author:** ![system](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/system/32/443519_2.png) [@system](https://meta.discourse.org/u/system)\
**Post date:** [16 يناير 2022، 8:07ص UTC](https://meta.discourse.org/t/download-remote-images-to-local-silently-failing/105473/5 "2022-01-16T08:07:16Z")

</div>


