Oneboxed image links do not update after being downloaded to local

Hi,

Suddenly remote images aren’t downloaded on my server anymore even though I didn’t change anything on my config and disk space is also good.
I tried to look in the logs but I don’t see anything suspicious.
The only thing is that I don’t see sidekiq process for the image download in the sideqik queue job.

Am I missing something here ?

Best regards

2 Likes

Is your disk full (you should get a notification on your admin dash if that’s why it’s stopped.

You could reboot and/or rebuild to see that sidekiq is running properly. See also /sidekiq

EDIT:

Oops. Sorry about that.

1 Like

As I said, disk is not full (30% of use), Sidekiq is running.
Correction: Jobs::UpdateHotlinkedRaw is running but I can’t see any error in the logs

I’ll try a rebuild to see if there’s something new

2 Likes

No problem :wink:

I tried a rebuild but there’s nothing new, remote images are still not downloaded.

1 Like

Hmm. Well. And no errors in /logs?

1 Like

Seems like the rebuild made logs error work again because now I finally see errors relative to images.

default: Failed to pull hotlinked image (https://www.2dgalleries.com/planches/800W/2017/139/marvano-les-sept-nains-3140.jpg) post: 945243
PG::UniqueViolation: ERROR:  duplicate key value violates unique constraint "index_post_hotlinked_media_on_post_id_and_url_md5"
DETAIL:  Key (post_id, md5(url::text))=(945243, af709cfc810efb23bda285d236a393a4) already exists.

But I don’t think that’s the problem here because many others images aren’t downloaded and I don’t see the errors concerning these.

EDIT: Ok so there’s something odd but might be helpful. The thumbnails are generated for the images. Has there been a change in the code recently where pasted image url just generate a thumbnail with a link to the original source ?

2 Likes

We did recently refactor this subsystem. I pinged the right engineer, hold tight.

4 Likes

Hi @Oni - please can you share an example post which isn’t working? (via PM is fine, if you prefer not to share publicly)

^^ this suggests that the images were downloaded - we can’t generate thumbnails from remote images. So it’s strange that the images are still appearing hotlinked in the post :thinking:

Here’s a link

The remote image in this post should have been downloaded. It’s been 10-15 days that this problem appeared.

You can see that this is the thumbnail that is displayed but the original source is just linked. It should be the original downloaded image.

Ah I see - looks like the image itself has been pointed to the locally-downloaded version, but the <a href link has not

Let’s see if the same thing happens with this hotlinked image:

Edit: yup, same thing

Thanks for reporting @Oni - I’ll get this fixed up.

5 Likes

I saw that there’s the new media blocking parameter in the last build but the onebox link bug still remains.

1 Like

Yep, this bug is still on my todo list :+1:

3 Likes

I don’t want to be the complaining guy here but the bug is still here a month later when all there is to do is change a var name…

2 posts were split to a new topic: Fetching images from newspaper websites is failing

This should take care of it:

Directly oneboxed images will now be rewritten as markdown images. This was the old behaviour, before my recent refactoring. As before, there is a ~10min delay before the rewrite happens.

5 Likes

It doesn’t seem to work for me.
I made a rebuild.

1 Like

Ah yes, the new logic wasn’t working when the onebox was the only hotlinked media in a post. This should take care of it: FIX: Ensure pull-hotlinked can rewrite lone oneboxes (#17354) · discourse/discourse@2d5d15b · GitHub

Here’s a hotlinked image onebox - it should be re-written as an image in the next 10 mins or so :crossed_fingers: (https://pbs.twimg.com/media/FRdPEnnX0AIy0V8?format=jpg&name=large)

Edit: yup, that seemed to work. Please can you try another update @oni, and let us know how you get on?

3 Likes

I can confirm that it is working like a charm :wink:

2 Likes