We are trying to cope with a huge batch of PullHotLinkedImages (and ProcessoPost/UpdateGravatar/CrawlLink) in our sidekiq queue after migrating our forum. And as the posts are being processed, the queue is getting even bigger! To be honest, I dont even understand why we have so many PullHotLinkedImages/etc, since our posts have almost no images attached. Could/should we just remove all these jobs? [image] Any hints?

These other jobs aren’t required but they definitely enhance the presentations of your old posts. But if you want, you can “rebake” your posts later either manually or automatically via a rake task.

This looks normal – Discourse needs some time to process these posts. You might consider increasing the number of workers in the container definition if you can dedicate more system resources for it.

would it be a problem if we just delete the queue? what would we lose not processing these old posts?

Not much. PullHotlinkedImages will only try to download external images in order to prevent link rot. But that might be fine in your case.

1.8M PullHotLinkedImages in our sidekiq backlog

Product Support

Paulo_Silveira (Paulo Silveira) December 11, 2015, 10:44pm 5

@zogstrip one last question: the other jobs are also optional right? like ProcessPost/CrawlExternalLink? and is there any way to disable these services?

Topic		Replies	Views
PullHotlinkedImages Job in Sidekiq Product Support	1	385	August 31, 2020
Changing sidekiq job urgency Development	5	1076	February 21, 2019
Sidekiq queue keeps growing Product Support	15	3435	February 9, 2020
Image lightbox stopped appearing on beta 8 update Product Support	16	1387	January 17, 2019
Discourse not downloading remote images anymore? Product Support	6	1023	July 4, 2022

1.8M PullHotLinkedImages in our sidekiq backlog

Related topics