We are trying to cope with a huge batch of PullHotLinkedImages (and ProcessoPost/UpdateGravatar/CrawlLink) in our sidekiq queue after migrating our forum. And as the posts are being processed, the queue is getting even bigger!
To be honest, I dont even understand why we have so many PullHotLinkedImages/etc, since our posts have almost no images attached. Could/should we just remove all these jobs?
Any hints?
This looks normal – Discourse needs some time to process these posts. You might consider increasing the number of workers in the container definition if you can dedicate more system resources for it.
would it be a problem if we just delete the queue? what would we lose not processing these old posts?
Not much. PullHotlinkedImages will only try to download external images in order to prevent link rot. But that might be fine in your case.
@zogstrip one last question: the other jobs are also optional right? like ProcessPost/CrawlExternalLink? and is there any way to disable these services?
These other jobs aren’t required but they definitely enhance the presentations of your old posts. But if you want, you can “rebake” your posts later either manually or automatically via a rake task.
