Changing sidekiq job urgency

AI-generated summary

In a discussion about improving the speed of image previews in Discourse’s Topic List Previews (TLP), merefield initially believed the delay was due to the PullHotlinkedImages Sidekiq job running in a low-priority queue. They attempted to override the job’s queue to ‘critical’ in plugin.rb, but it had no effect. zogstrip clarified that PullHotlinkedImages is not responsible for immediate preview generation — it downloads external images for reliability, not speed. Instead, the topic.image_url is set almost immediately by CookedPostProcessor, triggered during post processing. merefield realized they may have misdiagnosed the delay and, after further investigation, identified the issue as a bug in the TLP plugin, which overrides CookedPostProcessor. They thanked zogstrip for helping redirect their focus, and noted a fix is pending. The discussion highlights the importance of understanding job responsibilities in Discourse’s architecture before attempting performance tweaks.

If you’ve ever used Topic List Previews, you may have noticed there’s a delay before the Topic List image preview includes the preview image.

The topic list image seems to be dependent on the sidekiq job PullHotlinkedImages which is why these plugin/themes won’t work without sidekiq running.

Recently I’ve been trying to persuade a Facebook group to move over to Discourse and whilst there are many (many!) reasons you’d do so, one of the features I like about Facebook is the almost instant image Preview you get when posting a link.

So i’m trying to increase the frequency of the PullHotlinkedImages so that TLP generates the Topic image asap so you don’t have to wait the usual 1 to 5 minutes for it to happen.

However, despite attempting to override the jobs code in my plugin.rb like so, the job still runs in the ‘low’ queue.

Any ideas?

after_initialize do

  Jobs.module_eval do

    class PullHotlinkedImages < Jobs::Base
      sidekiq_options queue: 'critical'

      def initialize

No errors in the logs, no issue starting up discourse (No errors in the console).

Am I making a newb error?

Is there a better way to override the job urgency?

PS And yes, I know I’m messing with a carefully balanced system here, but I’d still like to experiment with how far I can stress the server.

The PullHotlinkedImages isn’t about “almost instant image preview” but about “downloading images that are hosted elsewhere so that they don’t disappear”.

Ah, so is topic.image_url available sooner? Have I misunderstood how/when that is available?

The topic.image_url is set in CookedPostProcessor which is called in the ProcessPost job which is immediately triggered in PostCreator.

So if you have at least one image in your post, it should immediately (give or take a few seconds) show up at topic.image_url.

Thanks I’ll take another look at the delay I was experiencing. Perhaps I totally misread it. Cheers!

Turns out this is a “bug” in TLP (which overrides and adds some methods to CookedPostProcessor) and I believe I’ve found the culprit. Fix TBD.

Thanks @zogstrip for encouraging me to pursue that direction.