How to use prevent overlapping jobs? DistributedMutex?

AI-generated summary

In a discussion about preventing overlapping jobs in Discourse plugins, Mark_Schmucker reports that a long-running job (approx. 1 hour) is being triggered every 30 minutes via Sidekiq, causing duplicates. They attempted to use DistributedMutex.synchronize("custom_digest", validity: 180.minutes) to lock the job, but it didn’t prevent concurrent execution. pfaffman suggests reviewing the summary email job, which reportedly handles similar concurrency checks. Mark_Schmucker notes their implementation mirrors recent core changes (e.g., review.discourse.org/t/feature-allow-post-process-mutex-to-be-held-longer/5017 and review.discourse.org/t/fix-post-and-topic-creation-race-condition/5247), differing only in syntax (curly braces vs do/end), which they believe is irrelevant. Falco clarifies that placing the mutex inside the execute block only causes jobs to wait — not prevent them — and recommends checking for an active job and returning early if one exists. The core issue remains unresolved: ensuring only one instance runs at a time, regardless of duration.

Plugin newbie here. I’m trying to adapt a plugin which takes about an hour to run. Looking at /sidekiq, I see the job is running every 30 minutes. So before the first job is finished, another instance of the same job starts, creating duplicate results. How can I prevent this?

One option of course is to make the job run in its allowed 30 minutes, but there are other constraints, and I would rather let it run as long as it wants.

I tried this, but it doesn’t seem to prevent a second instance of the job from running:

DistributedMutex.synchronize("custom_digest", validity: 180.minutes)

I think that the summary email job does such a test that was added early this year. You might have a look at that.

I think I am doing the same as https://review.discourse.org/t/feature-allow-post-process-mutex-to-be-held-longer/5017 and https://review.discourse.org/t/fix-post-and-topic-creation-race-condition/5247, with the exception of using curly braces instead of do/end, but I don’t think that matters.

DistributedMutex.synchronize(“custom_digest”, validity: 180.minutes) {
do_stuff
}

Yet do_stuff is running multiple times concurrently, and well within the 180 minutes.

If the mutex is inside the execute block it just means it will block there waiting for it, meaning that you will see two jobs running, one actually running and another waiting for the mutex.

Maybe you want to check for it and return early if there is another instance running? Hard to guess knowing so little about the exact use case.