Multisite scheduled backup is only running for one site

AI-generated summary

In a Discourse multisite setup, user brahn reported that scheduled backups were only running for one of three sites, despite manual backups working fine. The issue appeared after upgrading from v1.4 to v1.5 (and persisted in v1.6), with no relevant errors in logs. brahn speculated the bug might relate to a recent commit involving cancel_scheduled_job not passing arguments, potentially causing each site’s backup job to cancel the previous one. zogstrip confirmed the bug was unrelated to that commit but acknowledged the root cause: the scheduler was incorrectly canceling backup jobs for all sites, leaving only the last one to run. A fix was committed at https://github.com/discourse/discourse/commit/7db2083d4553001acbb12771a261e8dc61151eaf. brahn asked if the fix would be backported to stable, to which sam replied that only critical bugs and security fixes are backported — suggesting a cron job as a workaround. brahn also noted a UI quirk where marking a post as the solution temporarily hid a link preview, which sam acknowledged as a bug; brahn later confirmed it resolved itself, and sam noted they had rebuilt the HTML.

I have 3 sites in a standalone multisite setup and recently noticed that only one of them is actually performing the scheduled backup.
There does not appear to be anything in the logs about the other sites failing to backup. The unicorn.stdout.log shows only the one site being backed up. A manual backup of all sites works fine.
It seems to have stopped working around the time I updated from 1.4 to 1.5 (on 1.6 now but no change).

Any ideas on how to diagnose this?

Could the issue be connected to this fix?

https://github.com/discourse/discourse/commit/737c6067104e06a9b96c11956062e10eccebf9e9

The cancel_scheduled_job call in ScheduleBackup is different to the others I found in that it doesn’t supply any args.

My WAG is that perhaps each RailsMultisite iteration in the base scheduler is cancelling the previous scheduled backup leaving only the last one to run?!

While the issue was not related to that commit (the bug was present before), it got me to the right point: we were indeed canceling jobs for all the sites on a multisite install… :expressionless:

https://github.com/discourse/discourse/commit/7db2083d4553001acbb12771a261e8dc61151eaf

That’s awesome news, thanks!
I’m tracking the stable branch so I assume I will need to wait until v1.7 before I see this fix?

Yes, in general we only backport egregious bugs and security fixes into stable.

If you are going to be stuck on stable you can always look at adding a cron job to backup your sites.

Thanks @sam, understood.

BTW, when I marked the earlier post a the solution it seems to have removed the preview of the link. Is that supposed to happen?

That is some weird bug…

Oh, its back now anyway.

Yeah I rebuilt the html