Botched restore led to Sidekiq job failures

[Simsnet (Glenn Sims)]

Backed up my server prior to a mass-delete and a typo occurred (sleep comes first ) so I ran a restore.

It quit halfway through the first time I ran it and I was never messaged me a log. I ran the restore a second time and it completed, this time with the messaged log.

Now, I’m getting the Sidekiq update error:
A check for updates has not been performed. Ensure sidekiq is running.

Tried triggering it manually from /sidekiq/scheduler but it keeps failing after 60 seconds.
The error in “retries” shows this error:
Jobs::HandledExceptionWrapper: Wrapped Excon::Error::Timeout: connect_write timeout reached

No clue what’s going on now. Forum is still up and a few other Sidekiq tasks are failing.

[pfaffman (Jay Pfaffman)]

Maybe reboot. But likely you’re swamped running post restore processes. How much ram, disk, cpu, posts?

[gerhard (Gerhard Schlager)]

You can ignore that error. A network interruption at one of our data centers is currently affecting version checks for self-hosters. They will start working again when the incident has been resolved.

[Simsnet (Glenn Sims)]

I’ve rebooted. The forum has a total of 100k+ posts (not sure where to pull an exact number) and there are several image hotlinks queuing every few minutes.

Other specs: 2 CPUs, 80GB storage, 4GB RAM

[Simsnet (Glenn Sims)]

I saw you mention that in another post. Have other users hosting forums reported the issue as well?

[gerhard (Gerhard Schlager)]

No, they haven’t. It’s a minor issue and isn’t noticeable unless you stare at logs at the right time. It doesn’t affect Discourse at all. No need to worry.

[Simsnet (Glenn Sims)]

All of the image hotlink tasks have pushed through and the Jobs::VersionCheck worker is now saying OK. Seems like that backlog kept it from working right.

@pfaffman @gerhard Thank you both very much!