# Sidekiq está sendo pausado, como posso descobrir o motivo?
**URL:** https://meta.discourse.org/t/sidekiq-is-being-paused-how-can-i-discover-why/97340
**Category:** Support
**Created:** [Setembro 17, 2018, 10:18am UTC](https://meta.discourse.org/t/sidekiq-is-being-paused-how-can-i-discover-why/97340 "2018-09-17T10:18:09Z")
**Posts on this page:** 19
**Page:** 1
### Author:  [@cintiadr](https://meta.discourse.org/u/cintiadr)
#### Post date: [Setembro 17, 2018, 10:18am UTC](https://meta.discourse.org/t/sidekiq-is-being-paused-how-can-i-discover-why/97340/1 "2018-09-17T10:18:09Z")
Hi everyone,
For the last few weeks, my discourse instance stopped sending emails on three different occasions.
In all of them, it was because sidekiq was somehow paused.
I followed [https://meta.discourse.org/t/sidekiq-stopped-working/26479](https://meta.discourse.org/t/sidekiq-stopped-working/26479) , and all the queue was cleared and all emails sent.
Interestingly enough, they all happened a few hours after an upgrade.
My question is: how can I discover what caused sidekiq to be paused?
---
### Author:  [@merefield](https://meta.discourse.org/u/merefield)
#### Post date: [Setembro 17, 2018, 10:20am UTC](https://meta.discourse.org/t/sidekiq-is-being-paused-how-can-i-discover-why/97340/2 "2018-09-17T10:20:33Z")
This happened to me recently and was driving me nuts!
It also cost me several new user sign-ups … 😠
Spot where I fixed it! 😌
[Use the rails console to unpause sidekiq.](https://meta.discourse.org/t/sidekiq-kicking-it/52390) (This is only a temporary fix though)
Check your logs, it’s likely that you have an error. I was seeing Access Denied like messages. Check your S3 backup process if you have one. For some reason there must have been an update to it at some point that required additional permissions on Amazon S3. [A broader policy on AWS S3 fixed it for me.](https://meta.discourse.org/t/s3-backup-suspect-access-issue/97134)
YMMV
---
### Author:  [@cintiadr](https://meta.discourse.org/u/cintiadr)
#### Post date: [Setembro 17, 2018, 11:10am UTC](https://meta.discourse.org/t/sidekiq-is-being-paused-how-can-i-discover-why/97340/3 "2018-09-17T11:10:20Z")
Oooooooooh. That’s unexpected!
Yes, in fact, I saw that message a few minutes ago when trying to search for any indication why sidekiq was paused, but it didn’t occur to me it could be related.
I did fix that option (I don’t want to delete files from S3) a couple of minutes before raising this topic, but I didn’t think much of it. When I imported the data to the new server, it appears some options were lost.
I was going to follow it to make sure the error went away, but now I’m hopeful that will fix the sidekiq pausing as well!
Thanks
---
### Author:  [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [Setembro 17, 2018, 11:44am UTC](https://meta.discourse.org/t/sidekiq-is-being-paused-how-can-i-discover-why/97340/4 "2018-09-17T11:44:47Z")
Hmm why would S3 problems cause Sidekiq to completely stop @gerhard?
---
### Author:  [@merefield](https://meta.discourse.org/u/merefield)
#### Post date: [Setembro 17, 2018, 12:00pm UTC](https://meta.discourse.org/t/sidekiq-is-being-paused-how-can-i-discover-why/97340/5 "2018-09-17T12:00:16Z")
Just a guess - isn’t Sidekiq paused during a backup? (and perhaps understandably so because backup process takes up so much local computational resource)
So if this job falls over, it’s never automatically unpaused?
---
### Author:  [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [Setembro 17, 2018, 12:03pm UTC](https://meta.discourse.org/t/sidekiq-is-being-paused-how-can-i-discover-why/97340/6 "2018-09-17T12:03:58Z")
Oh yes, right. It gets paused during backup and never recovers. You’re exactly correct and I remember us running into this before.
I wonder if we should have a safety mechanism where sidekiq cannot be “paused for backup” for more than x hours, where x is maybe four? What do you think @gerhard?
An arbitrary admin pause of sidekiq should always be possible, but a “stuck forever for backup” pause doesn’t seem right to me.
---
### Author:  [@cintiadr](https://meta.discourse.org/u/cintiadr)
#### Post date: [Setembro 17, 2018, 12:05pm UTC](https://meta.discourse.org/t/sidekiq-is-being-paused-how-can-i-discover-why/97340/7 "2018-09-17T12:05:29Z")
I started following that lead, and it seems pretty promising. \
I went to check my backups, and just the first one after upgraded was in S3.
I couldn’t even start a new backup in discourse, I had to cancel several times (I’d cancel it, and refresh the page, and cancel it again and again).
So it appears the timeline is more or less like:
- We upgrade
- in less than 24h, a backup will happen and it’s uploaded to S3
- sidekiq is paused
Even if I unpaused sidekiq, no new backups were created, which was unexpected.
For the record, this is what I had for S3:
```
Access Denied
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.21.2/lib/seahorse/client/plugins/raise_response_errors.rb:15:in `call'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/aws-sdk-s3-1.14.0/lib/aws-sdk-s3/plugins/sse_cpk.rb:22:in `call'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/aws-sdk-s3-1.14.0/lib/aws-sdk-s3/plugins/dualstack.rb:26:in `call'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/aws-sdk-s3-1.14.0/lib/aws-sdk-s3/plugins/accelerate.rb:35:in `call'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.21.2/lib/aws-sdk-core/plugins/jsonvalue_converter.rb:20:in `call'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.21.2/lib/aws-sdk-core/plugins/idempotency_token.rb:17:in `call'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.21.2/lib/aws-sdk-core/plugins/param_converter.rb:24:in `call'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.21.2/lib/aws-sdk-core/plugins/response_paging.rb:10:in `call'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.21.2/lib/seahorse/client/plugins/response_target.rb:23:in `call'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.21.2/lib/seahorse/client/request.rb:70:in `send_request'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/aws-sdk-s3-1.14.0/lib/aws-sdk-s3/client.rb:1248:in `delete_object'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/aws-sdk-s3-1.14.0/lib/aws-sdk-s3/object.rb:571:in `delete'
/var/www/discourse/lib/s3_helper.rb:42:in `remove'
/var/www/discourse/app/models/backup.rb:64:in `remove_from_s3'
/var/www/discourse/app/models/backup.rb:40:in `after_remove_hook'
/var/www/discourse/app/models/backup.rb:31:in `remove'
/var/www/discourse/app/models/backup.rb:89:in `each'
/var/www/discourse/app/models/backup.rb:89:in `remove_old'
/var/www/discourse/lib/backup_restore/backuper.rb:257:in `remove_old'
/var/www/discourse/lib/backup_restore/backuper.rb:59:in `run'
/var/www/discourse/lib/backup_restore/backup_restore.rb:16:in `backup!'
/var/www/discourse/app/jobs/regular/create_backup.rb:8:in `execute'
/var/www/discourse/app/jobs/base.rb:137:in `block (2 levels) in perform'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/rails_multisite-2.0.4/lib/rails_multisite/connection_management.rb:63:in `with_connection'
/var/www/discourse/app/jobs/base.rb:127:in `block in perform'
/var/www/discourse/app/jobs/base.rb:123:in `each'
/var/www/discourse/app/jobs/base.rb:123:in `perform'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:187:in `execute_job'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:169:in `block (2 levels) in process'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/middleware/chain.rb:128:in `block in invoke'
/var/www/discourse/lib/sidekiq/pausable.rb:81:in `call'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/middleware/chain.rb:133:in `invoke'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:168:in `block in process'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:139:in `block (6 levels) in dispatch'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/job_retry.rb:98:in `local'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:138:in `block (5 levels) in dispatch'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq.rb:36:in `block in '
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:134:in `block (4 levels) in dispatch'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:199:in `stats'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:129:in `block (3 levels) in dispatch'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/job_logger.rb:8:in `call'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:128:in `block (2 levels) in dispatch'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/job_retry.rb:73:in `global'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:127:in `block in dispatch'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/logging.rb:48:in `with_context'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/logging.rb:42:in `with_job_hash_context'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:126:in `dispatch'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:167:in `process'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:85:in `process_one'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:73:in `run'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/util.rb:16:in `watchdog'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/util.rb:25:in `block in safe_thread'
```
I’m not particularly sure if the automatic backup will work, but I managed to run 3 backups manually (and they are in S3).
---
### Author:  [@merefield](https://meta.discourse.org/u/merefield)
#### Post date: [Setembro 17, 2018, 12:27pm UTC](https://meta.discourse.org/t/sidekiq-is-being-paused-how-can-i-discover-why/97340/8 "2018-09-17T12:27:12Z")
Yes because it halts other processes like notifications and sign-up emails.
However the really important thing imho is to bring this to the admin’s attention ASAP.
It was only because I navigated to Sidekiq that I found the issue. This lead me eventually to check logs etc.
---
### Author:  [@gerhard](https://meta.discourse.org/u/gerhard)
#### Post date: [Setembro 17, 2018, 12:28pm UTC](https://meta.discourse.org/t/sidekiq-is-being-paused-how-can-i-discover-why/97340/9 "2018-09-17T12:28:54Z")
The backup process fails to delete old backups from S3. Unfortunately it crashes inside the `ensure` block and prevents Sidekiq from starting again. I’m going to fix that.
>
---
### Author:  [@cintiadr](https://meta.discourse.org/u/cintiadr)
#### Post date: [Setembro 17, 2018, 1:07pm UTC](https://meta.discourse.org/t/sidekiq-is-being-paused-how-can-i-discover-why/97340/11 "2018-09-17T13:07:54Z")
> [@merefield](#):
>
> However the really important thing imho is to bring this to the admin’s attention ASAP.
>
> It was only because I navigated to Sidekiq that I found the issue. This lead me eventually to check logs etc.
I agree.
If a backup fails (even partially, to delete old files), I do not expect every other background process to be silently paused. I do expect admins to be somehow notified.
I just went to sidekiq because some support topics mentioned it, but I didn’t know my backups weren’t working.
---
### Author:  [@gerhard](https://meta.discourse.org/u/gerhard)
#### Post date: [Setembro 17, 2018, 1:39pm UTC](https://meta.discourse.org/t/sidekiq-is-being-paused-how-can-i-discover-why/97340/12 "2018-09-17T13:39:58Z")
There is a warning on the admin dashboard for the rare case that Sidekiq stopped working.
But I agree, that a failed backup shouldn’t prevent Sidekiq from running. It will be fixed!
---
### Author:  [@merefield](https://meta.discourse.org/u/merefield)
#### Post date: [Setembro 17, 2018, 2:01pm UTC](https://meta.discourse.org/t/sidekiq-is-being-paused-how-can-i-discover-why/97340/13 "2018-09-17T14:01:55Z")
Thanks! That’s weird, because _I did not see that message_ and looked at the Dashboard dozens of times during the paused phase. Perhaps it was just me …
Good to know this feature is there, though!
Appreciate the bug fix, many thanks!
---
### Author:  [@merefield](https://meta.discourse.org/u/merefield)
#### Post date: [Setembro 17, 2018, 3:55pm UTC](https://meta.discourse.org/t/sidekiq-is-being-paused-how-can-i-discover-why/97340/14 "2018-09-17T15:55:12Z")
@Gerhard I just paused my Sidekiq using `rails c` and do not see this warning … why might that be?:
---
### Author:  [@gerhard](https://meta.discourse.org/u/gerhard)
#### Post date: [Setembro 17, 2018, 4:01pm UTC](https://meta.discourse.org/t/sidekiq-is-being-paused-how-can-i-discover-why/97340/15 "2018-09-17T16:01:34Z")
The warning only appears when there are jobs enqueued and the last job was executed more than 2 minutes ago.
>
---
### Author:  [@merefield](https://meta.discourse.org/u/merefield)
#### Post date: [Setembro 17, 2018, 4:05pm UTC](https://meta.discourse.org/t/sidekiq-is-being-paused-how-can-i-discover-why/97340/16 "2018-09-17T16:05:31Z")
Thanks for clarification! 🙂
---
### Author:  [@cintiadr](https://meta.discourse.org/u/cintiadr)
#### Post date: [Setembro 18, 2018, 1:30am UTC](https://meta.discourse.org/t/sidekiq-is-being-paused-how-can-i-discover-why/97340/17 "2018-09-18T01:30:38Z")
I saw the same thing as @merefield. Sidekiq had that big banner, but there was no banner in the admin dashboard.
I don’t have a screenshot, but from memory I had a huge number in scheduled (a few thousands), but enqueued was still zero.
---
### Author:  [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [Setembro 18, 2018, 2:18am UTC](https://meta.discourse.org/t/sidekiq-is-being-paused-how-can-i-discover-why/97340/18 "2018-09-18T02:18:17Z")
> [@cintiadr](#):
>
> I saw the same thing as @merefield. Sidekiq had that big banner, but there was no banner in the admin dashboard.
We clearly need a better way of triggering this @gerhard
---
### Author:  [@gerhard](https://meta.discourse.org/u/gerhard)
#### Post date: [Setembro 19, 2018, 7:25pm UTC](https://meta.discourse.org/t/sidekiq-is-being-paused-how-can-i-discover-why/97340/19 "2018-09-19T19:25:47Z")
I made some changes to the backup and restore process so that it always unpauses Sidekiq.
[FIX: Always unpause Sidekiq after backup and restore · discourse/discourse@469a2c3 · GitHub](https://github.com/discourse/discourse/commit/469a2c36edf63923eadcb2b673f734bc592b81cd)
> [@cintiadr](#):
>
> Sidekiq had that big banner, but there was no banner in the admin dashboard.
>
> I don’t have a screenshot, but from memory I had a huge number in scheduled (a few thousands), but enqueued was still zero.
You remembered correctly. There’s no warning in the dashboard when Sidekiq is paused, since jobs are scheduled but not enqueued. The warning appears only when Sidekiq isn’t running at all.
I could add an additional warning, but I don’t think it’s needed. Sidekiq should now always be unpaused – no matter what.
---
### Author:  [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [Setembro 20, 2018, 3:31am UTC](https://meta.discourse.org/t/sidekiq-is-being-paused-how-can-i-discover-why/97340/20 "2018-09-20T03:31:27Z")