### Author:  [@Mark\_Schmucker](https://meta.discourse.org/u/Mark_Schmucker)
#### Post date: [October 17, 2021, 12:57am UTC](https://meta.discourse.org/t/reply-by-email-was-working-now-broken/206247/1 "2021-10-17T00:57:34Z")
Our Reply-by-Email worked for two years, then stopped working 17 days ago for no apparent reason. I’ve searched the staff action logs for that time period and there were no relevant configuration changes. no upgrades, and no new plugins. What could have broken?
* * *
No received emails after Sep 30:
* * *
They’re not being rejected either:
* * *
Nothing obviously related in the error logs, but there is this:
> ****
>
> ```plaintext
> aws-sdk-core-3.112.1/lib/seahorse/client/plugins/raise_response_errors.rb:17:in `call'
>
> aws-sdk-s3-1.96.1/lib/aws-sdk-s3/plugins/sse_cpk.rb:24:in `call'
>
> aws-sdk-s3-1.96.1/lib/aws-sdk-s3/plugins/dualstack.rb:36:in `call'
>
> aws-sdk-s3-1.96.1/lib/aws-sdk-s3/plugins/accelerate.rb:50:in `call'
>
> aws-sdk-core-3.112.1/lib/aws-sdk-core/plugins/jsonvalue_converter.rb:22:in `call'
>
> aws-sdk-core-3.112.1/lib/aws-sdk-core/plugins/idempotency_token.rb:19:in `call'
>
> aws-sdk-core-3.112.1/lib/aws-sdk-core/plugins/param_converter.rb:26:in `call'
>
> aws-sdk-core-3.112.1/lib/seahorse/client/plugins/request_callback.rb:71:in `call'
>
> aws-sdk-core-3.112.1/lib/aws-sdk-core/plugins/response_paging.rb:12:in `call'
>
> aws-sdk-core-3.112.1/lib/seahorse/client/plugins/response_target.rb:24:in `call'
>
> aws-sdk-core-3.112.1/lib/seahorse/client/request.rb:72:in `send_request'
>
> aws-sdk-s3-1.96.1/lib/aws-sdk-s3/client.rb:1274:in `copy_object'
>
> /var/www/discourse/lib/backup_restore/s3_backup_store.rb:61:in `block in vacate_legacy_prefix'
>
> /var/www/discourse/lib/backup_restore/s3_backup_store.rb:60:in `each'
>
> /var/www/discourse/lib/backup_restore/s3_backup_store.rb:60:in `vacate_legacy_prefix'
>
> /var/www/discourse/app/jobs/onceoff/vacate_legacy_prefix_backups.rb:7:in `execute_onceoff'
>
> /var/www/discourse/app/jobs/onceoff/onceoff.rb:25:in `execute'
>
> /var/www/discourse/app/jobs/base.rb:232:in `block (2 levels) in perform'
>
> rails_multisite-3.1.0/lib/rails_multisite/connection_management.rb:80:in `with_connection'
>
> /var/www/discourse/app/jobs/base.rb:221:in `block in perform'
>
> /var/www/discourse/app/jobs/base.rb:217:in `each'
>
> /var/www/discourse/app/jobs/base.rb:217:in `perform'
>
> sidekiq-6.2.2/lib/sidekiq/processor.rb:196:in `execute_job'
>
> sidekiq-6.2.2/lib/sidekiq/processor.rb:164:in `block (2 levels) in process'
>
> sidekiq-6.2.2/lib/sidekiq/middleware/chain.rb:138:in `block in invoke'
>
> /var/www/discourse/lib/sidekiq/pausable.rb:138:in `call'
>
> sidekiq-6.2.2/lib/sidekiq/middleware/chain.rb:140:in `block in invoke'
>
> sidekiq-6.2.2/lib/sidekiq/middleware/chain.rb:143:in `invoke'
>
> sidekiq-6.2.2/lib/sidekiq/processor.rb:163:in `block in process'
>
> sidekiq-6.2.2/lib/sidekiq/processor.rb:136:in `block (6 levels) in dispatch'
>
> sidekiq-6.2.2/lib/sidekiq/job_retry.rb:112:in `local'
>
> sidekiq-6.2.2/lib/sidekiq/processor.rb:135:in `block (5 levels) in dispatch'
>
> sidekiq-6.2.2/lib/sidekiq.rb:38:in `block in '
>
> sidekiq-6.2.2/lib/sidekiq/processor.rb:131:in `block (4 levels) in dispatch'
>
> sidekiq-6.2.2/lib/sidekiq/processor.rb:257:in `stats'
>
> sidekiq-6.2.2/lib/sidekiq/processor.rb:126:in `block (3 levels) in dispatch'
>
> sidekiq-6.2.2/lib/sidekiq/job_logger.rb:13:in `call'
>
> sidekiq-6.2.2/lib/sidekiq/processor.rb:125:in `block (2 levels) in dispatch'
>
> sidekiq-6.2.2/lib/sidekiq/job_retry.rb:79:in `global'
>
> sidekiq-6.2.2/lib/sidekiq/processor.rb:124:in `block in dispatch'
>
> sidekiq-6.2.2/lib/sidekiq/logger.rb:11:in `with'
>
> sidekiq-6.2.2/lib/sidekiq/job_logger.rb:33:in `prepare'
>
> sidekiq-6.2.2/lib/sidekiq/processor.rb:123:in `dispatch'
>
> sidekiq-6.2.2/lib/sidekiq/processor.rb:162:in `process'
>
> sidekiq-6.2.2/lib/sidekiq/processor.rb:78:in `process_one'
>
> sidekiq-6.2.2/lib/sidekiq/processor.rb:68:in `run'
>
> sidekiq-6.2.2/lib/sidekiq/util.rb:43:in `watchdog'
>
> sidekiq-6.2.2/lib/sidekiq/util.rb:52:in `block in safe_thread'
>
> ```
* * *
Using manual polling, not pop3 polling:
* * *
Senders trying to reply by email do not get any rejection notice or error message.
Sending emails from the forum works fine. Nothing bad at AWS SES (although I don’t think that has anything to do with reply-by-email?).
Nothing bad in discourse-doctor (except it does note non-official plugins, but those did not change in this time period).
---
### Author:  [@anon73664359](https://meta.discourse.org/u/anon73664359)
#### Post date: [October 17, 2021, 2:02am UTC](https://meta.discourse.org/t/reply-by-email-was-working-now-broken/206247/2 "2021-10-17T02:02:49Z")
This could merely be a coincidence (and part of me thinks it is), but…
> **[Millions of iPhones, TVs and other devices could go offline Thursday — here's...](https://www.tomsguide.com/news/internet-disconnect-sept-30)**
>
> Expiring security standard may cripple internet access
---
### Author:  [@Mark\_Schmucker](https://meta.discourse.org/u/Mark_Schmucker)
#### Post date: [October 17, 2021, 2:07am UTC](https://meta.discourse.org/t/reply-by-email-was-working-now-broken/206247/3 "2021-10-17T02:07:21Z")
It’s quite an affluent group- doubt there are many old devices being used.
---
### Author:  [@anon73664359](https://meta.discourse.org/u/anon73664359)
#### Post date: [October 17, 2021, 2:15am UTC](https://meta.discourse.org/t/reply-by-email-was-working-now-broken/206247/4 "2021-10-17T02:15:11Z")
I was thinking that it’s more of a server-side issue, not a client-side issue.
> **[Top websites see outages as Let's Encrypt's CA certificate expires](https://www.techradar.com/news/slack-down-top-websites-see-outages-as-lets-encrypts-ca-certificate-expires)**
>
> Shopify and Fortinet amongst those affected
---
### Author:  [@pfaffman](https://meta.discourse.org/u/pfaffman)
#### Post date: [October 17, 2021, 5:29am UTC](https://meta.discourse.org/t/reply-by-email-was-working-now-broken/206247/5 "2021-10-17T05:29:09Z")
I think that you need to rebuild your mail ~~NFL~~ container.
---
### Author:  [@Mark\_Schmucker](https://meta.discourse.org/u/Mark_Schmucker)
#### Post date: [October 17, 2021, 5:35am UTC](https://meta.discourse.org/t/reply-by-email-was-working-now-broken/206247/6 "2021-10-17T05:35:52Z")
I’m afraid I’ll need more explanation- there’s no other mention in all of meta about NFL (except for football teams). NFS? No such containers in /var/discourse/containers.
---
### Author:  [@pfaffman](https://meta.discourse.org/u/pfaffman)
#### Post date: [October 17, 2021, 10:41am UTC](https://meta.discourse.org/t/reply-by-email-was-working-now-broken/206247/7 "2021-10-17T10:41:04Z")
Sorry. Auto correct.
Rebuild your mail container.
```plaintext
cd /var/discourse
./launcher rebuild mail-receiver
```
---
### Author:  [@Mark\_Schmucker](https://meta.discourse.org/u/Mark_Schmucker)
#### Post date: [October 19, 2021, 3:37am UTC](https://meta.discourse.org/t/reply-by-email-was-working-now-broken/206247/8 "2021-10-19T03:37:17Z")
Thanks @pfaffman, that fixed it. Sounds like you’ve seen this before- is there something that should have been done better? In our case, there were repercussions beyond just not receiving user emails:
> [@Configure VERP to handle bouncing e-mails](https://meta.discourse.org/t/configure-verp-to-handle-bouncing-e-mails/45343/1):
>
> Discourse uses the [VERP](https://en.wikipedia.org/wiki/Variable_envelope_return_path) technique to handle bouncing e-mails.
VERP relies on incoming emails. So we weren’t handling bounces and complaints for a few days, our reputation with SES plummeted, and they suspended our outgoing emails.
---
### Author:  [@pfaffman](https://meta.discourse.org/u/pfaffman)
#### Post date: [October 19, 2021, 1:43pm UTC](https://meta.discourse.org/t/reply-by-email-was-working-now-broken/206247/9 "2021-10-19T13:43:26Z")
> [@Mark\_Schmucker](#):
>
> is there something that should have been done better?
It’s a result of [DST Root CA X3 Expiration (September 2021) - Let's Encrypt](https://letsencrypt.org/docs/dst-root-ca-x3-expiration-september-2021/). It caught a few people off guard and the issues that it caused for Discourse self-hosters are subtle enough that it got overlooked. And there are still issues that need to be resolved.
> [@Mark\_Schmucker](#):
>
> So we weren’t handling bounces and com
I suspect that they stopped being handled October 1.
I never figured out how to get SES working. An emergency stopgap would be to set up outgoing mail with mailgun.
---
### Author:  [@system](https://meta.discourse.org/u/system)
#### Post date: [November 18, 2021, 1:44pm UTC](https://meta.discourse.org/t/reply-by-email-was-working-now-broken/206247/10 "2021-11-18T13:44:07Z")
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.