Our Reply-by-Email worked for two years, then stopped working 17 days ago for no apparent reason. I’ve searched the staff action logs for that time period and there were no relevant configuration changes. no upgrades, and no new plugins. What could have broken? No received emails after Sep 30: [image] They’re not being rejected either: [image] Nothing obviously related in the error logs, but there is this: aws-sdk-core-3.112.1/lib/seahorse/client/plugins/raise_response_errors.rb:17:in `call' aws-sdk-s3-1.96.1/lib/aws-sdk-s3/plugins/sse_cpk.rb:24:in `call' aws-sdk-s3-1.96.1/lib/aws-sdk-s3/plugins/dualstack.rb:36:in `call' aws-sdk-s3-1.96.1/lib/aws-sdk-s3/plugins/accelerate.rb:50:in `call' aws-sdk-core-3.112.1/lib/aws-sdk-core/plugins/jsonvalue_converter.rb:22:in `call' aws-sdk-core-3.112.1/lib/aws-sdk-core/plugins/idempotency_token.rb:19:in `call' aws-sdk-core-3.112.1/lib/aws-sdk-core/plugins/param_converter.rb:26:in `call' aws-sdk-core-3.112.1/lib/seahorse/client/plugins/request_callback.rb:71:in `call' aws-sdk-core-3.112.1/lib/aws-sdk-core/plugins/response_paging.rb:12:in `call' aws-sdk-core-3.112.1/lib/seahorse/client/plugins/response_target.rb:24:in `call' aws-sdk-core-3.112.1/lib/seahorse/client/request.rb:72:in `send_request' aws-sdk-s3-1.96.1/lib/aws-sdk-s3/client.rb:1274:in `copy_object' /var/www/discourse/lib/backup_restore/s3_backup_store.rb:61:in `block in vacate_legacy_prefix' /var/www/discourse/lib/backup_restore/s3_backup_store.rb:60:in `each' /var/www/discourse/lib/backup_restore/s3_backup_store.rb:60:in `vacate_legacy_prefix' /var/www/discourse/app/jobs/onceoff/vacate_legacy_prefix_backups.rb:7:in `execute_onceoff' /var/www/discourse/app/jobs/onceoff/onceoff.rb:25:in `execute' /var/www/discourse/app/jobs/base.rb:232:in `block (2 levels) in perform' rails_multisite-3.1.0/lib/rails_multisite/connection_management.rb:80:in `with_connection' /var/www/discourse/app/jobs/base.rb:221:in `block in perform' /var/www/discourse/app/jobs/base.rb:217:in `each' /var/www/discourse/app/jobs/base.rb:217:in `perform' sidekiq-6.2.2/lib/sidekiq/processor.rb:196:in `execute_job' sidekiq-6.2.2/lib/sidekiq/processor.rb:164:in `block (2 levels) in process' sidekiq-6.2.2/lib/sidekiq/middleware/chain.rb:138:in `block in invoke' /var/www/discourse/lib/sidekiq/pausable.rb:138:in `call' sidekiq-6.2.2/lib/sidekiq/middleware/chain.rb:140:in `block in invoke' sidekiq-6.2.2/lib/sidekiq/middleware/chain.rb:143:in `invoke' sidekiq-6.2.2/lib/sidekiq/processor.rb:163:in `block in process' sidekiq-6.2.2/lib/sidekiq/processor.rb:136:in `block (6 levels) in dispatch' sidekiq-6.2.2/lib/sidekiq/job_retry.rb:112:in `local' sidekiq-6.2.2/lib/sidekiq/processor.rb:135:in `block (5 levels) in dispatch' sidekiq-6.2.2/lib/sidekiq.rb:38:in `block in ' sidekiq-6.2.2/lib/sidekiq/processor.rb:131:in `block (4 levels) in dispatch' sidekiq-6.2.2/lib/sidekiq/processor.rb:257:in `stats' sidekiq-6.2.2/lib/sidekiq/processor.rb:126:in `block (3 levels) in dispatch' sidekiq-6.2.2/lib/sidekiq/job_logger.rb:13:in `call' sidekiq-6.2.2/lib/sidekiq/processor.rb:125:in `block (2 levels) in dispatch' sidekiq-6.2.2/lib/sidekiq/job_retry.rb:79:in `global' sidekiq-6.2.2/lib/sidekiq/processor.rb:124:in `block in dispatch' sidekiq-6.2.2/lib/sidekiq/logger.rb:11:in `with' sidekiq-6.2.2/lib/sidekiq/job_logger.rb:33:in `prepare' sidekiq-6.2.2/lib/sidekiq/processor.rb:123:in `dispatch' sidekiq-6.2.2/lib/sidekiq/processor.rb:162:in `process' sidekiq-6.2.2/lib/sidekiq/processor.rb:78:in `process_one' sidekiq-6.2.2/lib/sidekiq/processor.rb:68:in `run' sidekiq-6.2.2/lib/sidekiq/util.rb:43:in `watchdog' sidekiq-6.2.2/lib/sidekiq/util.rb:52:in `block in safe_thread' Using manual polling, not pop3 polling: [image] Senders trying to reply by email do not get any rejection notice or error message. Sending emails from the forum works fine. Nothing bad at AWS SES (although I don’t think that has anything to do with reply-by-email?). Nothing bad in discourse-doctor (except it does note non-official plugins, but those did not change in this time period).

Sorry. Auto correct. Rebuild your mail container. cd /var/discourse ./launcher rebuild mail-receiver

This could merely be a coincidence (and part of me thinks it is), but… [תמונה] Millions of iPhones, TVs and other devices could go offline Thursday — here's... Expiring security standard may cripple internet access

It’s quite an affluent group- doubt there are many old devices being used.

I was thinking that it’s more of a server-side issue, not a client-side issue. [תמונה] Top websites see outages as Let's Encrypt's CA certificate expires Shopify and Fortinet amongst those affected

I’m afraid I’ll need more explanation- there’s no other mention in all of meta about NFL (except for football teams). NFS? No such containers in /var/discourse/containers.

Thanks @pfaffman, that fixed it. Sounds like you’ve seen this before- is there something that should have been done better? In our case, there were repercussions beyond just not receiving user emails: [תמונה]Configure VERP to handle bouncing e-mails Discourse uses the VERP technique to handle bouncing e-mails. VERP relies on incoming emails. So we weren’t handling bounces and complaints for a few days, our reputation with SES plummeted, and they suspended our outgoing emails.

[תמונה] Mark_Schmucker: is there something that should have been done better? It’s a result of DST Root CA X3 Expiration (September 2021) - Let's Encrypt. It caught a few people off guard and the issues that it caused for Discourse self-hosters are subtle enough that it got overlooked. And there are still issues that need to be resolved. [תמונה] Mark_Schmucker: So we weren’t handling bounces and com I suspect that they stopped being handled October 1. I never figured out how to get SES working. An emergency stopgap would be to set up outgoing mail with mailgun.

Reply-by-Email עבד, עכשיו שבור

תמיכה

pfaffman (Jay Pfaffman) 17 באוקטובר,‏ 2021,‏ 5:29am 5

I think that you need to rebuild your mail ~~NFL~~container.

נושא		תגובות	צפיות
Reply by email stops working Product Support	1	431	16 בספטמבר,‏ 2021
Self hosted Reply by Email stopped working after latest update Product Support	25	2611	26 באוקטובר,‏ 2021
Email-in replies thread wrongly Bug	17	6983	22 ביוני,‏ 2017
Replying through email suddenly stopped due to Let's Encrypt changes Product Support	3	1144	12 בינואר,‏ 2022
Amazon SES/SNS email complaints not received, bounces missing information Self-hosting email	5	1679	6 באפריל,‏ 2023

Reply-by-Email עבד, עכשיו שבור

נושאים קשורים