אימיילים חוזרים מ-Amazon SES/SNS לא עובדים

מפתח

סיכום מבוסס בינה מלאכותית

The user wlandgraf is troubleshooting bounced emails from Amazon SES/SNS in Discourse. While AWS notifications are reaching the /webhooks/aws endpoint and Discourse responds with a 200 status, the emails do not appear in the list of bounced emails. The user identifies a Encoding::UndefinedConversionError in the logs, suggesting an issue with processing the SNS message, possibly due to encoding mismatch when passing JSON to Sidekiq. The user notes that other webhook handlers process bounces directly, but the AWS handler enqueues a job, which may be contributing to the problem. wlandgraf further investigates, linking the error to a known issue in the Sidekiq backend, and questions the design choice of queuing the AWS bounce processing instead of handling it immediately.

1

I was aware that Discourse offers an endpoint to handle SNS notifications from Amazon SES. The endpoint is /webhooks/aws, and here is where I got aware of it:

https://meta.discourse.org/t/handling-bouncing-e-mails/45343/177?u=wlandgraf

Everything is “almost” working. I can see that AWS is sending the notifications to that endpoint, via CloudWatch log. I also see that Discourse is responding the notification with successful status code (200), after I configured it to send the raw message.

However, the e-mail doesn’t appear in the list of “Bounced” e-mails in Discourse. Is there anything else I should do? How could I check some log that shows me what the endpoint effectively did after receiving the notification from AWS?

2

Ok, I was able to find the log about the failed request, but I’m still lost about what could this be?

Encoding::UndefinedConversionError ("\xC3" from ASCII-8BIT to UTF-8)
app/jobs/base.rb:314:in `block in enqueue'
lib/mini_sql_multisite_connection.rb:68:in `after_commit'
app/jobs/base.rb:314:in `enqueue'
app/controllers/webhooks_controller.rb:117:in `aws'
lib/middleware/omniauth_bypass_middleware.rb:71:in `call'
lib/content_security_policy/middleware.rb:12:in `call'
lib/middleware/anonymous_cache.rb:355:in `call'
config/initializers/100-quiet_logger.rb:23:in `call'
config/initializers/100-silence_logger.rb:31:in `call'
lib/middleware/enforce_hostname.rb:23:in `call'
lib/middleware/request_tracker.rb:177:in `call'
3

Looks like it’s related to this: Encoding::UndefinedConversionError from ASCII-8BIT to UTF-8 · Issue #24 · karafka/sidekiq-backend · GitHub.

Discourse is passing the JSON received from Amazon SNS directly to Sidekiq, which in turn is not being able to handle it properly.

I also wonder why in webhooks_controller.rb all webhooks are calling process_bounce directly and the aws one is the only one which is enqueuing a job for processing it?