Bounced e-mails from Amazon SES/SNS not working

Dev
1

I was aware that Discourse offers an endpoint to handle SNS notifications from Amazon SES. The endpoint is /webhooks/aws, and here is where I got aware of it:

https://meta.discourse.org/t/handling-bouncing-e-mails/45343/177?u=wlandgraf

Everything is “almost” working. I can see that AWS is sending the notifications to that endpoint, via CloudWatch log. I also see that Discourse is responding the notification with successful status code (200), after I configured it to send the raw message.

However, the e-mail doesn’t appear in the list of “Bounced” e-mails in Discourse. Is there anything else I should do? How could I check some log that shows me what the endpoint effectively did after receiving the notification from AWS?

2

Ok, I was able to find the log about the failed request, but I’m still lost about what could this be?

Encoding::UndefinedConversionError ("\xC3" from ASCII-8BIT to UTF-8)
app/jobs/base.rb:314:in `block in enqueue'
lib/mini_sql_multisite_connection.rb:68:in `after_commit'
app/jobs/base.rb:314:in `enqueue'
app/controllers/webhooks_controller.rb:117:in `aws'
lib/middleware/omniauth_bypass_middleware.rb:71:in `call'
lib/content_security_policy/middleware.rb:12:in `call'
lib/middleware/anonymous_cache.rb:355:in `call'
config/initializers/100-quiet_logger.rb:23:in `call'
config/initializers/100-silence_logger.rb:31:in `call'
lib/middleware/enforce_hostname.rb:23:in `call'
lib/middleware/request_tracker.rb:177:in `call'
3

Looks like it’s related to this: Encoding::UndefinedConversionError from ASCII-8BIT to UTF-8 · Issue #24 · karafka/sidekiq-backend · GitHub.

Discourse is passing the JSON received from Amazon SNS directly to Sidekiq, which in turn is not being able to handle it properly.

I also wonder why in webhooks_controller.rb all webhooks are calling process_bounce directly and the aws one is the only one which is enqueuing a job for processing it?