KCAS
November 11, 2023, 10:24pm
1
Hello,
Have started to use the sendgrid webhook with verification key. Now are getting a lot of these errors. Had sent out a bulk invite with sendgrid.
app/controllers/webhooks_controller.rb:29:in `block in sendgrid'
app/controllers/webhooks_controller.rb:24:in `each'
app/controllers/webhooks_controller.rb:24:in `sendgrid'
actionpack (7.0.7) lib/action_controller/metal/basic_implicit_render.rb:6:in `send_action'
actionpack (7.0.7) lib/abstract_controller/base.rb:215:in `process_action'
actionpack (7.0.7) lib/action_controller/metal/rendering.rb:165:in `process_action'
actionpack (7.0.7) lib/abstract_controller/callbacks.rb:234:in `block in process_action'
activesupport (7.0.7) lib/active_support/callbacks.rb:107:in `run_callbacks'
actionpack (7.0.7) lib/abstract_controller/callbacks.rb:233:in `process_action'
actionpack (7.0.7) lib/action_controller/metal/rescue.rb:23:in `process_action'
actionpack (7.0.7) lib/action_controller/metal/instrumentation.rb:67:in `block in process_action'
activesupport (7.0.7) lib/active_support/notifications.rb:206:in `block in instrument'
activesupport (7.0.7) lib/active_support/notifications/instrumenter.rb:24:in `instrument'
activesupport (7.0.7) lib/active_support/notifications.rb:206:in `instrument'
actionpack (7.0.7) lib/action_controller/metal/instrumentation.rb:66:in `process_action'
actionpack (7.0.7) lib/action_controller/metal/params_wrapper.rb:259:in `process_action'
activerecord (7.0.7) lib/active_record/railties/controller_runtime.rb:27:in `process_action'
actionpack (7.0.7) lib/abstract_controller/base.rb:151:in `process'
actionview (7.0.7) lib/action_view/rendering.rb:39:in `process'
rack-mini-profiler (3.1.1) lib/mini_profiler/profiling_methods.rb:85:in `block in profile_method'
actionpack (7.0.7) lib/action_controller/metal.rb:188:in `dispatch'
actionpack (7.0.7) lib/action_controller/metal.rb:251:in `dispatch'
actionpack (7.0.7) lib/action_dispatch/routing/route_set.rb:49:in `dispatch'
actionpack (7.0.7) lib/action_dispatch/routing/route_set.rb:32:in `serve'
actionpack (7.0.7) lib/action_dispatch/journey/router.rb:50:in `block in serve'
actionpack (7.0.7) lib/action_dispatch/journey/router.rb:32:in `each'
actionpack (7.0.7) lib/action_dispatch/journey/router.rb:32:in `serve'
actionpack (7.0.7) lib/action_dispatch/routing/route_set.rb:852:in `call'
lib/middleware/omniauth_bypass_middleware.rb:64:in `call'
rack (2.2.8) lib/rack/tempfile_reaper.rb:15:in `call'
rack (2.2.8) lib/rack/conditional_get.rb:40:in `call'
rack (2.2.8) lib/rack/head.rb:12:in `call'
actionpack (7.0.7) lib/action_dispatch/http/permissions_policy.rb:38:in `call'
lib/content_security_policy/middleware.rb:12:in `call'
lib/middleware/anonymous_cache.rb:389:in `call'
lib/middleware/gtm_script_nonce_injector.rb:10:in `call'
rack (2.2.8) lib/rack/session/abstract/id.rb:266:in `context'
rack (2.2.8) lib/rack/session/abstract/id.rb:260:in `call'
actionpack (7.0.7) lib/action_dispatch/middleware/cookies.rb:704:in `call'
actionpack (7.0.7) lib/action_dispatch/middleware/callbacks.rb:27:in `block in call'
activesupport (7.0.7) lib/active_support/callbacks.rb:99:in `run_callbacks'
actionpack (7.0.7) lib/action_dispatch/middleware/callbacks.rb:26:in `call'
actionpack (7.0.7) lib/action_dispatch/middleware/debug_exceptions.rb:28:in `call'
actionpack (7.0.7) lib/action_dispatch/middleware/show_exceptions.rb:29:in `call'
logster (2.13.1) lib/logster/middleware/reporter.rb:40:in `call'
railties (7.0.7) lib/rails/rack/logger.rb:40:in `call_app'
railties (7.0.7) lib/rails/rack/logger.rb:27:in `call'
config/initializers/100-quiet_logger.rb:20:in `call'
config/initializers/100-silence_logger.rb:29:in `call'
actionpack (7.0.7) lib/action_dispatch/middleware/remote_ip.rb:93:in `call'
actionpack (7.0.7) lib/action_dispatch/middleware/request_id.rb:26:in `call'
lib/middleware/enforce_hostname.rb:24:in `call'
rack (2.2.8) lib/rack/method_override.rb:24:in `call'
actionpack (7.0.7) lib/action_dispatch/middleware/executor.rb:14:in `call'
rack (2.2.8) lib/rack/sendfile.rb:110:in `call'
actionpack (7.0.7) lib/action_dispatch/middleware/host_authorization.rb:131:in `call'
rack-mini-profiler (3.1.1) lib/mini_profiler.rb:260:in `call'
message_bus (4.3.8) lib/message_bus/rack/middleware.rb:60:in `call'
lib/middleware/request_tracker.rb:233:in `call'
railties (7.0.7) lib/rails/engine.rb:530:in `call'
railties (7.0.7) lib/rails/railtie.rb:226:in `public_send'
railties (7.0.7) lib/rails/railtie.rb:226:in `method_missing'
rack (2.2.8) lib/rack/urlmap.rb:74:in `block in call
rack (2.2.8) lib/rack/urlmap.rb:58:in `each'
rack (2.2.8) lib/rack/urlmap.rb:58:in `call'
unicorn (6.1.0) lib/unicorn/http_server.rb:634:in `process_client'
unicorn (6.1.0) lib/unicorn/http_server.rb:739:in `worker_loop'
unicorn (6.1.0) lib/unicorn/http_server.rb:547:in `spawn_missing_workers'
unicorn (6.1.0) lib/unicorn/http_server.rb:143:in `start'
unicorn (6.1.0) bin/unicorn:128:in `<top (required)>'
vendor/bundle/ruby/3.2.0/bin/unicorn:25:in `load'
vendor/bundle/ruby/3.2.0/bin/unicorn:25:in `<main>'
ENV
|REQUEST_URI|/webhooks/sendgrid|
|REQUEST_METHOD|POST|
|HTTP_USER_AGENT|SendGrid Event API|
Seems to be the same as here, which was a fixed issue a while back: Sendgrid error: NoMethodError (undefined method `[]' for nil:NilClass) - #3 by Terrapop
pfaffman
(Jay Pfaffman)
November 12, 2023, 1:34pm
2
Are those generated for bounces from your list that are not discourse users?
KCAS
November 12, 2023, 3:59pm
3
Not sure about if they’re bounces, how could we check? It’s most likely around the time we did a bulk invite to non-discourse users.
selase
(Selase Krakani)
November 13, 2023, 11:21am
4
There might be something going on with the event payload being received from Sendgrid. It seems status
isn’t being set on the bounce
events as expected.
Assuming you got the backtrace from logster at <YOUR_SITE>/logs
, the env tab should have the request details. Could you provide a sanitized copy of the payload from there?
KCAS
November 13, 2023, 12:35pm
5
There’s not a lot of info there, most of it i’ve put here.
For the full reference, i’ve put it below as requested
hostname
serverhostname
process_id
248749
application_version
a8d6dc4d3a5c3d937ff0bb162c93ec628428cda1
HTTP_HOST
forum.URL.co.uk
REQUEST_URI
/webhooks/sendgrid
REQUEST_METHOD
POST
HTTP_USER_AGENT
SendGrid Event API
HTTP_X_FORWARDED_FOR
IPADDRESS
HTTP_X_REAL_IP
IPADDRESS
time
Su 5:54 pm
selase
(Selase Krakani)
November 13, 2023, 1:23pm
6
I am sorry, I should have been more specific. The request body has the clues needed. You should be able to find this after the time
row. You may have to scroll up or expand panel to see this.
KCAS
November 13, 2023, 1:58pm
7
Hello,
There is no further information after the Time row, i’ve checked all of the 34 of the errors. Seems odd however it’s not included?
selase
(Selase Krakani)
November 14, 2023, 12:02pm
8
Yes, it’s odd. Could you retrieve the payload from the SendGrid dashboard instead?
Per the latest SendGrid webhooks docs, the current endpoint in Discourse should work without issues but the error points to an issue with the request payload so sample will be great.
cultiv
(Sebastiaan Janssen)
February 23, 2025, 8:48am
9
Did you ever figure this one out? I have the same error, on the same line of code:
"Received a Sendgrid webhook, but no verification key has been configured. This is unsafe behaviour and will be disallowed in the future.",
)
end
events = params["_json"] || [params]
events.each do |event|
message_id = Email::MessageIdService.message_id_clean((event["smtp-id"] || ""))
to_address = event["email"]
error_code = event["status"]
if event["event"] == "bounce"
if error_code[Email::SMTP_STATUS_TRANSIENT_FAILURE]
process_bounce(message_id, to_address, SiteSetting.soft_bounce_score, error_code)
else
process_bounce(message_id, to_address, SiteSetting.hard_bounce_score, error_code)
end
elsif event["event"] == "dropped"
process_bounce(message_id, to_address, SiteSetting.hard_bounce_score, error_code)
end
end
success
I am not familiar with Ruby but it looks like it can parse the json just fiine up until trrying to parse the error code, Email::SMTP_STATUS_TRANSIENT_FAILURE
points to:
I checked in Webhook.site what Sendgrid actually sends when I test the webhook and it looks like this for a bounce:
[
{
"email": "example@test.com",
"timestamp": 1740136261,
"smtp-id": "<14c5d75ce93.dfd.64b469@ismtpd-555>",
"event": "bounce",
"category": [
"cat facts"
],
"sg_event_id": "ovGQ2rRo8ytNezHPDq-7Ig==",
"sg_message_id": "14c5d75ce93.dfd.64b469.filter0001.16648.5515E0B88.0",
"reason": "500 unknown recipient",
"status": "5.0.0"
}
]
Seems like it should work!
cultiv
(Sebastiaan Janssen)
February 23, 2025, 9:02am
10
Aha… answering my own question here, the Webhook “Test integration” button lies to you…
Here is the actual payload if I actually send an email to a non-existing email address:
[
{
"bounce_classification": "Unclassified",
"email": "noemail@this.does.not.exist.tld",
"event": "bounce",
"reason": "unable to get mx info: failed to get IPs from PTR record: lookup <nil>: unrecognized address",
"sg_event_id": "Ym91bmNlLTQtNTA0ODUxOTUtZXVvMmlLeGRTYXlQRjRZRTQtLUk3QS0w",
"sg_message_id": "euo2iKxdSayPF4YE4--I7A.recvd-5f54b5d587-pczjm-1-67BADEEA-6.0",
"smtp-id": "<870b3a2a-160c-4fc8-bc9a-bd0d5b943b81@forum.umbraco.com>",
"timestamp": 1740300320,
"tls": 0,
"type": "blocked"
}
]
And there we have it: no status
field.
This is when I purposely use a domain that does not exist, I feel like this should be handled, perhaps "type": "blocked"
can be the indicator Discourse looks for.
As a different attempt I put some nonsense in front of outlook.com
and that does give me a working payload:
[
{
"bounce_classification": "Invalid Address",
"email": "oeihoiwehitwiohtriuweruiwerwierhwuerguiwerg@outlook.com",
"event": "bounce",
"ip": "149.72.1.78",
"reason": "550 5.5.0 Requested action not taken: mailbox unavailable (S2017062302). [HK3PEPF0000021E.apcprd03.prod.outlook.com 2025-02-23T08:54:35.950Z 08DD502499E1A0AA]",
"sg_event_id": "Ym91bmNlLTAtNTA0ODUxOTUtQWJFZ2pVejZUUFd3MnJNTnJabDg4Zy0w",
"sg_message_id": "AbEgjUz6TPWw2rMNrZl88g.recvd-786d47b7ff-tsp86-1-67BAE249-D.0",
"smtp-id": "<d8dc253e-6e9a-4418-8478-60780eb4898c@forum.umbraco.com>",
"status": "5.5.0",
"timestamp": 1740300876,
"tls": 1,
"type": "bounce"
}
]
tgxworld
(Alan Tan)
February 25, 2025, 5:17am
12
@cultiv Thank you for diving into the problem here and taking the time to submit a PR. I’ve used your PR as a base and merged a fix for the issue in FIX: No method error in `WebhooksController#sendgrid` (#31495) · discourse/discourse@209d289 · GitHub .
2 Likes
cultiv
(Sebastiaan Janssen)
February 25, 2025, 2:08pm
13
Fantastic, that was very helpful. Wouldn’t have been able to do all of that on my own
Thanks, I’ve updated and a manual test now does things correctly, error log is clean and I can see the email bounced in the mail log
1 Like