Emails failing to send, images not working, onebox issues and other strange things after updating

[Richie (Richie Rich)]

I update our Discourse once a week so we’re never too far behind and this morning I updated to 2026.7.0-latest +182 GitHub

Everything seemed ok but we’re seeing some odd issues in the last hour or two with images not appearing correctly.

eg:

If I right-click and open the image in a new window, it’s accessible ok on our AWS S3 at https://d1q8cepst0v8xp.cloudfront.net/original/3X/8/4/84ade505efc54e3fce3515a02cadcec55f2d8e4f.png

As you’ll see here:

We’re also seeing onebox previews not working anymore.

The onebox thing is weird, it worked in the composer:

But when posted, no image was visible:

---

We’re also seeing links to images not actually displaying the images:

---

A bigger issue and concern is that all emails are now failing to send.

Sidekiq is showing a “Jobs::HandledExceptionWrapper: Wrapped Net::OpenTimeout: execution expired” error.

I updated Discourse around six hours ago and emails do appear to have gone out since the updates, and only seem to have stopped working two hours or so ago.

Puzzling, to say the least.

We use AWS SES for email.

---

Here’s a screenshot of the /logs screen:

And a screenshot from htop:

---

For reference, we have a separate website on a different domain which uses the same AWS SES and emails are going out just fine from there.

That same website also uses AWS S3 and we’re able to upload fine there too.

If I didn’t know any better I’d say Amazon were having some issues, but everywhere I look I can find no issues reported.

Is anyone else having unexplained issues?

Does anyone know where I might start in trying to diagnose these issues? Are there any services or processes I could restart via rails or something?

The biggest concern is no email going out, new user activations, lost passwords, all being queued up.

[Richie (Richie Rich)]

Strangely enough and perhaps a total red herring, but these issues only seem to have started (or at least been noticed), when I turned off the new “ME TOO” thing, and altered these other site settings two hours ago.

This can’t be related, surely…

[Richie (Richie Rich)]

@Richie

Everything seemed ok but we’re seeing some odd issues in the last hour or two with images not appearing correctly.

eg:

I’m seeing a pattern.

When the image is posted, it’s visible ok.

Five minutes later when the post gets baked (is that the right terminology), the image disappears.

---

None of which explains why all emails have suddenly started failing though

[Richie (Richie Rich)]

Update…

I took my forum offline and did another time ./launcher rebuild app which seems to have now fixed every single issue.

Any ideas what might have happened here?

Could a background service have randomly failed or something?

[Andrew_Rowe]

@Richie

these issues only seem to have started (or at least been noticed), when I turned off the new “ME TOO” thing, and altered these other site settings two hours ago.

at that time did you try running in ‘safe mode’ to make sure it wasn’t a theme component or plugin not playing nice with the updated version?

[Moin]

Wouldn’t safe mode also disable the upcoming change because it’s part of the solved plugin?

[Richie (Richie Rich)]

@Andrew_Rowe

did you try running in ‘safe mode’

I didn’t, no.

Only because it didn’t cross my mind.

But that also wouldn’t have resolved all the emails suddenly stopping?

I’ve since turned off the “ME TOO” thing again (interesting choice of button-name, given the #MeToo movement) and nothing has fallen over yet so a possible red herring.