Issues with Discourse 3.5.0.beta2-dev - SMTP and Background Jobs

Bug
1

The following report was prepared for me by ChatGPT, based on my actual experience installing Discourse (not being very tech savvy myself, I’ve being relying on AI for help with tasks that I find difficult). I don’t need any help for now but I hope that the report may prove to be useful feedback about 3.5.0.beta2-dev.

==================

Dear Discourse Team,

I want to share some troubleshooting details about email issues I faced while running Discourse 3.5.0.beta2-dev. I don’t need direct assistance since I’ve decided to revert to 3.4.0.beta4-dev, but I hope this report is useful for identifying potential issues with the latest development version.

1. Problem Summary

I performed a clean installation of Discourse on a new Vultr instance, using 20i’s SMTP service (smtp.stackmail.com ). However, emails were never delivered, despite:

  • A successful SMTP connectivity test (openssl s_client -connect smtp.stackmail.com:587 -starttls smtp ).
  • No errors appearing in 20i’s mail logs (suggesting Discourse wasn’t actually sending emails).
  • Sidekiq failing to process email jobs.

This was unexpected because a few weeks ago, I successfully installed Discourse 3.4.0.beta4-dev on an identical instance without any email issues.

After thorough debugging, I discovered that my current installation had unexpectedly installed Discourse 3.5.0.beta2-dev, which likely contributed to the problems.

2. Key Issues Identified

A. Emails Were Not Being Delivered

  • SMTP settings were correct, verified via manual testing (swaks and openssl tests succeeded).
  • Emails were enqueued in Sidekiq but never received.
  • 20i’s mail logs showed no rejection or delivery attempts (suggesting the messages never left Discourse).
  • No email-related errors appeared in production.log (grep "smtp" returned nothing).

B. Sidekiq and Redis Issues

  • Initially, Sidekiq was not running (Sidekiq::Workers.new.size returned 0 ).
  • Restarting Sidekiq manually (sv restart sidekiq ) failed because Sidekiq was missing as a service.
  • Redis was running (redis-cli ping returned PONG ), but Discourse logs still showed Redis connection errors (Errno::ECONNREFUSED ).
  • Sidekiq logs indicated job failures due to Redis connection issues, even though Redis was confirmed to be active.
  • Manually starting Sidekiq (bundle exec sidekiq ) allowed jobs to process, but emails still did not send.

C. Version Mismatch Between Installations

  • My previous installation (which worked) was on 3.4.0.beta4-dev.
  • My current installation unexpectedly installed 3.5.0.beta2-dev, despite using the same setup method.
  • Since email worked perfectly in 3.4.0.beta4-dev, I suspect a regression or breaking change in 3.5.

3. Actions Taken (All Failed)

We systematically tried the following solutions:

:white_check_mark: Confirmed SMTP connectivity with:

openssl s_client -connect smtp.stackmail.com:587 -starttls smtp  # Successful
swaks --to my-email --server smtp.stackmail.com --port 587 --auth LOGIN  # Successful

:white_check_mark: Verified that emails were enqueued:

Jobs.enqueue(:user_email, type: :test_message, to_address: 'masden@kumagaku.ac.jp')  # Returned a job ID
Sidekiq::Queue.new("default").size  # Returned 0 (indicating the job was processed)

:white_check_mark: Checked Discourse logs:

grep "smtp" /shared/log/rails/production.log  # No SMTP-related errors
  • Redis connection failures (Errno::ECONNREFUSED in production.log ).

:white_check_mark: Restarted and manually triggered services:

sv restart sidekiq  # Failed, service missing
redis-cli ping  # Confirmed running
bundle exec sidekiq  # Jobs processed but no emails sent

:white_check_mark: Checked if emails were being blocked by 20i:

  • No rejections or delivery attempts in 20i mail logs.
  • If emails were being rejected, there should be a log entry, but nothing appeared.

4. Conclusion: Rolling Back to 3.4.0.beta4-dev

After extensive troubleshooting, I have decided to wipe the installation and reinstall 3.4.0.beta4-dev, since my previous install worked flawlessly on that version.

While I don’t need direct help, I wanted to report these issues because:

  • Email handling might have changed in 3.5, preventing SMTP handoff.
  • Sidekiq’s missing service and Redis errors suggest background job processing problems in 3.5.
  • Since I was able to install 3.4.0.beta4-dev without issues, this suggests a possible regression in 3.5.

I hope this report is helpful for identifying potential issues in Discourse 3.5.0.beta2-dev.

Best regards,
Kirk

2 Likes
2

That was expected, because it is nowadays version.

Quite many is using same latest version, and sidekiq works, and therefore emails too.

So you might need help, because then you can’t rebuild either.

And much better knowing may take over this topic, but I would say we have here topics about stopped sidekiq. The search may help.

4 Likes
3

I’m continuing to have difficulties installing Discourse. Here’s another report put together by ChatGPT. I believe that it accurately represents my experience.

Report: Issues with Discourse Installation & Missing Sidekiq Service

Context:
We attempted a fresh installation of Discourse on a Vultr instance, intending to set up a stable and working deployment. However, we encountered significant issues, particularly with missing services such as Sidekiq, which prevented email delivery and background job processing.

Summary of Issues Encountered

  1. Unexpected Version Installed
  • Instead of the expected v3.4.0.beta4-dev, the installation defaulted to tests-passed, pulling a potentially unstable version.
  • The VERSION file was missing, making it unclear what exact version was installed.
  1. Sidekiq Service Missing
  • The /etc/service/sidekiq directory was absent inside the Discourse container.
  • This prevented all background jobs (emails, notifications, scheduled tasks) from running.
  1. Discourse Git Repository Issues
  • Running git rev-parse --abbrev-ref HEAD inside the container returned tests-passed, confirming that an unintended version was installed.
  • Git threw a “detected dubious ownership” error, requiring manual intervention (git config --global --add safe.directory /var/www/discourse).
  1. Potential Dependency Issues
  • Even if an older version of Discourse is checked out, there is concern that newer dependencies (Ruby, Redis, Sidekiq) may be pulled during installation, potentially causing compatibility issues.
  • If Discourse dependencies are not pinned correctly, the installation may behave inconsistently across environments.
  1. SSL Certificate Rate Limiting
  • An attempt to obtain a new Let’s Encrypt SSL certificate failed due to the rate limit being exceeded.
  • This caused Nginx to fail to start, as it could not load the expected certificate file.

Planned Next Steps

  1. Perform a Full Wipe & Reinstall
  • Remove /var/discourse entirely and re-clone the repository.
  • Manually check out a stable version (e.g., v3.4.1) before running the installation.
  • Use ./discourse-setup instead of relying on defaults to ensure correct parameters.
  1. Ensure Sidekiq is Installed
  • Before rebuilding, verify that the sidekiq service is correctly included in the build process.
  • If Sidekiq is missing after the rebuild, manually check its installation via bundle list | grep sidekiq.
  1. Pin Dependencies to Stable Versions
  • Avoid dependency-related issues by explicitly using a known-stable Discourse Docker image (e.g., discourse/discourse:2.0.20240101).
  • Lock gem versions inside the container (bundle install --deployment --without test development).
  1. Reattempt SSL Certificate Issuance
  • Wait for the Let’s Encrypt rate limit reset and retry SSL certificate generation.
  • If issues persist, consider using a self-signed certificate temporarily for troubleshooting.

Request for Feedback

Given these challenges, we would appreciate input from the Discourse team and community regarding:

  • Sidekiq missing from /etc/service/ in a fresh install – Has anyone else encountered this issue?
  • Best practices for ensuring dependency stability – Is there a recommended way to pin dependency versions for Discourse installations?
  • Potential issues with tests-passed being installed by default – Could there be an issue with how versions are being retrieved?

Any insights would be helpful before we proceed with the reinstall. Thanks in advance!

4

It’s not actually unstable. It’s the default for all forums now. Discourse won’t put an undtable branch as the default.

Hmm… what do you see when you go to /sidekiq in your forum?

Which branch are you intending to install on? Perhaps this guide will help?

3 Likes
5

I think whatever may need is some details on your VPS configuration. Cpu, ram, disk space and ziS ie Ubuntu LTS 24

When you say you tested mail. Did you use the discourse test and receive an email?

In my XP in your app.yml file you should have 3 lines of main config for the SMPT.

  • Username:
  • Password
  • PORT

Now from what you said you had this working in an earlier discourse install?

As mentioned “Tests-Passed” is the recommended version to install. Stable is from my understanding more for ppl with custom plugins & themes - which could require updating more often to avoid breakage.

Can you share a rebuild log with errors? Remember to scroll up when capturing log as final error is just a failed result. You can always redact sensitive data like SMPT user and password type details.

There are also some topics on email SMPT issues that may have solutions

1 Like
7

Sorry for my late response. As I wrote in response to another comment, I’m not able to respond with details about my last failed attempt because I have erased much of the software in order to try again. I’ll report back as soon as I can.

8

Thanks for your detailed comments. I apologize for my slow response.

I’m attempting another install with earlier versions of various software. That makes it difficult for me to answer your questions about my previous failed attempt. As soon as I can complete my next installation, I’ll report back.

1 Like
9

Something to consider as an extra test if needed. Setup a free account on https://www.brevo.com; they have a free mail tier of 300 emails per day. I use this one ATM.

If you have trouble sending mail, this could help test if it might be related to your mail provider.

2 Likes
10

TL;DR don’t rely on ChatGPT, these are all false alarms.

It’s good to ask for help, it’s a bad idea to rely on it, as I will show below.
Thank you for reassuring I won’t be made jobless by ChatGPT soon. :wink:

beta4-dev is as potentially unstable as tests-passed

There is no such file.

There should not be such a directory.

I doubt that.

This is expected.

This is normal behaviour if you use git on a directory while not being the owner.

This does not make sense. And tests-passed is newer than what you ‘expected’, not older.

This is because you tried too many times.

6 Likes
11

I want to emphasize on this.
Most of the time, if you suggest chatGPT that there are issues to find, it will find issues, even if there aren’t. It’s also quick to make us go into an infinite loop of “fixes” it continuously suggests.

5 Likes
12

Thanks very much!

In regard to AI, it’s clear that ChatGPT has not been a reliable guide. On the other hand, I’ve found debugging to be difficult so the temptation to rely on it is great for someone like me.

I just completed a successful installation. ChatGPT (paid version) couldn’t get me to the goal but Claude from Anthropic (also paid version; I use both) finally did. It helped me fix issues that ChatGPT had introduced.

I’ll try to write back with more about my experience later. Thanks for all of your helpful comments!

2 Likes
13

I agree. From my usage, Claude is much more comfortable and usually gives an immediate correct answer.

I’m glad you succeeded!

5 Likes
14

Thanks! It’s remarkable how different they can be for certain tasks – almost like people with different personalities. Claude seems to be more “thoughtful” whereas ChatGPT tends to jump to conclusions and run out in front a bit more. I’ve been in situations where ChatGPT is better, though. I’m hoping they BOTH improve. AI has helped me write a lot of scripts that actually work, even though I’m not a script writer. It has the potential to be a great tool for tech-challenged people like me but it definitely needs to improve. It’s really frustrating to be led down a dead end. :frowning:

4 Likes
15

Here’s a brief report on my successful installation with a question about how to move forward.

I’m still not sure why I had trouble with SMTP in recent attempts. Actually, my first attempt a few weeks ago was successful but I was using a Vultr server that I decided was bigger than I needed and the only way to downgrade was to get a new smaller server and then erase the bigger one.

According to an email I received, the version of Discourse I installed a few weeks ago was 3.4.0.beta4-dev. The email was recommending that I upgrade to 3.4.0.beta4.

Since I had trouble with SMTP in 3.4.0.beta4-dev I thought I would attempt to install an earlier stable version of Discourse. In discussion with ChatGPT (unreliable, I know) we decided to try to install 3.4.1. I thought that version would be stable. Here’s what we wound up installing:

• Discourse Version: 3.5.0.beta2-dev
• Docker Version: 23.0.6
• sidekiq: 6.5.12
• PostgreSQL Version: PostgreSQL 15.12 (Debian 15.12-1.pgdg120+1)
• Redis Version: 7.0.15
• NGINX: 1.26.2
• Ubuntu Version: 22.04 LTS (Jammy Jellyfish)
• Git Version: 2.39.5

I think that regardless of our intent (that is, my intent assisted by AI) to install earlier stable versions, we wound up installing Discourse 3.5.0.beta2-dev after all.

It was difficult (lots of false starts and using commands given to me by AI to fix stuff that wasn’t working) but my Discourse is finally up and running.

Here are some questions:

  1. Unless I’m mistaken, current users are not yet being encouraged to upgrade to 3.5.0, presumably because it’s not yet fully tested. If so, why are newbies like me more-or-less forced to install it?

  2. I think my version of Docker is old (deprecated). Should I just use the terminal to upgrade to the latest version? Discourse is working now and since I’ve had so much trouble getting to this point, I don’t want to do anything to mess things up.

  3. I think the other software versions are fairly new. If you see anything that may be problematic or that needs upgrading please let me know.

  4. Is there a page or section of this community that addresses technical “care and maintenance” issues? I want to keep my community healthy and working and be prepared with backups, etc. for any problems that might come up.

2 Likes
16

I just completed a restoration from backup of the previous Discourse community. It seemed to go well and it may have updated Docker (issue mentioned above) as well:

Screenshot 2025-03-02 at 9.55.48
Screenshot 2025-03-02 at 9.55.48670×179 14.4 KB

I wonder about this message, which I receive immediately after the restoration: “Outgoing email has been disabled for non-staff users.”

Referring to another thread here

I found where to change the setting.

Screenshot 2025-03-02 at 10.04.03
Screenshot 2025-03-02 at 10.04.03710×149 6.56 KB

So, now I assume that I’m good to go but if possible I’d appreciate confirmation. “no” is the proper setting, right? I think regular users have to get email notifications so I’m a little puzzled by the disabling setting. I wonder what sorts of situations would require mail to be disabled.

17

I had a similar issue with email not sending on a brand new Discourse installation which I detailed here:

The issue was that the notifications email address in app.yml had not been authenticated with my SMTP provider (SendGrid).

Nothing appeared in the main Discourse logs, nor in the SendGrid logs. The error was surfaced when running discourse-doctor:

Reason: 550 The from address does not match a verified Sender Identity.

If you haven’t already done so I’d suggest running discourse-doctor as it may provide more insight into why emails aren’t being sent.

2 Likes
18

Thanks very much!!

Excellent advice. I was unaware of the Discourse-doctor but I think it may have saved me a lot of time.

I’ll keep it in mind if I run into other problems. :slight_smile:

1 Like