A few days ago I upgraded my Discourse from some version I don’t know, however I strongly believe (but can’t swear on it) it was a 2.4 beta, to currently 2.4.0.beta4.
Recently I noticed that e-mail sending is no longer working, I have a lot of failed jobs in Sidekiq. The error for all those jobs is: “Jobs::HandledExceptionWrapper: Wrapped OpenSSL::SSL::SSLError: SSL_connect returned=1 errno=0 state=error: unsupported protocol”.
My e-mail settings point to an old mail server of mine (which otherwise works fine for all clients, of different kinds), port 587 with plain authentication and enable_starttls_auto set to true. It has worked fine since I set up Discourse earlier this year, so I’m pretty sure that it’s since the latest upgrades that it stopped working. The operating system has not changed/been upgraded during this time, nor has the mail server.
I have read Discourse Version 2.4 and am not seeing anything mail or OpenSSL related there.
Q1: Where can I tell from which version the last upgrade and the one before that was done, so I can track the versions I’ve been using?
Q2: Where can I find more specific timestamps for when the mail jobs started failing? I clicked a job in Sidekiq and it tells me it was created two days ago, which I think is in line with when I did the upgrade. But I’d like to verify that mail jobs didn’t fail before that.
Q3: Presumably something relating to OpenSSL changed in the version I started running (relative to the one I was running before). What could this have been, and is there a setting anywhere that I can tweak? Or should I try to downgrade? Or what is there any way that I can get additional from the job processing so I can see what protocol it’s complaining about?
root@foo-app:/# openssl s_client -connect mail.foo.com:587 -starttls smtp
CONNECTED(00000003)
139861698753664:error:1425F102:SSL routines:ssl_choose_client_version:unsupported protocol:../ssl/statem/statem_lib.c:1922:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 320 bytes and written 353 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---
root@foo-app:/#
root@foo-app:/#
root@foo-app:/# openssl s_client -connect mail.foo.com:587 -starttls smtp -tls1_1
CONNECTED(00000003)
140427988595840:error:1425F102:SSL routines:ssl_choose_client_version:unsupported protocol:../ssl/statem/statem_lib.c:1922:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 320 bytes and written 174 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1.1
Cipher : 0000
Session-ID:
Session-ID-ctx:
Master-Key:
PSK identity: None
PSK identity hint: None
SRP username: None
Start Time: 1568985038
Timeout : 7200 (sec)
Verify return code: 0 (ok)
Extended master secret: no
---
root@foo-app:/#
root@foo-app:/#
root@foo-app:/# openssl s_client -connect mail.foo.com:587 -starttls smtp -tls1_2
CONNECTED(00000003)
140184139936896:error:1425F102:SSL routines:ssl_choose_client_version:unsupported protocol:../ssl/statem/statem_lib.c:1922:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 320 bytes and written 258 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1.2
Cipher : 0000
Session-ID:
Session-ID-ctx:
Master-Key:
PSK identity: None
PSK identity hint: None
SRP username: None
Start Time: 1568985044
Timeout : 7200 (sec)
Verify return code: 0 (ok)
Extended master secret: no
---
root@foo-app:/#
root@foo-app:/#
root@foo-app:/# openssl s_client -connect mail.foo.com:587 -starttls smtp -tls1_3
CONNECTED(00000003)
write:errno=0
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 262 bytes and written 278 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---
root@foo-app:/#
I’m am seeing an error message “protocol unsupported” when forcing TLS other than 1.3 (using additional arguments to the openssl command, see below), and “write:errno=0” only when forcing TLS 1.3. With your command (not forcing TLS) I’m not seeing any specific version being used/attempted.
If someone could answer questions #1 and #2 it would be helpful. I need to consider if there’s a way to downgrade this ASAP, and in order to do that I need to know what the last working version was.
Please edit /var/discourse/launcher and replace the base image version (image="discourse/base:2.0.20190906-0522") in line 91 with image="discourse/base:2.0.20190625-0946"
Thank you @gerhard! Your suggestion solved the problem. I changed the base image version, rebuild, and straight away the forum started sending the queued e-mails (about 10k :D).
I did run the commands in the container again and am getting other (successful) output. This output contains certificates and a bunch of other stuff, so unless you really need it I’d prefer to not paste it in here. Let me know if this is a problem and you really need it.
I will PM you the hostname of the mail server so you can debug this issue in more detail - please keep it to yourself Thanks!
Edit: I’m marking your last post as a solution because it solved the problem. However obviously we need to identify what’s causing it in the newer base image, so one can update in the future.
As a workaround, you should be able to add sed commands to the run section at the end of app.yml to remove the following two settings from the /etc/ssl/openssl.cnf file.
Thanks!