Hosted-by-discourse is using expired SSL certificates yet status page is green

This is breaking our site at the moment, all posts here seem to be about self hosted sites.

❯ openssl s_client -connect (removed).com:443 -servername (removed).com
CONNECTED(00000005)
depth=1 O = Digital Signature Trust Co., CN = DST Root CA X3
verify error:num=10:certificate has expired
notAfter=Sep 30 14:01:15 2021 GMT
verify return:0
depth=1 O = Digital Signature Trust Co., CN = DST Root CA X3
verify error:num=10:certificate has expired
notAfter=Sep 30 14:01:15 2021 GMT
verify return:0
depth=3 O = Digital Signature Trust Co., CN = DST Root CA X3
verify error:num=10:certificate has expired
notAfter=Sep 30 14:01:15 2021 GMT
verify return:0
---
Certificate chain
 0 s:/CN=(removed).com
   i:/C=US/O=Let's Encrypt/CN=R3
 1 s:/C=US/O=Let's Encrypt/CN=R3
   i:/C=US/O=Internet Security Research Group/CN=ISRG Root X1
 2 s:/C=US/O=Internet Security Research Group/CN=ISRG Root X1
   i:/O=Digital Signature Trust Co./CN=DST Root CA X3
---

API end points are returning an expired certificate error for us, verified through postman.

@team please help here

Hmm. This seems weird

This does sound like an issue on your side (i.e. the side of your website).
The certificate provided by the server is fine and it is accepted by almost all browsers - the problem is that a lot of older servers do not have the (relatively) new root certificate in their trust store so a lot of automated stuff is throwing errors at the moment.

I don’t know your exact site so I took an arbitrary hosted-by-discourse.com site and ran it through the Qualys SSL Server Test.

You can see that root of the second certification path is indeed expired, but this is mitigated by the ISRG Root X1 certificate being in the trust store (i.e. included in your browser and/or operating system) for the first certification path.

Servers typically do not update their trust store often so your server probably does not have the ISRG Root X1 certificate in its trust store. (That is what happened to the mail-receiver Docker image as well).

You can find a current bundle on for instance https://curl.se/ca/cacert.pem. On CentOS that file goes into /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem. On Ubuntu you can use the update-ca-certificates command which updates /etc/ssl/certs/ca-certificates.crt.

Alternatively, you could temporarily disable SSL certificate verification in your server code.

4 Likes

I posted the output of openssl to prove it is nothing to do with our website, that and postman is run from a local windows machine.

edit: i’m not as knowledgable about the different levels of trust stores, i’ll look into it some more, it just happens from every machine we try.

It is not failing when you use an actual browser, correct?

EDIT more people having issues with Postman so I guess that’s not a valid test… :thinking:

1 Like

this is a reply I have from our devops guy.


The problem is, there are two copies of ISRG Root X1 - one of them was issued by DST Root CA X3 a long time ago, and the newer one was self-signed by the ISRG (the group that governs Let’s Encrypt, among other things) and that self-signed Root CA is now accepted as a trusted Root CA on most systems

They are the same signing certificate, so you can validate anything that claims to be issued by ‘ISRG Root X1’ against either version but the one that was signed by DST is no longer valid as a certification path because the DST certificate that signed it is now expired

If the server was doing the right thing and supplying the newer ISRG Root X1 that’s self-signed, then the client would compare the supplied one against its own trust store, verify that it’s the same cert, and be OK but instead it sees the supplied cert, goes “Ok I need to validate this against DST Root CA X3 that I have on file… oops that’s expired” and fails verification

1 Like

Supplying a certificate in the certificate chain that is expected to be in the trusted root CA store is very much not “doing the right thing”.

Let’s take a look at the certificate chain from letsencrypt.org itself: SSL Server Test: letsencrypt.org (Powered by Qualys SSL Labs)

Hey, look. They’re doing the same, the certificate chain is identical (except for the server cert of course).

So either your devops guy is wrong, or CDCK and LetsEncrypt and Qualys are wrong.
Tell him to update the root CA store. It will resolve the issue. Believe me.

1 Like

Hi @bpaterson2000 - sorry to hear you’re having problems connecting to our hosted service. As @RGJ mentioned, this isn’t something wrong on the Discourse end - it needs to be fixed on your client. (Our servers don’t ‘supply’ the root certificate, they are managed by your client)

You can find more details from Let’s Encrypt on the change here:

4 Likes

Thanks for the info, it is an unexpected issue for us as the errors were apparently coming back from the hosted server.

We use heroku which bundles the stuff you are talking about via node/packages apparently - and it looks like updating node was the key to getting it successfully connecting to the discourse rest API successfully again.

Thanks for your help.

3 Likes