Discourse docker images are not signed

Installation
1

Hello,

I am currently trying to set up Discourse on a new machine. I checked out the discourse_docker repository and ran ./launcher rebuild app. (I skipped bootstrapping because I had configurations from before).

Rebuilding resulted in the following output:

WARNING: We are about to start downloading the Discourse base image
This process may take anywhere between a few minutes to an hour, depending on your network speed

Please be patient

ERRO[0001] Metadata for targets expired
ERRO[0002] Metadata for targets expired
/usr/bin/docker: Error: remote repository docker.io/discourse/base out-of-date: targets expired at Mon Nov 18 12:52:09 -0500 2019.
See '/usr/bin/docker run --help'.
Your Docker installation is not working correctly

See: https://meta.discourse.org/t/docker-error-on-bootstrap/13657/18?u=sam

To get to the root of the problem I checked the command that is run and ran it in debug mode:

/usr/bin/docker --debug run -i --rm -a stdout -a stderr discourse/base:2.0.20191219-2109 echo working

See: https://github.com/discourse/discourse_docker/blob/1b3dd3a41b544592be46b39db563bb757f88bbad/launcher#L217

The output of the command: 
DEBU[0000] reading certificate directory: /root/.docker/tls/notary.docker.io
DEBU[0000] No yubikey found, using alternative key storage: no library found
DEBU[0000] Making dir path: /root/.docker/trust/tuf/docker.io/discourse/base/changelist
DEBU[0000] entered ValidateRoot with dns: docker.io/discourse/base
DEBU[0000] found the following root keys: [5f5ba22b20768b8f2d49e9035400a1922ae761aa70b5bb0d930fe91dd06bd5d8]
DEBU[0000] found 1 valid leaf certificates for docker.io/discourse/base: 5f5ba22b20768b8f2d49e9035400a1922ae761aa70b5bb0d930fe91dd06bd5d8
DEBU[0000] found 1 leaf certs, of which 1 are valid leaf certs for docker.io/discourse/base
DEBU[0000] checking root against trust_pinning config for docker.io/discourse/base
DEBU[0000] checking trust-pinning for cert: 5f5ba22b20768b8f2d49e9035400a1922ae761aa70b5bb0d930fe91dd06bd5d8
DEBU[0000]  role has key IDs: 5f5ba22b20768b8f2d49e9035400a1922ae761aa70b5bb0d930fe91dd06bd5d8
DEBU[0000] verifying signature for key ID: 5f5ba22b20768b8f2d49e9035400a1922ae761aa70b5bb0d930fe91dd06bd5d8
DEBU[0000] root validation succeeded for docker.io/discourse/base
DEBU[0000] entered ValidateRoot with dns: docker.io/discourse/base
DEBU[0000] found the following root keys: [5f5ba22b20768b8f2d49e9035400a1922ae761aa70b5bb0d930fe91dd06bd5d8]
DEBU[0000] found 1 valid leaf certificates for docker.io/discourse/base: 5f5ba22b20768b8f2d49e9035400a1922ae761aa70b5bb0d930fe91dd06bd5d8
DEBU[0000] found 1 leaf certs, of which 1 are valid leaf certs for docker.io/discourse/base
DEBU[0000] checking root against trust_pinning config for docker.io/discourse/base
DEBU[0000] checking trust-pinning for cert: 5f5ba22b20768b8f2d49e9035400a1922ae761aa70b5bb0d930fe91dd06bd5d8
DEBU[0000]  role has key IDs: 5f5ba22b20768b8f2d49e9035400a1922ae761aa70b5bb0d930fe91dd06bd5d8
DEBU[0000] verifying signature for key ID: 5f5ba22b20768b8f2d49e9035400a1922ae761aa70b5bb0d930fe91dd06bd5d8
DEBU[0000] root validation succeeded for docker.io/discourse/base
DEBU[0000] updating TUF client
DEBU[0000] Loading timestamp...
DEBU[0001] 200 when retrieving metadata for timestamp
DEBU[0001] timestamp role has key IDs: db679e1f17a2a2bdd9952b6f37c9048635611f162bc6a0957ec30fbb1412d366
DEBU[0001] verifying signature for key ID: db679e1f17a2a2bdd9952b6f37c9048635611f162bc6a0957ec30fbb1412d366
DEBU[0001] timestamp role has key IDs: db679e1f17a2a2bdd9952b6f37c9048635611f162bc6a0957ec30fbb1412d366
DEBU[0001] verifying signature for key ID: db679e1f17a2a2bdd9952b6f37c9048635611f162bc6a0957ec30fbb1412d366
DEBU[0001] successfully verified downloaded timestamp
DEBU[0001] Loading snapshot...
DEBU[0001] snapshot role has key IDs: e88a99d5b23301178976a4a103df4716f7943301d9972db0541bfa17c94cb75b
DEBU[0001] verifying signature for key ID: e88a99d5b23301178976a4a103df4716f7943301d9972db0541bfa17c94cb75b
DEBU[0001] successfully verified cached snapshot
DEBU[0001] Loading targets...
DEBU[0001] no targets in cache, must download
DEBU[0001] 200 when retrieving metadata for targets.09b50021560effa664eb8271cd16e2212ed55b7b63b2127cf9a0e69cfe2b47b7
DEBU[0001] targets role has key IDs: c016860a9293a48bc98f27b9c36d81515f4bcc3dd2ec93bb2d6d8af8f37a39d2
DEBU[0001] verifying signature for key ID: c016860a9293a48bc98f27b9c36d81515f4bcc3dd2ec93bb2d6d8af8f37a39d2
ERRO[0001] Metadata for targets expired
DEBU[0001] downloaded targets.09b50021560effa664eb8271cd16e2212ed55b7b63b2127cf9a0e69cfe2b47b7 is invalid: targets expired at Mon Nov 18 12:52:09 -0500 2019
DEBU[0001] Client Update (Targets): targets expired at Mon Nov 18 12:52:09 -0500 2019
DEBU[0001] Error occurred. Root will be downloaded and another update attempted
DEBU[0001] Resetting the TUF builder...
DEBU[0001] entered ValidateRoot with dns: docker.io/discourse/base
DEBU[0001] found the following root keys: [5f5ba22b20768b8f2d49e9035400a1922ae761aa70b5bb0d930fe91dd06bd5d8]
DEBU[0001] found 1 valid leaf certificates for docker.io/discourse/base: 5f5ba22b20768b8f2d49e9035400a1922ae761aa70b5bb0d930fe91dd06bd5d8
DEBU[0001] found 1 leaf certs, of which 1 are valid leaf certs for docker.io/discourse/base
DEBU[0001] found the following root keys: [5f5ba22b20768b8f2d49e9035400a1922ae761aa70b5bb0d930fe91dd06bd5d8]
DEBU[0001] found 1 valid leaf certificates for docker.io/discourse/base: 5f5ba22b20768b8f2d49e9035400a1922ae761aa70b5bb0d930fe91dd06bd5d8
DEBU[0001] found 1 valid root leaf certificates for docker.io/discourse/base: 5f5ba22b20768b8f2d49e9035400a1922ae761aa70b5bb0d930fe91dd06bd5d8
DEBU[0001]  role has key IDs: 5f5ba22b20768b8f2d49e9035400a1922ae761aa70b5bb0d930fe91dd06bd5d8
DEBU[0001] verifying signature for key ID: 5f5ba22b20768b8f2d49e9035400a1922ae761aa70b5bb0d930fe91dd06bd5d8
DEBU[0001] checking root against trust_pinning config for docker.io/discourse/base
DEBU[0001] checking trust-pinning for cert: 5f5ba22b20768b8f2d49e9035400a1922ae761aa70b5bb0d930fe91dd06bd5d8
DEBU[0001]  role has key IDs: 5f5ba22b20768b8f2d49e9035400a1922ae761aa70b5bb0d930fe91dd06bd5d8
DEBU[0001] verifying signature for key ID: 5f5ba22b20768b8f2d49e9035400a1922ae761aa70b5bb0d930fe91dd06bd5d8
DEBU[0001] root validation succeeded for docker.io/discourse/base
DEBU[0001] successfully verified cached root
DEBU[0001] retrying TUF client update
DEBU[0001] Loading timestamp...
DEBU[0002] 200 when retrieving metadata for timestamp
DEBU[0002] timestamp role has key IDs: db679e1f17a2a2bdd9952b6f37c9048635611f162bc6a0957ec30fbb1412d366
DEBU[0002] verifying signature for key ID: db679e1f17a2a2bdd9952b6f37c9048635611f162bc6a0957ec30fbb1412d366
DEBU[0002] successfully verified downloaded timestamp
DEBU[0002] Loading snapshot...
DEBU[0002] snapshot role has key IDs: e88a99d5b23301178976a4a103df4716f7943301d9972db0541bfa17c94cb75b
DEBU[0002] verifying signature for key ID: e88a99d5b23301178976a4a103df4716f7943301d9972db0541bfa17c94cb75b
DEBU[0002] successfully verified cached snapshot
DEBU[0002] Loading targets...
DEBU[0002] no targets in cache, must download
DEBU[0002] 200 when retrieving metadata for targets.09b50021560effa664eb8271cd16e2212ed55b7b63b2127cf9a0e69cfe2b47b7
DEBU[0002] targets role has key IDs: c016860a9293a48bc98f27b9c36d81515f4bcc3dd2ec93bb2d6d8af8f37a39d2
DEBU[0002] verifying signature for key ID: c016860a9293a48bc98f27b9c36d81515f4bcc3dd2ec93bb2d6d8af8f37a39d2
ERRO[0002] Metadata for targets expired
DEBU[0002] downloaded targets.09b50021560effa664eb8271cd16e2212ed55b7b63b2127cf9a0e69cfe2b47b7 is invalid: targets expired at Mon Nov 18 12:52:09 -0500 2019
DEBU[0002] Client Update (Targets): targets expired at Mon Nov 18 12:52:09 -0500 2019
/usr/bin/docker: Error: remote repository docker.io/discourse/base out-of-date: targets expired at Mon Nov 18 12:52:09 -0500 2019.
See '/usr/bin/docker run --help'.

I have no idea why this happens. I checked the git history and reverted to the previously used base image to check if that works. The result was the same.

See: https://github.com/discourse/discourse_docker/commit/1b3dd3a41b544592be46b39db563bb757f88bbad#diff-f3e08b5119358a8f58f17a678759f60c

Any help would be welcome :slight_smile:

Update #1

I have created a post in the Docker forum to discuss this problem as suggested by @Falco.

2

Can you update docker to the latest version available for your OS and try again?

2 Likes
3

Thank you for your reply @Falco.

I am running the latest version of Docker:

Docker version 19.03.5, build 633a0ea838

See: Docker Engine release notes | Docker Documentation

Sorry, for not mentioning this before :roll_eyes:

4

So a simple

docker pull discourse/base:2.0.20191219-2109

is failing?

5

Yes. This command fails the same way as described in my initial post:

ERRO[0001] Metadata for targets expired
ERRO[0002] Metadata for targets expired
Error: remote repository docker.io/discourse/base out-of-date: targets expired at Mon Nov 18 12:52:09 -0500 2019
6

That is very weird. Is your server clock right? Do you have any weird caching, firewall or something?

7

@Falco My clock seems to be synced:

$ timedatectl status

                      Local time: Fr 2019-12-27 19:57:48 CET
                  Universal time: Fr 2019-12-27 18:57:48 UTC
                        RTC time: Fr 2019-12-27 18:57:49
                       Time zone: Europe/Berlin (CET, +0100)
       System clock synchronized: yes
systemd-timesyncd.service active: yes
                 RTC in local TZ: no

I am running Ubuntu 18.04 LTS.

8

I also want to mention that other containers run just fine. So it seems to be somehow related to the Discourse image.

9

Well, I can docker pull discourse/base:2.0.20191219-2109 just fine, across many OSs, in different cloud providers in different continents so I’d say it’s something particular to your local computer/docker install.

As this comes from the docker service, try seeking help on https://forums.docker.com/

3 Likes
10

Will do @Falco. Thank you for your help so far :slight_smile:

11

@Falco I have created a post on the Docker forums and got a reply.

The problem is connected to DOCKER_CONTENT_TRUST=1.

To reproduce this issue you can run the following commands:

$ export DOCKER_CONTENT_TRUST=1
$ docker pull discourse/base:2.0.20191219-2109
12

Oh I see. Why did you set this flag on your local machine? It doesn’t come enabled by default in Ubuntu.

13

I am setting up a server on the internet. This is not on my local machine. The server is supposed to be “hardened” and that’s why this option is enabled.

14

It doesn’t come enabled by default in Ubuntu Server either.

15

That is correct @Falco . I enabled it. As far as I can tell the Discourse images are either not properly signed or not signed at all (I am currently investigating).

Either way, it would be desirable to have signed images/tags. I could imagine that I am not the only one that uses DCT.

It is part of auditing tools like Docker Bench Security (see below):

4.5  - Ensure Content trust for Docker is Enabled

For now I can deactivate it but it would be awesome to get a signed image/tag for the Discourse images as well :slight_smile:
https://github.com/docker/docker-bench-security

16

Should I create a feature request so that we can close this issue?

17

For now you’re stepping outside the instructions for a recommended install, so this will be tagged as unsupported, so that others don’t make the same mistake.

18

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.