Custom Docker-Discourse bootstrapping from private GIT repository

Product Support Self-hosting

AI-generated summary

The discussion revolves around bootstrapping a custom Docker-Discourse image from a private GIT repository using SSH key authentication.

RabidFX initially faces the issue of the repository requiring SSH key authentication, which fails during bootstrapping. sam suggests determining the command-line solution and then applying it to the correct hook.

RabidFX attempts to leverage Pups’ automatic import of environment variables as params but encounters difficulties. They later discover that the solution lies in passing the environment variable through Docker args without setting the variable, allowing it to default to the host value.

rking proposes using a file directive to drop the private key file into the container. Radu_Luncasu shares a working solution by adding the SSH key to the ssh-agent and forwarding the agent to the container, which resolves the issue.

1

Hello all o/

I’m working on a custom theme for Discourse. Everything works well in the Vagrant VM, and it’s now time to try a “real” install.
I pulled the standard Docker image and modified the templates so the bootstrapping would pull from my fork of the discourse/discourse repo, and not the original.

Thing is, the repo I will pull from uses SSH key authentication for the all users, including git.

As expected, it fails on bootstrapping with the following message:

I, [2015-07-06T12:25:59.424104 #36]  INFO -- : > cd /var/www/discourse && git fetch custom-origin release                                                       
Host key verification failed.                                                                                                                                
fatal: Could not read from remote repository.

How can I pull from a private repo expecting a key when bootstrapping?

If the alternative is easier: this repo is hosted on the docker host machine itself. So I could also read directly from the container in the host filesystem… But I assume this is harder than SSHing in, this being a possible Docker security hole…

2

Step 1

Figure out how to do that from command line

Step 2

Plug in what you type in command line into the correct hook

All hooks are named in templates

3

Thank you @sam , I went the little extra effort and started hacking a solution myself. However I’m partially stuck at setp 1.

The most logical would be to give the container an already pre-authorized SSH key. I’m trying to leverage Pups’ automatic import of environment variables as params. I saved the key in such a variable, and modified the sshd.template.yml in the following fashion:

Shell command

discourse@machine:/var/discourse$ export GIT_PRIVATE_KEY=`cat /home/discourse/.ssh/id_rsa`

sshd.template.yml (last two lines of this block are mine)

  - exec:                                                                                                                                                    
      hook: sshd                                                                                                                                             
      cmd:                                                                                                                                                   
        - mkdir -p /var/run/sshd                                                                                                                             
        - mkdir -p /root/.ssh                                                                                                                                
        - echo "$ssh_key" >> /root/.ssh/authorized_keys
        - echo "$$ENV_SSH_PUB_KEY" >> /root/.ssh/authorized_keys                                                                                             
        - echo "$GIT_PRIVATE_KEY" >> /root/.ssh/id_rsa                                                                                                      
        - cat /root/.ssh/id_rsa

However, woth syntaxes (with one or two $ did not yield the expected result.

(I know the sshd template and web ones are not called in the correct order by default for this to work :wink: But the cat command itself should produce the expected output, which it doesn’t.)

Edit - Environment variable insanity.

I realized pups is run in the container, not in the host, so this would be the container environment variable. I tried passing it through Docker args, but it fails. The variable is not properly escaped, even with the added quotes.

discourse@machine:/var/discourse$ ./launcher bootstrap app --docker-args "-e GIT_PRIVATE_KEY=$GIT_PRVATE_KEY" 
Unable to find image 'RSA:latest' locally                                                                                                                    
Invalid repository name (RSA), only [a-z0-9-_.] are allowed                                                                                                                                                                                              
cd /pups && git pull && /pups/bin/pups --stdin                                                                                                               
User args:                                                                                                                                                   
GIT_PRIVATE_KEY="-----BEGIN RSA PRIVATE KEY----- MIIEowIBAAKCAQEAuRquuVBtx06RM9/bU104PLPzxX0fHmjkIILvYyXe/h44yyMo (snip) b/njJsNjvo9zGfhomcs2QOh8Yb7jIzooHqI6ApSUOhu7e0+uZ3Lu -----END RSA PRIVATE KEY-----"                                   
Unable to find image 'RSA:latest' locally                                                                                                                    
Invalid repository name (RSA), only [a-z0-9-_.] are allowed                                                                                                  
cat: cids/app_bootstrap.cid: No such file or directory                                                                                                       
docker: "rm" requires a minimum of 1 argument.                                                                                                               
See '/usr/bin/docker rm --help'.                                                                                                                             
                                                                                                                                                             
Usage: docker rm [OPTIONS] CONTAINER [CONTAINER...]                                                                                                          
                                                                                                                                                             
Remove one or more containers                                                                                                                                
** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one

Edit 2 - Key apparently passed to pups.

The solution was, surprisingly, simpler, and in Docker’s manual.

discourse@machine:/var/discourse$ ./launcher bootstrap app --docker-args "-e GIT_PRIVATE_KEY"

By not setting the variable, it defautls to the value of the same variable on the host.
It shows in pups’ logs, but authentication is still refused. I will try with a different certificate soon, but it might be mishandled whitespace problem still. If push somes to shove, I’ll lower the security to a simple passphrase.

Don’t hesitate if you have an idea.

4

You could use a file: directive to drop the private key file into the container.

Also, you are using a deploy (read-only) key, right?

5

Вот что я сделал:

Я добавил SSH-ключ в ssh-agent:

ssh-add /path/to/private/key

Перенаправил агент в контейнер:

./launcher rebuild app --docker-args "--volume $SSH_AUTH_SOCK:/ssh-agent --env SSH_AUTH_SOCK=/ssh-agent"

Сработало как по маслу.