this is what I did:
I added the ssh key to ssh-agent:
ssh-add /path/to/private/key
I forwarded the agent to the container:
./launcher rebuild app --docker-args "--volume $SSH_AUTH_SOCK:/ssh-agent --env SSH_AUTH_SOCK=/ssh-agent"
Worked like a charm.