Container rebuild is failing with permission error


(Edward Nigma) #1

So I set up my first Discourse instance back in mid February. based on the all-in-one standalone Docker image. If I recall correctly the version was 1.3.0 beta1. I was in a bit of a hurry at the time and since I didn’t have anything else running on the server I just let Discourse grab external port 80. Fast forward to yesterday and I finally decided that I needed to move Discourse to a different port so that I could put up an instance of Nginx outside of Docker and run some other web apps on the same VM. I made a copy of /var/discourse to a backup folder, stopped the container, edited app.yml to change the port, and then ran launcher rebuild app. And that’s when things went horribly wrong leading to my post here. I’m thinking that the rebuild is updating my instance to 1.3.0 beta4 which includes some updates to the Postgresql instance within the container. That’s where an error is occurring. A script is calling a chown on /shared/postgres_run and chown is returning a permission denied error. That then leads to a failed bootstrap. I’m an experienced *nix admin, but very new to Docker and Discourse so I’ve had no luck fixing this myself. Hopefully someone will have a good solution to this. I’m including the launcher output below.

[root@ip-172-31-32-99 discourse]# ./launcher rebuild app
which: no docker.io in (/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin)
Updating discourse docker
remote: Counting objects: 133, done.
remote: Compressing objects: 100% (34/34), done.
remote: Total 133 (delta 34), reused 16 (delta 16), pack-reused 83
Receiving objects: 100% (133/133), 34.83 KiB | 0 bytes/s, done.
Resolving deltas: 100% (69/69), done.
From https://github.com/discourse/discourse_docker
   dfdd027..80a6f24  master     -> origin/master
Updating dfdd027..80a6f24
Fast-forward
 .gitignore                          |   1 +
 README.md                           |   5 +-
 image/base/Dockerfile               |   4 +-
 image/base/runit-1                  |   2 +-
 image/base_22/Dockerfile            |  16 +++++
 image/build.rb                      |  52 +++++++++++++++
 image/discourse/Dockerfile          |   4 +-
 image/discourse_22/Dockerfile       |  21 ++++++
 image/discourse_test/Dockerfile     |  13 +---
 launcher                            | 296 +++++++++++++++++++++++++++++++++++++++++++++------------------------------------
 samples/data.yml                    |  12 +++-
 samples/standalone.yml              |  17 ++++-
 samples/web_only.yml                |   1 -
 templates/postgres.9.2.template.yml |   5 ++
 templates/postgres.template.yml     |  21 ++++--
 templates/web.socketed.template.yml |  12 ++++
 16 files changed, 318 insertions(+), 164 deletions(-)
 create mode 100644 image/base_22/Dockerfile
 create mode 100644 image/build.rb
 create mode 100644 image/discourse_22/Dockerfile
 create mode 100644 templates/web.socketed.template.yml
Stopping old container
Error response from daemon: no such id: aae4fd0785cc774fe4af708dfce5de170d5952fabb27bfe18a073932deac8811
FATA[0000] Error: failed to stop one or more containers
Calculated ENV: -e LANG=en_US.UTF-8 -e RAILS_ENV=production -e UNICORN_WORKERS=3 -e UNICORN_SIDEKIQS=1 -e RUBY_GC_MALLOC_LIMIT=40000000 -e RUBY_HEAP_MIN_SLOTS=800000 -e DISCOURSE_DB_SOCKET=/var/run/postgresql -e DISCOURSE_DB_HOST= -e DISCOURSE_DB_PORT= -e HOME=/root -e DISCOURSE_DEVELOPER_EMAILS=jaseymour@gmail.com -e DISCOURSE_HOSTNAME=discourse.guiltypleasurez.net -e DISCOURSE_SMTP_ADDRESS=172.31.32.99 -e DISCOURSE_SMTP_PORT=25
cd /pups && git pull && /pups/bin/pups --stdin
From https://github.com/SamSaffron/pups
   ee8fdfb..4203610  master     -> origin/master
Updating ee8fdfb..4203610
Fast-forward
 README.md | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
I, [2015-04-03T17:05:19.306027 #42]  INFO -- : Loading --stdin
I, [2015-04-03T17:05:19.311509 #42]  INFO -- : > mkdir -p /shared/postgres_run
I, [2015-04-03T17:05:19.313762 #42]  INFO -- :
I, [2015-04-03T17:05:19.314112 #42]  INFO -- : > chown postgres:postgres /shared/postgres_run
chown: changing ownership of ‘/shared/postgres_run’: Permission denied
I, [2015-04-03T17:05:19.316571 #42]  INFO -- :


FAILED
--------------------
RuntimeError: chown postgres:postgres /shared/postgres_run failed with return #<Process::Status: pid 45 exit 1>
Location of failure: /pups/lib/pups/exec_command.rb:105:in `spawn'
exec failed with the params "chown postgres:postgres /shared/postgres_run"
636e1e19598654f74f512c0640201409b23a24565f0f7b94256da4b6b32bd634
FAILED TO BOOTSTRAP

(Jeff Atwood) #2

No changes to postgres that I know of. Did you do anything non-standard or deviate from our install guide in any way?


(Edward Nigma) #3

I don’t believe that I did. I’ve been trying to treat this like a black box until I get more familiar with Docker in a sandbox.


(Edward Nigma) #4

Now that I think about it there is another change that is relevant. Earlier this week I ran a yum update and I recall seeing Docker upgraded from 1.3.x to 1.5.0. I don’t know Docker well enough to know if that would matter.


(Kane York) #5

Here’s a relevant error, I think:

Error response from daemon: no such id: aae4fd0785cc774fe4af708dfce5de170d5952fabb27bfe18a073932deac8811
FATA[0000] Error: failed to stop one or more containers

(Edward Nigma) #6

I think that’s left over from making a backup copy of /var/discourse before shutting down the container. When I’ve tried to bootstrap again that error doesn’t appear anymore.


(Edward Nigma) #7

Assuming for the moment that this isn’t fixable for some strange reason, how would I go about transferring data from this broken Discourse instance to a fresh instance?


(Kane York) #8

Well, that rather involves having it working :stuck_out_tongue:

Unless it was making daily backups?


(Edward Nigma) #9

It was not unfortunately. But I do have the original data files and I think I might be able to start it again if I free up port 80 for it. I’ll have to test that again.


(Kane York) #10

Actually, have you seen this topic? (Sounding like a broken record here…)

Note that the first step is “have a working Discourse install” :stuck_out_tongue:


(Edward Nigma) #11

Heh, I did when I started the process. Things stopped working when I shut down Discourse to change the http port and performed a rebuild. Or at least attempted to perform a rebuild.


(Kane York) #12

I see that you pulled down web.socketed.template.yml, you can use that and not even use any ports at all :slight_smile:


(Edward Nigma) #13

So I’m hanging my head in shame this morning as well as swearing up a blue cloud. I managed to solve my problem this morning. I have one word: SELinux. For whatever reason it was ok with me creating my Discourse instance in the first place, but wasn’t down with updating it. I’ve switched SELinux mode from enforcing to permissive and all is right with my world again. Thanks for the feedback everyone.


(Kane York) #14

I think what you’re supposed to do is this:


(Srinivas Nidadavolu) #15

what is the issue here
Error response from daemon: No such image: zeppelinhub:latest


(Joshua Rosenfeld) #16

@Srinivas_Nidadavolu, I’m sorry but I do not understand your issue. The link you shared is not a Discourse forum…