Docker dev database ownership and strange path problems

There are a couple snags with docker_dev last I tried. The biggest is that I couldn’t connect to the database without entering the container and editing the postgres config file.

I was also unable to get reset_db to work. I was able to create a new database like this:

cd ~/src/discourse
docker stop discourse_dev # stop the development instance
sudo rm -rf data/postgres/ # delete the database
docker start discourse_dev # re-start the development instance
sleep 5
bin/docker/rake db:migrate # set up an empty database

What I’d really like to be able to do with it is to use it to run one import while I’m running (or debugging) another with my dev instance, but they share the same uploads directory.

Step 1 in supporting “I had issues with docker dev in the past” is going to be reproducing the issues today.

I can not really support a time machine.

4 Likes

Fair enough; When I experienced the problem I was rather frantically trying to get things to work a week or two back and thought it was my imagination or something spurious. Here’s what happened just now.

This is as user 1000, Ubuntu 17.10.

There is an error in boot_dev and then rails won’t crank up.

(The problem that I remember was that when I entered the container and tried to run an importer there was a database permissions problem.)

pfaffman@tenkay:~$ git clone https://github.com/discourse/discourse.git                                          
Cloning into 'discourse'...                                                                                      
remote: Counting objects: 293747, done.                                                                          
remote: Compressing objects: 100% (5/5), done.                                                                   
remote: Total 293747 (delta 0), reused 3 (delta 0), pack-reused 293742                                           
Receiving objects: 100% (293747/293747), 186.25 MiB | 2.06 MiB/s, done.                                          
Resolving deltas: 100% (185287/185287), done.                                                                    
Checking out files: 100% (24834/24834), done.                                                                    
pfaffman@tenkay:~$ bin/docker/boot_dev                                                                           
-bash: bin/docker/boot_dev: No such file or directory                                                            
pfaffman@tenkay:~$ cd discourse/                                                                                 
/home/pfaffman/discourse                                                                                         
pfaffman@tenkay:~/discourse$ bin/docker/boot_dev                                                                 
bin/docker/boot_dev: line 3: cd: $'/home/pfaffman/discourse/bin/docker\n/home/pfaffman/discourse/bin/docker': No 
such file or directory                                                                                           
Using source in:                                                                                                 
Using data in:   /data/postgres                                                                                  
f1818215c4ef7d551d00082c28087a038dc909507b40fb8c46b8f8d4244df39e                                                 
pfaffman@tenkay:~/discourse$ docker ps                                                                           
CONTAINER ID        IMAGE                             COMMAND             CREATED             STATUS             
 PORTS                                            NAMES                                                          
f1818215c4ef        discourse/discourse_dev:release   "/sbin/boot"        32 seconds ago      Up 32 seconds      
 0.0.0.0:1080->1080/tcp, 0.0.0.0:3000->3000/tcp   discourse_dev                                                  
9c0f6f3803db        linuxserver/duplicati             "/init"             5 weeks ago         Up 4 days          
 0.0.0.0:8200->8200/tcp                           duplicati                                                      
pfaffman@tenkay:~/discourse$ bin/docker/rails server -b 0.0.0.0                                                  
Usage:                                                                                                           
4 Likes

That looks an issue with:

SCRIPTPATH=$(cd "$(dirname "$0")"; pwd -P)
SOURCE_DIR=$(cd "$SCRIPTPATH" && cd ../.. && pwd -P)
DATA_DIR="$SOURCE_DIR/data/postgres"

Can you help debug this by making a bash script with, in the same dir as ./boot_dev

#!/bin/bash
SCRIPTPATH=$(cd "$(dirname "$0")"; pwd -P)
echo $SCRIPTPATH
SOURCE_DIR=$(cd "$SCRIPTPATH" && cd ../.. && pwd -P)
echo $SOURCE_DIR
DATA_DIR="$SOURCE_DIR/data/postgres"
echo $DATA_DIR

Also … work through the dirname there … the cd … and so on. SOURCE_DIR is not being set right. is pwd -P somehow returning a newline?

4 Likes

No. It’s stranger than that. cd is echoing, so SCRIPTPATH gets two directory names.

SCRIPTPATH=$(cd "$(dirname "$0")" > /dev/null; pwd -P)

fixed that. Not sure why that machine (laptop, ubuntu 17.10) was echoing the cd. It doesn’t on my desktop (Ubuntu 16.04).

But that wasn’t my problem “in the past”.

Trying to

bin/docker/shell
cd src
./script/discourse backup

fails with a database error (as in the examples below).

running bin/docker/rails s and doing a backup from the web interface also fails with:

[2017-12-14 20:53:31] pg_dump: [archiver (db)] connection to database "discourse_development" failed: FATAL: Peer authentication failed for user "postgres"

And doing bin/docker/bundle exec script/discourse backup also fails with the same error.

I’ve solved this by bin/docker/shell_root and editing /etc/postgresql/9.5/main/pg_hba.conf, changing “peer” to “trust” and restarting the container (one day I’ll figure out how to restart postgres from within the container).

If only I Would read my own notes. (this draft post on my site) describes how to fix the database problem with sed. (Oh! but it would be easier to fix it in data and not fool with entering the container.)

My guess is that most folks who use the docker development aren’t running importers, but it seems like a potentially fairly painless way to get a Normal Person to be able to run an import.

It looks like the problem I’ve had with reset_db was the same strange problem with SCRIPTPATH not getting set.

I had great intentions of submitting a PR with the two trivial bash changes, but got in another fight with git that I childishly let it win.

3 Likes

The bash crazy is fixed per: https://github.com/discourse/discourse/commit/32d881399fff12b5de0fc4f41d2a091a20ea76e7

5 Likes

This happens when you set the CDPATH env var. Maybe you did that by accident (check your command history to know for sure)

2 Likes

That’s it! It was no accident. I’m starting to see why people hate bash. :slight_smile:

1 Like

I ran into this same error… it’s a bit more obscure when you hit it during restore because of the errors about no metadata to extract and cannot restore into different schema make it less obvious that the real problem is FATAL: Peer authentication failed for user "postgres".

Any reason not to update data/postgres/pg_hba.conf method from peer to trust for the default dev image?

The config file has moved, now is:

/etc/postgresql/10/main/pg_hba.conf (inside container)
data/postgres/pg_hba.conf (on host)

When complete, restart the service in the container with:
sv restart postgres

Unfortunately, I’m still unable to complete a restore… After the create index are all complete I’m getting:

[2018-10-19 17:14:32] EXCEPTION: PG::InsufficientPrivilege: ERROR:  permission denied for relation schema_migrations
: SELECT "schema_migrations"."version" FROM "schema_migrations" ORDER BY "schema_migrations"."version" ASC

But when I run that query via ./bin/docker/psql I’m just getting an empty set, not a Postgres “Insufficient Priviliges” error…

discourse-# \c discourse_development ;
You are now connected to database "discourse_development" as user "postgres".
discourse_development-# SELECT "schema_migrations"."version" FROM "schema_migrations" ORDER BY "schema_migrations"."version" ASC
discourse_development-# \c discourse;
You are now connected to database "discourse" as user "postgres".
discourse-# SELECT "schema_migrations"."version" FROM "schema_migrations" ORDER BY "schema_migrations"."version" ASC

Maybe there’s something else I should change in pg_hba.conf? @sam