Cannot backup (connect to postgres) in docker_dev

(Jay Pfaffman) #1

Using the docker_dev you can’t make a backup from the web interface or from within the container using ./script/discourse backup. Both fail due to postgres permissions issues. This makes using the container to do an import problematic.

What I’ve been doing is editing /etc/postgresql/9.5/main/pg_hba.conf and changing the connections from “peer” to “trust”, like this:

local   all             postgres                                trust
local   all             all                                     trust

And then restarting the container, as I’ve yet to figure out how to restart postgres from within it.

This likely isn’t the best solution to this problem, and I don’t even know where to look to fix it if I did.

And while I’m complaining, it would be nice if the container included libmysqlclient-dev

(Sam Saffron) #2

Feel free to submit a couple of PRs to improve this.

(Jay Pfaffman) #3

Can you give me a hint where that happens? I’m not clear where that container gets built. Its not in discourse/discourse, is it?

(Sam Saffron) #4

The container is built here:

Sorry for being somewhat abrupt I am very :airplane: :mantelpiece_clock:ed

(Jay Pfaffman) #5

Welcome home, or away. :slight_smile:

So I’d fix up ~/src/discourse_docker/image/discourse_dev/postgres_dev.template.yml and submit the PR and then after you accepted the PR and rebuilt the image, I could test whether it really worked, right?

If that’s the process, then I think I could contrive to modify ~/src/discourse/bin/docker/boot_dev to pull my version of that image to test it.

I’ve got three imports running right now. I think that I may end up just spinning up Digital Ocean droplets to run them, but this could be one way to manage running multiple imports at once on a single machine. Or maybe I should be cranking up a VM in VirtualBox. . .

(Jay Pfaffman) #6

Well, this isn’t quite the specific problem that I was dealing with, but FIX: reset_db by pfaffman · Pull Request #5617 · discourse/discourse · GitHub gets bin/docker/reset_db to work.

(Jay Pfaffman) #7

Sorry to be dense, but how is the container built?

I was happy to learn that image/ says

To build a new set of images, update the `Makefile` with the new version number, and then `make all`. 

Sounds simple. Sadly, there is no Makefile to be found.

Then I thought that I could just build with

cd images; docker build discourse_dev

(And I tried running that from some other directories…) but that fails with

Step 6/17 : ADD redis.template.yml /pups/redis.yml
ADD failed: stat /var/lib/docker/tmp/docker-builder410839948/redis.template.yml: no such file or directory

I’m running out of ideas.

(Sam Saffron) #8

Oh we probably want to update this, I think build.rb in the repo takes care of it these days.

(Jay Pfaffman) #9

OK. That’d be great. Per the above, I wanted to do something like this:

I wanted to add this to Dockerfile:

# Add libmysqlclient-dev for running imports
RUN apt-get update && apt-get install -y libmysqlclient-dev

And also, the database stuff in the OP. I don’t fully understand the security implications of the trust that I suggested, but I can’t get backups to work without the change. This is what I’m doing to run an import in discourse_dev

rm /etc/apt/sources.list.d/google-chrome.list
apt update
apt -y install libmysqlclient-dev
sed -i -e 's/peer/trust/g' /etc/postgresql/9.5/main/pg_hba.conf
docker restart discourse_dev
time bundle exec ruby /src/script/import_scripts/vbulletin.rb # for example