Migrating old Discourse to the new Dockerball :)

(versvs) #1


I’m facing a problem that has been tangentially treated in a previous thread, but to which I can’t seem to find a solution.

I was running a legacy Discourse install (coming back from version 0.5-0.6 or so, but updated up to if i remember well), installed using the then-fancy Ubuntu install guide. My server died mercilessly yesterday. I have backups… but not the kind of modern backups that Discourse sports today (no dashboard dump file), instead I have a postgresql dump and a lot of files.

As I intend to start from scratch using Docker now that destiny has given me the chance ( :tada: ), my problems are centered in getting back the public/uploads into a container. At the same time, I can restore my database into Postgresql, but I don’t know how to make my containers use it.

In the previously linked thread we can find some solution that is in part what I need (as it delves into the problem of reconverting old installs into docker installs) and in part exactly what I dont need, as my server is dead and I don’t have a running instance to which I can do this git/rake work.

I managed to get a docker install alive and running in, it is completely blank, any ideas on how may i get my database and uploaded files loaded and rake:migrated to the latest numbered version so I can do a proper dashboard export and reimport in the now recommended-easy way?

Thank you all in advance,

(Jens Maier) #2

I’d set up a development environment (i.e. outside of a Docker container).

git clone https://github.com/discourse/discourse.git
cd discourse
git checkout v0.9.8.5 -b old
bundle install

Restore your database dump and uploaded files, then

git checkout master
bundle install
rake db:migrate

Dump this version of the database with pg_dump and tar up your uploaded files. Install a production Discourse in a docker container, copy the database dump and uploads to /var/docker/shared (on the host), then ssh into the container. Once inside, your files should be available under /shared. Restore the database and uploads (extract the files as the discourse user, not root), restart the container, sacrifice a ritual chicken and pray to the god of bits. :wink:

(versvs) #3

Thanks for your suggestions @elberet.

I will proceed to try them now, but maybe you can remove a couple of interrogants I still have.

I checked the schema_migrations in the database and I found that my schema is updated up to 20140109205940, which I just googled and landed me in this commit :smiley:


and consequently to this release…

I read this morning the post by @sam explaining the basics of Docker, but how do I know the name of the discourse user or database inside the docker? What’s the purpose of checking the old branch? (is that what i’m doing with the -b oldright?).

Excuse me for my questions, I just know the basics… And today I’m learning about a lot of new things, hehehe

(Jens Maier) #4

git checkout v0.9.8.5 tells git to move your working directory to the tag v0.9.8.5 which I picked because you mentioned that version number in your post. The -b old part of the command tells it create a new branch in your local repository. It’s optional, but without it git will complain that you’re working in “detached HEAD mode”.

Inside the container, most of the names are straight forward:

  • The user who owns all of Discourse’s files is called discourse. It doesn’t have a password; when you ssh into the container you are root and can su - discourse if you want to run interactive commands as that user.
  • In the default install, a PostgreSQL server is started inside the container (but its data files are stored in /shared which is mapped to /var/docker/shared on the host). As usual, the (production environment) database is named discourse and I believe that Postgres is configured to use peer authentication.

However, please keep in mind that on my Gentoo box things will almost definitely look a bit different then on your server. I’m happy to share my experience and answer questions where I can, but I’m neither psychic nor a replacement for something like man git-checkout:wink:

(versvs) #5

I know the basics of git, I just didnt think of the -b old trick :slight_smile: This tip is going to be helpful a lot of times i guess… thanks :slight_smile:

The version I know you adapted to my testimony, that part is ok :slight_smile:

Thanks again!

(versvs) #6

i came back after a short while and now I can say I’m almost there. I followed the steps in #2 and now I have a development site that has everything correctly placed.

I just need to get the files available when I’m in the container. Right now I copy the files into /var/docker/shared but they dont appear visible when I ssh into the container. I tried chown-ing the files to both root and my local user, to no avail.

For what I read in the forums, there is nothing more to it: you put the files in /var/docker/shared and they should be there when you go to /shared after ssh-ing to the container. In this case, I’m sure I missed one important step somewhere… he he.

(Jens Maier) #7

Out of curiosity, can you post the output of ls -la /shared from within the container and ls -la /var/docker/shared from the host? :smile:

(versvs) #8

Sure, there we have it :slight_smile:


root@maelstrom-app:~# ls -la /shared/
    total 40
    drwxr-sr-x 10 root   134 4096 May 31 15:43 .
    drwxr-xr-x 74 root  root 4096 Jun  1 08:20 …
    drwxr-sr-x  2 discourse www-data 4096 May 31 15:43 backups
    drwxr-sr-x  3 root   134 4096 May 31 15:43 log
    drwxr-sr-x  2 postgres  postgres 4096 May 31 15:43 postgres_backup
    drwx------ 15 postgres  postgres 4096 Jun  1 08:20 postgres_data
    drwxrwsr-x  2 postgres  postgres 4096 Jun  1 08:20 postgres_run
    drwxr-sr-x  2 redis redis4096 Jun  1 09:00 redis_data
    drwxr-sr-x  4 discourse www-data 4096 May 31 15:46 uploads
    drwxr-sr-x  3 root   134 4096 May 31 15:44 vendor_bundle

Host :slight_smile:

versvs@maelstrom:/var/docker$ ls -la shared/
    total 7440
    drwxrwsr-x  3 versvs docker    4096 jun  1 02:40 .
    drwxrwsr-x 10 root   docker    4096 may 31 17:32 …
    -rw-r–r--  1 versvs versvs 6065884 jun  1 02:40 discourse_dev.sql
    -rw-rw-r--  1 versvs docker       0 may 31 17:32 .gitkeep
    drwxr-sr-x 10 root   docker    4096 may 31 17:43 standalone
    -rw-r–r--  1 versvs versvs 1539722 jun  1 02:40 uploads.zip

(Jens Maier) #9

Oh, I see. It’s the standalone folder. :wink: Move your files there.

(versvs) #10

thanks again :slight_smile: I got to restore the db but I cant seem to be able for the discourse inside the docker to understand that this has changed :S

(versvs) #11

Finally I surrendered and did as suggested here:

I already had the dev environment up-to-date thanks to your assistance, so I did a regular export/restore in the new site. I was fearing some settings would be lost… and certainly the site didnt load the old settings (like site name, or url of the custom logo, and some gravatars), but everything else was OK, and I fixed the uploads manually ssh-ing into the container.

Thank you guys :slight_smile:

(Jens Maier) #12

Hm, yeah, in retrospect I probably overcomplicated the last step a bit. :sweat_smile: