Moving Discourse Docker install to other VPS with 1:1 sync


(Michał Frąckiewicz) #1

This method is different than restoring file in UI.
Maybe there is a more downtime but it’s simpler for linux folks without Discourse knowledge and it’s easy to automate.

Prepare new VPS

First, prepare our new Ubuntu 14.04 VPS with empty Discourse dir, OS upgrade and Docker install

sudo mkdir -p /var/discourse
sudo apt-get update
sudo apt-get dist-upgrade
wget -qO- https://get.docker.com/ | sh

Then, make sure that we are on latest installed kernel and other libraries by just restarting

sudo reboot

Copy files to new VPS

Let’s copy all Discourse files, this first sync pass will reduce our downtime later.

On our new VPS let’s run

rsync -rvz example.com:/var/discourse/ /var/discourse/

where example.com is our old VPS with forum files.
This may take a while so grab some favourite drink.

Turn off current forum and sync again

After first sync pass, turn off forum on old VPS via

cd /var/discourse/
./launcher stop app

Now when forum is stopped, let’s run sync on new VPS again

rsync -rvz --delete example.com:/var/discourse/ /var/discourse/

--delete makes sure that will be 1:1 copy

Start forum on new host

After final sync we can finally run Discourse.

Rebuild and start container.
This may take a few minutes.

./launcher rebuild app

Done!
All settings, posts, threads, backups were copied to new location, no UI interaction was needed.


#2

This should really be:
rsync -rvz example:com:/var/discourse /var

Otherwise, you’ll end up with a discourse directory under /var/discourse.

Update: After doing this, all images in Assets for Site Design are now broken. Any ideas, what’s going on? @codinghorror Here’s our site: https://discuss.dgraph.io/


(Michał Frąckiewicz) #3

True, my mistake, forgot to add slashes,
Edited and fixed :slight_smile:

Did you modified links config somehow?


#4

No. Didn’t modify any configs. All the previous assets don’t load. But any new ones that we add work just fine. The files are definitely there.


(Michał Frąckiewicz) #5

Check permissions and owner of assets.


#6

So, the permissions are okay, owner ubuntu and group www-data. The real problem I just identified was that I think I must have changed the settings for the static resources to upload to Amazon S3. So, all the new resources are going there, and that’s where discourse is probably reading them from.

So, it’s no longer looking at the stored data in /var/discourse/shared/standalone/uploads.

Btw, so 2 things:

  • Is using S3 for static resources not recommended?. I understand it’s not deprecated, but checking to ensure we’re following the recommended setup for discourse.
  • If it is recommended, what’re my options? I suppose I could just upload all the existing assets on disk to S3. Then, we should be able to retrieve them just like before. Right?

(Michał Frąckiewicz) #7

You have an answer then :slight_smile:

I think this questions belong to other topics.


(Stephen Chung) #8

OK, not sure if the instructions here are out-dated or anything.

I have a forum that I mucked up the O/S. So, like a good cloud citizen, I spun up a brand new shiny Ubuntu 17.10 VM and followed the above instructions.

First thing that I found out immediately is that I need to be root to rsync the files. The rsync command came back with a pile of permission denied errors without copying anything.

Turns out that /var/discourse is owned by root (thanks to sudo mkdir -p /var/discourse in the instructions). Nothing a little hack won’t fix, so I did sudo chown 777 /var/discourse and did the rsync again.

This time it copied most of the files (but unfortunately all owned by me instead of root… probably needs a chown somewhere down the line).

The PostgreSQL files won’t copy and came back as permission denied. I checked the directories on the remote machine and lo and behold those directories are accessible only by root, not by mere mortals.

Using sudo on the rsync didn’t work either because it then asked for the password for root on the remote machine, which is also Ubuntu and thus root doesn’t have a password.

At the end, I gave up and installed a brand new Discourse installation then restored from backup.

I think the above instructions need serious rework especially wrt file ownerships and permissions.


(Michael Howell) #9

To perform an rsync as root on the sender and as ubuntu on the receiver, put ubuntu@ I’m front of the host, like this:

rsync -rvz ubuntu@example.com:/var/discourse /var

(Stephen Chung) #10

Will that copy the entire tree under the same owner as well as the same permissions?

I suppose rsync preserves permissions, but will it work if all the files are copied under root? I can see that the PostgreSQL data files are owned by some other user etc. And obviously the uploads and backups directories are owned by the admin user, not root.


(Michael Howell) #11

It won’t be able to preserve the permissions in this case, because ubuntu can’t just change the files it be owned by root.


(Stephen Chung) #12

In this case I wonder whether, after copying the directory tree, it will just work?

Next time I muck up something and have to move to a fresh VM, I’ll try it out! :sweat_smile:


(Andrew Waugh) #13

I’m not exactly an rsync guru, but why not copy from as root and to as root, and use the -p switch to preserve permissions?


(Stephen Chung) #14

Apparently, rsync with -g and -o flags, while running as root, can preserve owners and groups. -p with rsync preserves permissions.

So technically speaking… we’ll want rsync -rvzgop ?


(Michael Howell) #15

The VM is set up to not allow you to SSH in as root directly.


(Andrew Waugh) #16

Both of them? i.e. can you “pull” as root from the node which you can’t login directly to as root?

(Like I wrote, I’m not the sharpest knife in the drawer.)


(Stephen Chung) #17

Apparently it may be possible: