Move a Discourse site to another VPS with rsync

Both of them? i.e. can you “pull” as root from the node which you can’t login directly to as root?

(Like I wrote, I’m not the sharpest knife in the drawer.)

Apparently it may be possible:

2 Likes

I have a very active site I wanted to quickly move over so I decided to use this rsync method rather than the backup/restore approach which is the standard recommended one. (I am also out of disk space on the old node for backups…) There were a few glitches in the above guide so I wrote out my own steps for future reference and thought others might be interested.

The main things I changed from the above is to always do root-root ssh (both of my trees are owned by root), and to use -avz as the rsync parameters — the -a is the archive mode and will preserve dates on files as well. Without -a the second rsync to only update changed files will have to sync all the files again anyway, saving no time at all. With -a in both runs the --delete run of rsync should only take a few minutes.

So here is the plan

  1. Set up the new node as above or however you do it. Make sure you run discourse-setup to generate a default app.yml file which has good parameters for your new machine. (Make sure to set this file aside so step 5. and 8. below don’t overwrite it.)

  2. (24 hours in advance) On your DNS provider page set DNS TTL to a small value on all IPs that will move. The default is often 24 hours, set to 5 minutes. This will let users see the new site sooner. Set up new.example.com as a public name for the new site IP for easy access.

  3. enable root login on new site — make sure line
    # PermitRootLogin no
    in /etc/ssh/sshd_config is commented out on new, if not comment and re-start sshd -
    service ssh restart

  4. Set up ssh key for old-root to new-root login via the usual method (ssh-keygen in the home dir of root user on both sides, move old public key to authorized_keys on new, etc). You can of course use password login instead but it is less secure.

  5. Perform initial rsync while logged in to old (old forum can still be running):

    sudo rsync -avz /var/discourse new.example.com:/var/

  6. Post a message on board that example.com migration is starting in X minutes and site will be down for an hour or so. You could suggest they could temporarily use new.example.com if the site continues to not load. (The latter suggestion is for the cases where their DNS is not getting refreshed like it was supposed to). Note they will need to re-log-in on the new site so you may want to also mention that.

  7. sudo ./launcher stop app
    on old site to shut it down (permanently we hope).

  8. Followup rsync --delete run on old to get any changes since previous rsync:

    sudo rsync -avz --delete /var/discourse new.example.com:/var/

  9. edit app.yml for configuration changes on the new site. This is basically a merge of your old app.yml and the default one you made in step 1. above, which sets the cache, workers, etc parameters based on your new machine.

  10. sudo ./launcher rebuild app on new site to fire it up

  11. If it works, change DNS for all *.example.com (for me I also had mg.example.com for mailgun) to the new IP.

  12. (If it failed, sudo ./launcher start app on old site and try again later.)

  13. restore TTL on DNS.

  14. disable root login on new.

Note I am not running a CDN or https and there are may be other steps needed for those.

5 Likes

Unless CDN is using an ip number rather than the hostname there is nothing to change there.

There no reason not to enable let’s encrypt.

If the new host has different ram and cpu resources, discourse-setup will update the memory settings.

1 Like

On my old machine I was running three different web servers, thats the reason https was not enabled. Yes I know I could do it but it is a lot harder with proxy servers in the middle. Its also one reason I did the move, now I have a completely vanilla install so I can easily enable https. Vanilla is a good flavor :slight_smile:

Good point about discourse-setup updating various settings … I merged the new machine parameters into my old app.yml during step 9 above. The default app.yml also was slightly revised in other minor ways, and I could also incorporate those changes into my config. I did this before step 7 and put the file aside and plopped it in for step 9, avoiding the need to be contemplating settings while the site was down. Will edit the above to mention this.

The whole changeover (step 7 to having new site up) took 20 minutes and DNS was working for me 10 more minutes later.

1 Like

Hi!

I’m trying to follow the steps by @scottfsmith. I manage to get rsync done. It is not important to me to get the most recent changes via rsync since I’m just testing a new Linux version with my existing site to see if all my plugins work. So I’m not doing the second run of rsync. Then trying to do ./launcher rebuild app produces errors.

2022-12-13 14:43:01.974 UTC [59] LOG:  database system was interrupted; last known up at 2022-12-13 10:23:29 UTC
2022-12-13 14:43:02.075 UTC [59] LOG:  invalid primary checkpoint record
2022-12-13 14:43:02.075 UTC [59] PANIC:  could not locate a valid checkpoint record
2022-12-13 14:43:03.137 UTC [56] LOG:  startup process (PID 59) was terminated by signal 6: Aborted
2022-12-13 14:43:03.137 UTC [56] LOG:  aborting startup due to startup process failure
2022-12-13 14:43:03.231 UTC [56] LOG:  database system is shut down
I, [2022-12-13T14:43:06.699692 #1]  INFO -- : 
I, [2022-12-13T14:43:06.711862 #1]  INFO -- : > su postgres -c 'createdb discourse' || true
createdb: error: could not connect to database template1: could not connect to server: No such file or directory
	Is the server running locally and accepting
	connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
I, [2022-12-13T14:43:06.917008 #1]  INFO -- : 
I, [2022-12-13T14:43:06.917421 #1]  INFO -- : > su postgres -c 'psql discourse -c "create user discourse;"' || true
psql: error: could not connect to server: No such file or directory
	Is the server running locally and accepting
	connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
I, [2022-12-13T14:43:07.007654 #1]  INFO -- : 
I, [2022-12-13T14:43:07.008155 #1]  INFO -- : > su postgres -c 'psql discourse -c "grant all privileges on database discourse to discourse;"' || true
psql: error: could not connect to server: No such file or directory
	Is the server running locally and accepting
	connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
I, [2022-12-13T14:43:07.087098 #1]  INFO -- : 
I, [2022-12-13T14:43:07.087319 #1]  INFO -- : > su postgres -c 'psql discourse -c "alter schema public owner to discourse;"'
psql: error: could not connect to server: No such file or directory
	Is the server running locally and accepting
	connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
I, [2022-12-13T14:43:07.167221 #1]  INFO -- : 
I, [2022-12-13T14:43:07.168041 #1]  INFO -- : Terminating async processes

I can’t make enough out of this to find a solution. Some search suggest the container needs to be stopped but it’s not started. Any ideas?

Thanks
David

I would Set up a staging server to solve your particular problem.

From those errors, it looks like the database is broken. You need to stop the database to be able to get a valid set of data for it to work. The second rsync isn’t optional.

1 Like

Wow!

A four year old thread and a response within 3 minutes! :slight_smile:

Anyway, It’s basically a staging server I’m aiming at and using this rsync method. But do you recommend not doing it this way with rsync but using a backup? I recall not getting all Customize settings from a previous staging server I set up, but maybe I’m wrong.

Thanks

1 Like

That’s what that link describes.

Everything, except the plugins (which are in your app.yml) is in the backup; the database and the uploads are all there is.

1 Like

From my testing of this method it seems to be enough to ./launcher stop app before the initial rsync. Of course one of the reasons to use this method seems to be to keep the forum running on the old server as long as possible, in which case it’s obviously necessary to run the second rsync to maintain consistency. But for the relatively common process of moving a forum to a different server and/or host where a brief downtime is acceptable I really like the simplicity and portability of this method.

1 Like

Right.

Right.

My preferred method is to do the rsync of the let’s encrypt and ssl stuff, put the old server in read-only mode, backup, restore on the new server, and then switch DNS (or better, a static IP when the new server is ready.

But if you don’t care about a bit of down time, your way is great.

2 Likes

I’m planning to migrate to a new VPS in January after some recent problems upgrading Discourse on my old Ubuntu.

My questions on migrating from an old Digital Ocean droplet to a new Digital Ocean droplet are:

  • I plan to lower the TTL on the DNS A record the day before my migration to something small, like 5 minutes. Does this sound reasonable?

  • The first post in this thread was last edited in June 2016. Is it still valid and correct?

  • Will this rsync method also copy the entire database from the old VPS to the new VPS?
    – We’re on a standard install

  • Will the existing Let’s Encrypt SSL certificate also be copied across? Is the SSL cert tied or linked to an IP address at all? Will it continue to automatically renew itself? Any gotchas here?

  • At what point should I change the public DNS A record to point to the new VPS?
    – And also change the TTL back to something higher again

That’s all correct.

If you’re using something that let’s to have a permanent ip that can be assigned to multiple vms, then you can do that so that you’re not counting on dns to make the switch.

The only caution i would add is to shut down the old site for the final rsync and then restart it in read only mode while the new one rebuilds.

The first post is still showing the incorrect /var/discourse/ path:

Could you edit/update please?

@Richie, @JammyDodger has now made this a wiki :+1:

2 Likes

I migrated to a new VPS today and thought I’d share my experiences as it looks like quite a few people are running in to the old-version operating system blocker on their updates lately :blush:

I’m on Digital Ocean, so I created a new droplet.

Old vps = Ubuntu Server 18.04.6 LTS

New vps = Ubuntu Server 23.10

I did the usual housekeeping on the new vps - please edit to suit yourself:

Apt-get update

Apt-get upgrade

Apt-get install fail2ban

ufw default deny incoming

ufw default allow outgoing

ufw allow ssh

ufw allow http

ufw allow https

ufw enable

I then created a new empty directory for Discourse:

sudo mkdir -p /var/discourse

Then I installed Docker:

wget -qO- https://get.docker.com/ | sh

Then I changed the TTL on my DNS from 30mins to 10mins (the minimum GoDaddy allows).

On my old server, I downloaded a local copy of last nights Discourse database backup (you can never have enough local backups). I also downloaded a copy of app.yml to my local pc too.

As suggested by a few people above, I did a “root-to-root” rsync. I used the IP address rather than the hostname, so I could avoid any DNS confusion. Also as suggested above, I used -avz switches:

rsync -avz root@old.ip.address.here:/var/discourse /var

For reference, my discourse folder is 25GB.

It took ~25mins to rsync from the old server to the new server. This was simply between two Digital Ocean droplets in the same LON1 region. Your experiences may differ.

After rsync’ing and trying a rebuild, I hit the same error that @piratdavid hit re postgres database system is shut down.

So I then stopped the app on the old vps:

./launcher stop app

And did another rsync, for just the changes this time:

rsync -avz --delete root@old.ip.address.here:/var/discourse /var

Then I started the old Discourse app again and very quickly put it in Maintenance Mode - this is so people can still get to it and will see the usual maintenance warning message.

This also now buys me some time to work on the new vps :blush:

I updated my HOSTS file on my local pc so I could get to the discourse on the new vps without browser warnings / issues.

On the new vps I then ran:

./discourse-setup

This was so it could update the ram and cpu settings in the app.yml file automatically.

I then did an app rebuild on the new vps:

./launcher rebuild app

Did some smoke tests, all good.

DNS updated - job done.

Thanks for the detailed topic, everyone :smiley:

2 Likes

Thanks guys, first post updated re /var/discourse paths.

1 Like

If anyone is having trouble doing the root to root rsync because maybe they disabled root login on the old server, or you just want to do this as a non-root user, I found this post to be helpful to figure out how to use sudo on the remote server: permissions - Using rsync with sudo on the destination machine - Ask Ubuntu

Let’s say you have a user, discourse, on both sides that have sudo privileges. On the remote machine, you’re going to edit the /etc/sudoers file with sudo visudo. You’re going to add the line:

discourse ALL=NOPASSWD:/usr/bin/rsync

Then on the new machine, you’re going to run (as your non root user):

sudo rsync -avz --delete --rsync-path="sudo rsync" discourse@old.ip.address.here:/var/discourse /var

This will allow you to run everything described here as non root users. If you’re keeping the old server around, I’d go back into the /etc/sudoers file and delete the line you just put in.

1 Like

If I understand correctly, this allows the bulk of the transfer to happen while Discourse is running. The restoring from backup strategy requires at least read-only for the backup and moving the backup to the new server (or transfer via S3 bucket). For large sites, this can result in considerable read-only time that the rsync strategy neatly avoids.

It might be possible to squeeze a bit more uptime by avoiding shutting down PostgreSQL on the old system and “fixing” the problem on the new system with pg_resetwal. NB: I haven’t tried this and letting the database shutdown gracefully is almost certainly a better idea.

I wonder if there’s a way to start Discourse is read-only? I suspect the fastest way is via the command line after the container is running.

At any rate, thanks for reporting back your experience! Seems like a useful process to have in your back pocket. :slight_smile:

Very.

So useful in fact, that I’m tempted to do it all again to create a staging environment (on a lower spec’d VPS), just for testing and preempting any issues before implementing any changes on production.

1 Like