Creating a duplicate of production environment

(Henti Smith) #1

I’m desperately trying to work out how to create a duplicate of our production system to allow testing of upgrades etc before taking it live. I’m hoping there is an easily reproducible set of steps to “export” the production system and then “import” into a new clear server.

I’ve looked at a number of different threads but nothing seem to cover this.

Is there a set of steps to duplicate a production system onto another machine with all data/attachements/etc in place ?


How to populate a dev server with data from Production?
(Rafael dos Santos Silva) #2

Download a backup from production and restore on staging.

(Andrew Waugh) #3

It’s pretty simple. Off the top of my head:

Create your sandbox, install discourse.
Decide if you want mail (you don’t really need it if you’re just testing how to do “stuff”)
Fudge app.yml so mail won’t go out.
Download a backup from live, restore.
Check it works.
Clean up the .db (Disk space usage by postgres)
Have at it.

EDIT: My sandbox is just for checking software and such, so I don’t have outgoing mail. In fact I deliberately mangled all the entries in app.yml so that it would never send mail (and I disabled all outgoing).

(cpradio) #4

I think you may want to run a remap domain in there too. That way all of the links do not go out to Production.

(Henti Smith) #5

Hi Andrew,

I originally thought that was the correct way, but I could not get past step 1. My installation of discourse fails and unfortunately, nobody seems to be able to assist with that.

I was hoping it’s the wrong process and there is a different way to do it.

Thank you for the information.


(Andrew Waugh) #6

I think you could do it by changing git head, but I wasn’t smart enough for that. In my case I was close enough to the live site that the restore ran.

Have you seen this:

(Henti Smith) #7

That is the process I tried.

I’ll continue trying to get that working today. Thanks again for all the assistance.

(Henti Smith) #8

Resolved the problem.

I had the failed data in /var/discourse that I didn’t know I needed to clean up.

(Henti Smith) #9

Just to close the loop. Steps I followed.

On Production:

Get latest backup from /var/discourse/shared/standalone/backups/default

On New/Dev instance :

mkdir /srv/discourse
git clone /srv/discourse
cd /srv/discourse
# If you need a specific commit, do:
# git reset --hard git_commit_hash_goes_here

Copy the app.yml from your production server to new/dev instance. Update app.yml

new_hostname=$(hostname -f)
sudo sed -i "s/^  DISCOURSE_HOSTNAME:.*$/  DISCOURSE_HOSTNAME: $new_hostname/g" containers/app.yml
# If you need to duplicate a specific version of discourse, use the git commit hash, otherwise use the branch name. Below we are using stable branch
sudo sed -i 's/^  #version: .*$/  version: stable/g' containers/app.yml
sudo sed -i "s/^  DISCOURSE_DEVELOPER_EMAILS: .*$/  DISCOURSE_DEVELOPER_EMAILS: ''/g" containers/app.yml

Bootstrap, start the app container

sudo ./launcher bootstrap app
sudo ./launcher start app

Copy your backup to restore location

sudo mkdir /var/discourse/shared/standalone/backups/default -p
sudo cp /tmp/discourse-backup/lgtm* /var/discourse/shared/standalone/backups/default

Now we run commands in the container to prepare, and restore the backup, and fix the permissions.

backup=$(basename /var/discourse/shared/standalone/backups/default/*)
sudo /usr/bin/docker exec -i app discourse restore $backup
sudo /usr/bin/docker exec -i app chown discourse.www-data /var/www/discourse/public/backups/default/ -R 

If you get an error about SiteSettings not allowing restore. (I think it’s older version of discourse that needs this)

sudo /usr/bin/docker exec -i app rails runner "SiteSetting.allow_restore = true"

I tried using the Advanced, manual method of manually creating and restoring Discourse backups method for my restore, but it failed with rake db:migrate creating an empty db. Might work for you, I did not test further.

(Henti Smith) #10

Slight update.

Replacing the DISCOURSE_HOSTNAME with $(hostname) bit me in the arse … I didn’t realise it’s used for url generation, so make sure it’s the url external users will be seeing if you use proxy/load balancers.


(Keith Guerrette) #11

I went about this in a different, although slightly uglier way - I cloned my entire droplet on DigitalOcean, then logged into the droplet and disabled all of the ssl/https settings in the app.yml. This at least gave me a clone of my community content to screw with settings on.

I have no elegant way to migrate changes from my test server to the live server except by manually repeating, so this is still total amateur hour, but it at least let’s me explore without breaking the live server.