Manually create and restore Discourse backups

As discussed below, a warning for newbies:

  1. this is an advanced method! there is an automated, web UI way to restore from backups at /admin/backups which is so much easier.
  2. with the method below, no automatic backup is kept of the /var/discourse/containers/app.yml file which contains essential SMTP credentials and SSL customizations. Keep a copy of this offsite whenever you change it.
  3. If you are using digital ocean, keep frequent snapshots of your droplets - preferably before each time you run ./launcher rebuild app or do any upgrades. Unfortunately in my case this takes an hour each time and it’s a manual process - but it’s totally worth it if you are not familiar with docker and want to avoid a headache later.

Continuing the discussion from Migrate db from old Discourse to new without updating:


Dropping and recreating the schema is required if the current database contains a table or view that the old one does not, because of referential integrity and pg_dump not adding CASCADE to its drop statements.
This could be easier if the extensions weren’t stored in the public schema.


Upgrading Discourse is not very hard, but there is always a risk involved. This is why creating a backup is always a strong recommendation before upgrades are installed – but what should you do when your current version is so old that it does not have the backup feature?

The safest way to do this is to not attempt any in-place upgrades on the existing, outdated Discourse instance. Instead, we will manually create a backup and import it into a blank but up-to-date Discourse instance.

Note regarding default path changes

The default path for the docker manager scripts has recently changed from /var/docker to /var/discourse. Since the main purpose of this howto is to help admins of older versions of Discourse to migrate to a new versions, I will use the old default path name throughout the howto.

If you have a newer version of Discourse and the docker manager, simply switch the path names mentioned above where appropriate.

Creating the backup

To create the backup, we need access to Discourse’s database and uploaded files. This depends a lot on your current setup. If you

  1. If your old Discourse…
  • lives in a Docker container, enter it:
    cd /var/docker
    git pull
    sudo ./launcher enter app
    su - discourse
  • does not live in a Docker container, you should at this point ensure that the user you’re logged in as can use pg_sql to connect to Discourse’s production database and has access to the uploaded files. You may also have to adjust the directory and file names below.
  1. Export the database and uploaded files:
    pg_dump -xOf /shared/discourse-backup.sql -d discourse -n public
    gzip -9 /shared/discourse-backup.sql
    tar -czf /shared/discourse-uploads.tar.gz -C /var/www/discourse/public uploads

  2. Almost done. Type exit twice to leave the container, then copy the files discourse-backup.sql.gz and discourse-uploads.tar.gz from the host’s shared folder, typicalls /var/docker/shared/standalone and store them somewhere safe. The SQL file contains password-equivalent material, so don’t put this file on a webserver and don’t send it unencrypted.

Restoring the backup

As above, this procedure depends on your particular setup, so this howto can only cover the case that you followed the supported setup instructions and have Discourse running in an up-to-date Docker container.

  1. Upload the backup files and copy them into /var/docker/shared/standalone, then run:
    gzip -d /var/docker/shared/standalone/discourse-backup.sql.gz

  2. As above, enter the container:
    cd /var/docker
    sudo ./launcher enter app

  3. Flush your new Discourse’s database. This step is destructive; you must be absolutely certain that you are in the correct container, connecting to the correct database and that it contains no important data.

sudo -u postgres psql discourse <<END
DROP SCHEMA public CASCADE;
CREATE SCHEMA public;
ALTER SCHEMA public OWNER TO discourse;
CREATE EXTENSION IF NOT EXISTS hstore;
CREATE EXTENSION IF NOT EXISTS pg_trgm;
END
  1. Restore the backup data:
    su - discourse
    rm -rf /var/www/discourse/public/uploads
    tar -C /var/www/discourse/public -xzf /shared/discourse-uploads.tar.gz
    psql -d discourse -f /shared/discourse-backup.sql

  2. Bring the old database up-to-date by migrating it:
    cd /var/www/discourse
    RAILS_ENV=production bundle exec rake db:migrate

  3. Exit and restart the container:
    exit
    exit
    sudo ./launcher restart app

2 Likes

Note if you’re migrating away from Heroku, you can’t use pg_dump on the server directly, so you’ll probably want to pg:pull your database to your local machine, and then follow the above directions from step 2.

You also will probably skip the commands related to uploaded files, since Heroku probably already forced you to use S3 for assets.

(Also, if for any reason you’re using pgbackup restore, be sure to pg:reset first or you might end up with a frankenschema like I did and your migrations won’t run.)

1 Like

(whoa, just found this old draft from months ago which apparently I never saved. Still relevant and hopefully useful to others who come after me)

I have some suggestions for improving this topic, having just wrangled (with success!) with restoring backups as a newbie.

  1. suggest adding text right at the top explaining that there is an automated, web UI way to restore from backups and link to a topic explaining backup and restore for beginners - I was not aware of this option until after attempting the manual process which is not for the faint of heart. I would even change the title of this howto to add “Advanced” or to specify that the topic is specifically for people with very old discourse sites.
  2. no automatic backup is kept of the /var/discourse/containers/app.yml file which contains essential SMTP credentials and SSL customizations. Keep a copy of this offsite whenever you change it.

If you are using digital ocean, keep frequent snapshots of your droplets - preferably before each time you run ./launcher rebuild app or do any upgrades. Unfortunately in my case this takes an hour each time and it’s a manual process - but it’s totally worth it if you are not familiar with docker and want to avoid a headache later. :slight_smile:

1 Like

It is wiki now, feel free to improve the howto.

1 Like

I am trying to restore from an earlier auto generated backup to S3.

The RAILS_ENV=production bundle exec rake db:migrate step seems to be overriding all the data even after I have restored the data with psql -d discourse -f /shared/discourse-backup.sql. I end up with a clean install in the end.

Probably of note: I have two containers, web and data.

What am I missing?

1 Like

Why aren’t you restoring the backup via the admin UI?

1 Like

Yep, @ernsheong restore through the admin UI then ./launcher restart both containers.

Wow, I love you guys! @riking @zogstrip :+1:

1 Like

permission denied (on the /shared folder).

1 Like

Not sure that I have permissions to edit the wiki post above. Is it a pilot error from my side?

Anyways, commands need to be adapted to /shared/ structure. For instance:

 tar -C /var/www/discourse/public -xzf /shared/discourse-uploads.tar.gz

Should be:

tar -cvzf /shared/discourse-uploads.tar.gz -C /shared uploads

And it’s a bit critical as the current backup command will silently create an empty backup uploads.tar.gz, as it doesn’t follow the symlink to a different mount.

Sorry to bump this topic up. :cry: This howto seems to be out of date and needs an update.

My instance has run into some issues, I have no longer access to it from the browser. So I’m trying to find a way to get a backup of my site manually and then I found this topic.

Any help would be appreciated!

Have you tried

cd /var/discourse
./launcher rebuild app

I don’t see anyhing about what you quoted that it out of date. Is this a standard install? Do you see anything in /var/discourse/containers?

Thanks for your quick response!

Yes, I have tried rebuilding many times but keep getting fails… So I created a new droplet and installed a fresh Discourse instance on it, with success. And I have restored a backup which is created a month ago, what I’m trying to do is get back the data of these days.

Yes, there are app.yml and one backup for it.

The /var/docker directory doesn’t exist, so I cd /var/discourse and followed the guide, until I got prompted

pg_dump: error: connection to database "discourse" failed: connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: FATAL:  role "root" does not exist

after executing pg_dump -xOf /shared/discourse-backup.sql -d discourse -n public.

I’m a newbie, sorry for the disturbing if this guide is actually not outdated.

did you

  su - postgres

before trying the pg_dump?

But you probably want to figure out why you can’t rebuild, as the issue might be something that will keep your database from restoring

2 Likes