Moving uploads and backups to DigitalOcean Block Storage

So you want to move all uploads and backups to DigitalOcean Block Storage?

Before you do this, though, you might instead consider Using Object Storage for Uploads (S3 & Clones). It is no more complicated to set up and provides some other advantages.

Setup a Discourse instance on DigitalOcean

Setup a Discourse instance on DigitalOcean using our official guide.

Add a Volume to your droplet

Add a block storage volume to your droplet. Note down the name of the volume. In my case the name of the volume was volume-nyc1-05.

Make sure the volume is formatted and mounted at /mnt/volume_nyc1_05.

Move /var/discourse/shared to the volume

For best performance we will keep postgres_data on our main drive and will move rest of the data to the newly attached volume.

Logged in as root, run these commands from your server:

mv /var/discourse/shared/standalone/postgres_data /var/postgres_data_discourse
mv /var/discourse/shared /mnt/volume_nyc1_05

Make sure to replace volume_nyc1_05 with the name of your volume.

Edit app.yml to reflect the changes

Open the /var/discourse/containers/app.yml file in your preferred editor and look for this block:

## The Docker container is stateless; all data is stored in /shared
volumes:
  - volume:
      host: /var/discourse/shared/standalone
      guest: /shared
  - volume:
      host: /var/discourse/shared/standalone/log/var-log
      guest: /var/log

Replace the above block with:

## The Docker container is stateless; all data is stored in /shared
volumes:
  - volume:
      host: /mnt/volume_nyc1_05/shared/standalone
      guest: /shared
  - volume:
      host: /mnt/volume_nyc1_05/shared/standalone/log/var-log
      guest: /var/log
  - volume:
      host: /var/postgres_data_discourse
      guest: /shared/postgres_data

Make sure to replace volume_nyc1_05 with the name of your volume.

Save the app.yml file and rebuild the container.

/var/discourse/launcher rebuild app

:tada: Voila!

That’s it all your uploads and backups are now moved to DigitalOcean Block Storage.


Credits

This guide is inspired from @sam’s suggestion here.

33 Likes

The biggest thing that you’ll need to do is frequently do a ./launcher cleanup. It’ll be a long while before you have considerable database size. You can do a

cd /var/discourse
du -hs shared/standalone/*

to see what Discourse is using (but not what Docker is using, which is your bigger concern that the above command helps with).

5 Likes

By default, backups do not include uploads stored on S3 as that’s a waste of bandwidth and storage. There’s a hidden site setting to change that behavior:

./launcher enter app
rails c
SiteSetting.include_s3_uploads_in_backups = true

We enable it on our hosting when customers cancel their subscription, so their last backup before we shutdown their site will always contain the uploads stored on S3.

5 Likes