Moving uploads and backups to DigitalOcean Block Storage

howto sysadmin
#1

So you want to move all uploads and backups to DigitalOcean Block Storage?

Before you do this, though, you might instead consider Using Object Storage for Uploads (S3 & Clones). It is no more complicated to set up and provides some other advantages.

Setup a Discourse instance on DigitalOcean

Setup a Discourse instance on DigitalOcean using our official guide.

Add a Volume to your droplet

Add a block storage volume to your droplet. Note down the name of the volume. In my case the name of the volume was volume-nyc1-05.

Make sure the volume is formatted and mounted at /mnt/volume_nyc1_05.

Move /var/discourse/shared to the volume

For best performance we will keep postgres_data on our main drive and will move rest of the data to the newly attached volume.

Logged in as root, run these commands from your server:

mv /var/discourse/shared/standalone/postgres_data /var/postgres_data_discourse
mv /var/discourse/shared /mnt/volume_nyc1_05

Make sure to replace volume_nyc1_05 with the name of your volume.

Edit app.yml to reflect the changes

Open the /var/discourse/containers/app.yml file in your preferred editor and look for this block:

## The Docker container is stateless; all data is stored in /shared
volumes:
  - volume:
      host: /var/discourse/shared/standalone
      guest: /shared
  - volume:
      host: /var/discourse/shared/standalone/log/var-log
      guest: /var/log

Replace the above block with:

## The Docker container is stateless; all data is stored in /shared
volumes:
  - volume:
      host: /mnt/volume_nyc1_05/shared/standalone
      guest: /shared
  - volume:
      host: /mnt/volume_nyc1_05/shared/standalone/log/var-log
      guest: /var/log
  - volume:
      host: /var/postgres_data_discourse
      guest: /shared/postgres_data

Make sure to replace volume_nyc1_05 with the name of your volume.

Save the app.yml file and rebuild the container.

/var/discourse/launcher rebuild app

:tada: Voila!

That’s it all your uploads and backups are now moved to DigitalOcean Block Storage.

Credits

This guide is inspired from @sam’s suggestion here.

33 Likes
Need a how-to guide for using DigitalOcean Block Storage for image uploads
How-to guide on DigitalOcean Block Storage for backups?
Docker uses root partition, might need more storage space
PostgreSQL 12 update
#6

The biggest thing that you’ll need to do is frequently do a ./launcher cleanup. It’ll be a long while before you have considerable database size. You can do a

cd /var/discourse
du -hs shared/standalone/*

to see what Discourse is using (but not what Docker is using, which is your bigger concern that the above command helps with).

5 Likes
#18

By default, backups do not include uploads stored on S3 as that’s a waste of bandwidth and storage. There’s a hidden site setting to change that behavior:

./launcher enter app
rails c
SiteSetting.include_s3_uploads_in_backups = true

We enable it on our hosting when customers cancel their subscription, so their last backup before we shutdown their site will always contain the uploads stored on S3.

5 Likes