How-to guide on DigitalOcean Block Storage for backups?


(Anton) #1

This topic complements using Digital Ocean Block Storage for image uploads. Hope the team will provide us with official ( recommended ) way of setting this up and move to the #howto category.

So, I simply would like to let Discourse make all backups in a block storage unit.

  • Any nuances or little tricks to pay account to, or just mount the block and configure its path?
  • Will it affect the speed of making backup?
  • Will it easily “restore” from such a backup?
  • Any other things to consider?

Many thanks!


(Jay Pfaffman) #2

This should do it:

cd /var/discourse
./launcher stop app
cd /var/discourse/shared/standalone
mv uploads /mnt/volume-sfo2-01/backups # script'd need to find this
ln -s /mnt/volume-sfo2-01/backups .
cd  -
./launcher start app

Given that I didn’t miss a typo in the above, it should work fine. I tested having symlinks on my development machine and did a backup and restore with no problems. The slower speed of the block storage shouldn’t matter for backups.

I’d like to integrate this into discourse-setup, but it hasn’t happened yet.


(Anton) #3

I’d like for the team to confirm the commands above, at least by liking the message, please :hamburger:


(Matt Palmer) #4

We don’t have any magic, secret knowledge about how Digital Ocean block storage works, and we don’t use it at all for our own infrastructure. You’d be better off trying it out yourself, determining if it meets your needs, and then reporting back for others in a similar position.


(Anton) #5

I’d gladly do, but I’m afraid to break something as I’m not knowledgeable enough in *nix to play with stuff like this and, more importantly, measure the outcomes.


(Jay Pfaffman) #6

Having backups on DO block storage is much less risky than having your images there. Has that been working for you? If so, it’d be helpful to know, as you may be the person with the most experience.

But wait! Now it looks like there’s a better solution for backups developed and tested by a Discourse @team member:


(Anton) #7

Dropbox is a more expensive alternative, no?

What’s more, it says it syncs, NOT moves, backups to Dropbox, which does not solve the problem of space shortage.


(Rafael dos Santos Silva) #8

It depends.

What I had in mind is a specific use case where the problem was: I have a dirty cheap vps provider, and I’m afraid of losing all my data. If things go south, I need a fresh backup to start over elsewhere.


(Jay Pfaffman) #9

Oh. I totally overlooked that. I’m already paying for dropbox, so for me it’s “free,” but $100/year is < than the $10/month for the DO space. I haven’t looked at the plugin, but I imagine that there’s a way to contrive having dropbox keep old copies (e.g., it keeps deleted files for a month or something like that).

But my instructions for moving backups to DO’s Block storage should work just fine, and since it’s backups (which you can, for example, store a couple offsite while you do the initial move), it’s pretty low-risk.


(Anton) #10

With DO Block Storage, one can buy 20GB / $2 / mo for backups, which is more than enough and much cheaper than Dropbox.

The question though is not about how to SYNC, but rather about how to MOVE backups to another location to free up space on the main VPS.


#11

Hi,
what is the logic behind moving the uploads? I’m just looking for a way to keep backups on a different place and your method seems perfect except I don’t understand the need to move uploads and turning it into backup destination…
Maybe I’m just going crazy :stuck_out_tongue:
Doing it your way results in:

ln: failed to create symbolic link ‘./backups’: File exists

Best regards,
Neso


(Jay Pfaffman) #12

I think that I was moving the uploads because that was what was asked. (but I’m using my phone and answering quickly rather than perfectly.)

The error you got suggests that backups didn’t get moved (hence the clash in the symlink). I think there was another thread about moving the backups.


(Dave Jensen) #13

I looked through the contents of /var/discourse and I began to wonder. Would there be any disadvantage to having the entire /var/discourse on block storage. It has uploads, backups, image, logs, and config but not the docker image itself. Basically it has all the things I need if I want to move to a new server or if the Droplet goes away I can quickly reattach the volume and I’m up and running again.

Thoughts?


(Jay Pfaffman) #14

It’s much slower, so you really don’t want your database stored there.


(Dave Jensen) #15

Oh, I’m running a web only instance so it would be everything except the DB.

Is DO Block Storage much slower than AWS EBS? I thought it was a common practice to use EBS for databases.

I have a MySQL DB running SMF and the files are on Block Storage with no issues. In fact, when I split the DB from the frontend, everything went faster.


(Arpit Jalan) #16

We now have a howto for using DigitalOcean Block Storage: