Need a how-to guide for using DigitalOcean Block Storage for image uploads

Will there be an update any time soon? We’re close to the out of space moment and would like to follow the official recommendations from Discourse team.

Bump. Anyone has used DigitalOcean Block Storage for image uploads since then? If so, please share your experience.

I’m still hoping to get an “official way” / recommendation from the team regarding how to use DO Block Storage for uploads. Have been hoping since it was said that it is

:wink: if anyone from the team can look into it that would be very appreciated.

1 Like

Hello!

So I’ve tried adding a DO block for our uploads folder following your script. All the permissions seem fine.

But there’s clearly something not configured as now all images are 404, even though I can see them as moved. Furthermore uploading new images isn’t working-- just stalls.

Any pointers on the final configuration needed for this to work?

1 Like

Sorry, but not offhand.

On the system where I’d done a very quick test, I noticed that there was something strange about how the images were stored. I didn’t take careful notice of just what the deal was, though.

Hey, @sam, do you see any obvious flaws in my code above to move images to a different volume?

3 Likes

I would recommend against symlinking, instead, copy images and amend your container config to point /shared/uploads at the correct location.

3 Likes

So, @tehn, you’d add something like this in the volumes section of your app.yml

   - volume:
       host: /other/volume/uploads
      guest: /shared/uploads
9 Likes

more or less, you want to get indenting right, but that is the gist of how to do it.

5 Likes

@pfaffman, so the idea is to let Discourse think it’s still using the same path for uploads, but move the path to a block storage unit and let the Docker do the mounting?

@tehn did you get it working? If so, how is it going? Any performance issues or any other issues?

This worked, thank you all.

I haven’t noticed any performance issues.

4 Likes

I think it should be fine for images, but not the database.

Thanks for the report!

What about backups — do they work just well?

They should work the same way. There’s even less concern about performance for backups, obviously.

Note: be careful when selecting the size of your block storage.
Once your reach its limits, resizing an existing block storage is not a trivial process.

Specifically for image uploads though, I think an easier “resizing” would be to just create a new volume, reconfigure the your .yml file and copying images from the old volume to the new one. Then rebuilding your container again.


Question

Is there any filesystem to format the partition with in terms of image storage? Will it make any different, or should we just use Ext4?

I can’t imagine what difference it would make. I’d use ext4.

3 Likes

So, I finally spent a few hours to implement all this, and it worked quite well as by the instructions above.

I had Discourse hosted in a region with no volumes support, so I also had to move to another region first.

Altogether:

  1. Turned off the droplet.
  2. Made a snapshot.
  3. Moved the snapshot to the target region.
  4. Created a new dropled from the given shapshot in the target region.
  5. Attached a volume and did all the work as described in Digital Ocean.
  6. Updated container as by Sam’s recommendation above:

7. Rebuilt the container
8. Updated all API keys to work with the new IP. For example, I had my SparkPost API key locked to a particular IP address.

All done and worked like a charm.

References:

9 Likes

Why would you advise against symlinks?

Yeah be careful here, everyone, as we found out block storage is quite a bit slower than native SSDs. Like one-half to one-third the speed, so…

4 Likes

So what is recommended? Which CAN we move to volumes without too much of performance loss? Does anyone have a 1-2-3 guide? Everyone in this topic seems to have scattered info for advanced crowds, still hoping to find a simple guide.

  1. Let’s say I have /mnt/someStorage/
  2. Edit /var/disocurse/containers/app.yaml section about volumes. Add a -volume … info gets a bit scattered here. Somehow link to somewhere in /mnt/someStorage/somewhere?
  3. ?
1 Like

Hmm, what do you mean by #scriptd need to find this

I mean that volume-sfo2-01 might not be the name of your volume.

1 Like