Dear community
Digital Ocean Block Storage has been available since some time ago.
Because Discourse officially endorses Digital Ocean in their “do it yourself” setup guide, it would be very helpful to have a how-to that explains how to enable Block Storage and use it for image upload.
Anyone keen on writing one?
Good idea.
Current block storage is like a remote drive you mount in any Linux / Unix.
Current DO pricing for storage is $1 per month per 10gb. Certainly more cost effective than upsizing the droplet if you don’t need more CPU and RAM.
There really isn’t anything Discourse-specific involved in using DO block storage… create storage volume, mount wherever you want to store lots of stuff (I’d say /var/discourse/shared, but YMMV) and you’re away.
@mpalmer a step-by-step guide similar to DO setup guide would be very appreciated.
Among the commands to run to follow what you said in one paragraph, it should also include some info about how to do it all for an existing Discourse setup with lots of images uploaded.
Indeed. How to mount the volume and then get stuff moved to it and the mount point moved to the right place is a bit tricky.
I’ve got a couple projects winding down and might be able to do it next week if there is interest.
Part of me thinks that if you need to attach extra storage then you need to have enough sysadmin skills to mount a volume wherever you want it and deal.
It seems too that droplets created before they started the service don’t support it, so many users would need to create a new droplet to take advantage of the service.
Another solution would be to create a script or modify discourse-setup such that if you have a volume mounted it would offer to move /var/discourse/shared (or maybe /var/discourse/standalone?) to the volume and symlink to it. The user would need to have created a volume and followed DO’s instructions for formatting and mounting it, but the script could handle the rest. This will make writing the instructions considerably easier since it’d be something like
cd /var/discourse; ./discourse-setupI’ve been wanting to fix discourse-setup so that it’ll read an existing app.yml to let people make changes to an already set up instance anyway.
Both basic Discourse DigitalOcean setup and additional configuration all can be done by going through step-by-step guides with no sysadmin skills.
Because the standard recommended setup guide is there, based on that setup it is possible to write another standard step-by-step guide, which I’m asking for 
A guide is at least something for people to try, even if they don’t have enough sysadmin skills. It is better than guide absence.
I would agree except that it’s easier to write a script that a computer can follow than one that a person can.
With a few tweaks to ./discourse-setup you’ll be able to have that block storage without having to get your hands dirty with mv and ln.
We need to determine performance before making recommendations.
EG: if this thing is slow then the recommendation would be to keep PG on local and images (both docker and pictures) on block storage. That would require a bit of fancy.
The man makes a point. It appears to be less than 1/3 the speed on the simplest of benchmarks. (I checked and both the host and the storage are at SFO2.)
root#:dd if=/dev/zero of=/tmp/output conv=fdatasync bs=384k count=1k
1024+0 records in
1024+0 records out
402653184 bytes (403 MB, 384 MiB) copied, 0.590498 s, 682 MB/s
root#:dd if=/dev/zero of=/mnt/volume-sfo2-01/test conv=fdatasync bs=384k count=1k
1024+0 records in
1024+0 records out
402653184 bytes (403 MB, 384 MiB) copied, 1.99937 s, 201 MB/s
It seems to me that moving the images to the other storage is as simple as shutting down Discourse and
cd /var/discourse
./launcher stop app
cd /var/discourse/shared/standalone
mv uploads /mnt/volume-sfo2-01/uploads # script'd need to find this
ln -s /mnt/volume-sfo2-01/uploads .
cd -
./launcher start app
(Hey, @meglio, there’s your HOWTO )
Docker images don’t get that big if you purge them periodically, so don’t I understand why to bother moving them. What am I missing?
Will there be an update any time soon? We’re close to the out of space moment and would like to follow the official recommendations from Discourse team.
Bump. Anyone has used DigitalOcean Block Storage for image uploads since then? If so, please share your experience.
I’m still hoping to get an “official way” / recommendation from the team regarding how to use DO Block Storage for uploads. Have been hoping since it was said that it is
if anyone from the team can look into it that would be very appreciated.
Hello!
So I’ve tried adding a DO block for our uploads folder following your script. All the permissions seem fine.
But there’s clearly something not configured as now all images are 404, even though I can see them as moved. Furthermore uploading new images isn’t working-- just stalls.
Any pointers on the final configuration needed for this to work?
Sorry, but not offhand.
On the system where I’d done a very quick test, I noticed that there was something strange about how the images were stored. I didn’t take careful notice of just what the deal was, though.
Hey, @sam, do you see any obvious flaws in my code above to move images to a different volume?
I would recommend against symlinking, instead, copy images and amend your container config to point /shared/uploads at the correct location.
So, @tehn, you’d add something like this in the volumes section of your app.yml
- volume:
host: /other/volume/uploads
guest: /shared/uploads
more or less, you want to get indenting right, but that is the gist of how to do it.
@pfaffman, so the idea is to let Discourse think it’s still using the same path for uploads, but move the path to a block storage unit and let the Docker do the mounting?
@tehn did you get it working? If so, how is it going? Any performance issues or any other issues?
This worked, thank you all.
I haven’t noticed any performance issues.
I think it should be fine for images, but not the database.
Thanks for the report!