Need a how-to guide for using DigitalOcean Block Storage for image uploads

[meglio (Anton)]

@tehn

This worked, thank you all.

I haven’t noticed any performance issues.

What about backups — do they work just well?

[pfaffman (Jay Pfaffman)]

They should work the same way. There’s even less concern about performance for backups, obviously.

[meglio (Anton)]

Note: be careful when selecting the size of your block storage.
Once your reach its limits, resizing an existing block storage is not a trivial process.

Specifically for image uploads though, I think an easier “resizing” would be to just create a new volume, reconfigure the your .yml file and copying images from the old volume to the new one. Then rebuilding your container again.

---

Question

Is there any filesystem to format the partition with in terms of image storage? Will it make any different, or should we just use Ext4?

[pfaffman (Jay Pfaffman)]

I can’t imagine what difference it would make. I’d use ext4.

[meglio (Anton)]

So, I finally spent a few hours to implement all this, and it worked quite well as by the instructions above.

I had Discourse hosted in a region with no volumes support, so I also had to move to another region first.

Altogether:

1. Turned off the droplet.
2. Made a snapshot.
3. Moved the snapshot to the target region.
4. Created a new dropled from the given shapshot in the target region.
5. Attached a volume and did all the work as described in Digital Ocean.
6. Updated container as by Sam’s recommendation above:

Need a how-to guide for using DigitalOcean Block Storage for image uploads

I would recommend against symlinking, instead, copy images and amend your container config to point /shared/uploads at the correct location.

7. Rebuilt the container
8. Updated all API keys to work with the new IP. For example, I had my SparkPost API key locked to a particular IP address.

All done and worked like a charm.

References:

• How to migrate snapshots between regions.
• How to use block storage with Digital Ocean.

[djensen47 (Dave Jensen)]

@sam

I would recommend against symlinking, instead, copy images and amend your container config to point /shared/uploads at the correct location.

Why would you advise against symlinks?

[codinghorror (Jeff Atwood)]

@sam

We need to determine performance before making recommendations.

Yeah be careful here, everyone, as we found out block storage is quite a bit slower than native SSDs. Like one-half to one-third the speed, so…

@sam

the recommendation would be to keep PG on local and images (both docker and pictures) on block storage

[dylanh724 (dh)]

So what is recommended? Which CAN we move to volumes without too much of performance loss? Does anyone have a 1-2-3 guide? Everyone in this topic seems to have scattered info for advanced crowds, still hoping to find a simple guide.

1. Let’s say I have /mnt/someStorage/
2. Edit /var/disocurse/containers/app.yaml section about volumes. Add a -volume … info gets a bit scattered here. Somehow link to somewhere in /mnt/someStorage/somewhere?
3. ?

[dylanh724 (dh)]

@pfaffman

mv uploads /mnt/volume-sfo2-01/uploads # script’d need to find this

Hmm, what do you mean by #scriptd need to find this

[pfaffman (Jay Pfaffman)]

@dylanh724

volume-sfo2-01

I mean that volume-sfo2-01 might not be the name of your volume.

[meglio (Anton)]

@dylanh724

So what is recommended? Which CAN we move to volumes without too much of performance loss?

I’m interested in an official recommendation too. In the meantime, DigitalOcean Block Storage has been working quite well for image uploads for us, except that we do not incur high traffic — just around 200-300k views per month as by Google Analytics.

[dylanh724 (dh)]

@pfaffman

Docker images don’t get that big if you purge them periodically, so don’t I understand why to bother moving them. What am I missing?

I didn’t even know you can purge, and sounds a bit scary to a Linux newb. This is why ;D I can move it and forget it

[pfaffman (Jay Pfaffman)]

Here’s how to remove extra docker stuff:

./launcher cleanup

You’re really not going to be able to forget it. If you want to forget it, get someone else to do it for you.

[mpalmer (Matt Palmer)]

If running ./launcher cleanup is “a bit scary to a Linux newb”, you’re really not going to like mounting extra filesystems and moving data around…

[dylanh724 (dh)]

@mpalmer

If running ./launcher cleanup is “a bit scary to a Linux newb”, you’re really not going to like mounting extra filesystems and moving data around…

That’s why I’m here – I made a backup and I’m asking for a guide much like any of your other guides I followed and it worked fine. Disliking it isn’t going to magically give me more storage on my main drive asking for a guide like any of the other guides sounds like a fair request. Surely I’m not the only one~ When I first started with Discourse and DO last year, I never used Linux in my life.

@pfaffman

You’re really not going to be able to forget it. If you want to forget it, get someone else to do it for you.

If it’s on another volume, if remembering to add +gb to my volume is the worst thing that can happen, I’m game; it’s significantly better than wondering what to do if I run out of space on my main drive.

[sunnyt7 (Sunny T)]

Would anyone recommend using Digital Ocean Block Storage as opposed to just integrating with Amazon S3 for image uploads?

[mpalmer (Matt Palmer)]

I’m sure DigitalOcean would.

[pfaffman (Jay Pfaffman)]

Last I looked, it claimed to be compatible with the Amazon api, but someone will need to do some work to support it. So right now, your options are to use S3 or (pay someone to) develop a plugin to support it.

[mpalmer (Matt Palmer)]

He’s asking about block storage, not object storage.

[pfaffman (Jay Pfaffman)]

In that case, you’re right!

If you don’t have and don’t want to have an account with the Amazon, then block storage is a fine solution. I’ve not bothered to figure out S3 (which doesn’t seem that hard) myself, so I’m using block storage right now!