Kind of unavoidable I think we bumped base image 2 times in the last few months. Not much we can do. Looks like the cleanup on your server shaved off 2 base images.
@anon43908006, there is a guide at:
It goes over many considerations for changing your domain, check it out.
To clarify, is there not much to do about the overall size of the upgrades increasing, or not much to do about the recent spike in bumping base image activity (which will not affect as much in the future)?
I’v been surprised, I’ve got all these tiny Discourses with very few users, and kept running into this issue lately. No uploads or anything. I was wondering if we are approaching a point where the cloud install will recommend the next size up on drive space (which is 2GB RAM/1vCPU/50GB SSD).
I asked @falco about this in chat and he said lately we’ve had a lot of base image changes due to dependencies being updated, so there’s been a higher than usual amount of disk space in upgrades in the last ~6 months.
Sorry to hear you’ve run into trouble changing your domain name, @anon43908006.
Because this is #support, I encourage you to create a new topic that explains your exact case: it may be that your situation requires more discussion than what’s happening in this topic, which is more of a general pattern we’ve been noticing.
If you’d like, you can mention me (@maiki) and I’d be happy to discuss what’s going on with your site.
I get the same
No space left on device error when I try to backup my Discourse:
[2022-11-15 08:23:38] EXCEPTION: /var/www/discourse/lib/discourse.rb:131:in `exec': Failed to gzip archive. gzip: /var/www/discourse/public/backups/default/forum-leasehackr-2022-11-15-080439-v20221110175456.tar.gz: No space left on device
My backup and image uploads are set up on DigitalOcean’s Spaces and have been working fine for a few years until recent months. Here are what I have tried so far:
- I cleared all hidden multipart uploads on my DO Space. There should be more than 100GiB of storage available on my DO Space.
- I tried rebuilding and cleaning up using the following commands:
cd /var/discourse apt-get update apt-get upgrade apt-get autoclean apt-get autoremove ./launcher rebuild app ./launcher cleanup
Anyone knows why my backups keep failing still? Thank you!
Check to see if you have leftover backups here and delete them:
This message is about your local disk. To create a backup you need to have almost twice the size of your database+uploads available to create the backup that gets uploaded to spaces. What do these show you?
df -h du -hs /var/discourse/shared/* du -hs /var/discourse/shared/standalone
This is what the commands show:
root@forum:~# df -h Filesystem Size Used Avail Use% Mounted on udev 7.9G 0 7.9G 0% /dev tmpfs 1.6G 169M 1.5G 11% /run /dev/disk/by-label/DOROOT 315G 277G 22G 93% / tmpfs 7.9G 4.9M 7.9G 1% /dev/shm tmpfs 5.0M 0 5.0M 0% /run/lock tmpfs 7.9G 0 7.9G 0% /sys/fs/cgroup cgmfs 100K 0 100K 0% /run/cgmanager/fs none 315G 277G 22G 93% /var/lib/docker/aufs/mnt/990236bd36313269b6683947dc21241f4387b2671676e8fdf68de05c0fb3e0f4 shm 512M 22M 491M 5% /var/lib/docker/containers/4684612f16f4f31589ae8075145bcde0e6e9576a22eaef3952e9d47986999b91/shm tmpfs 1.6G 0 1.6G 0% /run/user/0 root@forum:~# du -hs /var/discourse/shared/* 215G /var/discourse/shared/standalone root@forum:~# du -hs /var/discourse/shared/standalone 215G /var/discourse/shared/standalone
If it’s a local storage issue, can I simply add block storage (Volumes) to the DigitcalOcean Droplet?
There is a topic about that. I would likely move backups to Spaces first.
What about this
du -hs /var/discourse/shared/*
I have backups set up on DO Spaces. I thought the backup failed due to not having enough space locally?
du -hs /var/discourse/shared/standalone/*
Oh. Yes, local storage is the problem, but an easy way to solve it is moving backups, but you did that already…
But maybe you can do a
And get back space from docker
This is the output:
root@forum:~# du -hs /var/discourse/shared/standalone/* 124G /var/discourse/shared/standalone/backups 21M /var/discourse/shared/standalone/letsencrypt 2.9G /var/discourse/shared/standalone/log 4.0K /var/discourse/shared/standalone/postgres_backup 74G /var/discourse/shared/standalone/postgres_data 264K /var/discourse/shared/standalone/postgres_run 5.1G /var/discourse/shared/standalone/redis_data 44K /var/discourse/shared/standalone/ssl 28K /var/discourse/shared/standalone/state 16K /var/discourse/shared/standalone/tmp 9.2G /var/discourse/shared/standalone/uploads
./launcher cleanup two days ago and it didn’t solve the backup fail problem.
You have too much storage being consumed by your backups. You’ll need to rethink your backup strategy and possibly move backups to an external storage like S3.
Either you don’t have backups on spaces or you didn’t delete the local backups when you switched to it. You can delete those backups and be in good shape.
I believe it’s the latter because I do see my backup files on my DO Spaces. What is the best practice to delete the backups? Can I remove the content with the following command?
rm -r /var/discourse/shared/standalone/backups
That should do the trick!
That was indeed the cause. Just cleared out the local backup folder and backup works again. Thank you for your help!
Hooray! With 126Gb free, you should be good for a while!
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.