I have a 50GB droplet that has been running fine for years currently 11.7GB of uploads. As of a couple weeks ago I had at least 15-20ishGB free. I was in admin one day after being absent for a week or so and saw that I only had 700MB free (with a 5ishGB backup), I deleted that now I only have 5.7GB free again.
I’ve run ./launcher cleanup multiple times (this has cleared up 15+GB before for me) on different days after upgrading everything a couple times, and it always says 0bytes to reclaim.
So I’m not sure what happened. I can’t guarantee I had all that space available before I was gone for a week but I’m somewhat sure I did, and no updates were done during that that would have somehow took a bunch of extra space…
Any ideas? I was going to just add a block storage and look into moving uploads, but my uploads aren’t taking up all the space so there’s still some missing/something filling it. I also now don’t have enough space to make a new backup just in case so I’m a little iffy about doing that without one
Hi ! Did you check your backups folder if there’s no broken/temp backup that takes space? It happened to me several times.
To check where the disk space goes, I use the command line tool ncdu. This is how I figured out my issues.
Note: If you ran out of disk space, don’t forget to verify that the remote download images setting hasn’t been automatically disabled because of this. We are supposed to get notification about that, but I somehow missed it multiple times. I’m not even sure I got this notification.
I saved several GB on a 1GB/25GB system with that recently.
But if you’re making a backup that includes the 12GB uploads, that’ll take roughly twice that to make a backup, and a couple of backups will likely fill your disk.
I don’t have any backups saved on site I usually keep the newest one (which is 5.8ishGB) and delete the older ones (also have all those backed up off site), but right now I have none on the site the only space admin reports is the 12GB uploads. I was planning on either expanding the droplet or adding the block storage soon but then all the space evaporated and I hadn’t made a backup in a couple weeks while I was gone and I was looking at the blockstorage guide here and it’s more than just “buy and connect it” so I don’t want to risk moving and changing stuff without one
I wasn’t fully out it was down to 700mb but I did check and this isn’t disabled but thanks for letting me know.
I’ll try all the suggestions in here and see if any works/where it says the space is
This cleared nothing
This essentially confirms for me something is off since he’s talking about on his 25GB install, mine is on a 50GB droplet!
I checked the backup folder and there’s a handful of random files, the largest is 700MB dpkg.status.0 and then 6 (sequential) dpkg.status.1.gz at 200MB. Are these safe to delete? That doesn’t account for most of the space but that’s like 1.6GB which at least helps get closer to a backup…
Edit: ok so there was nothing in var/backups, but I did find one in var/discourse/shared/standalone/backups that wasn’t listed in my Admin cp, so I did get 6.5GB after nuking that
I installed ncdu and ran it, it says my biggest folder is 20GB and the rest almost nothing. So that’s 30GB missing somewhere not in var/discourse?
Outside of that I have a 2.2GB swapfile. So like 4GB including what’s in backups. There’s obviously some other random files, not counting those there’s 20GB unaccounted for
I guess I could take a snapshot, make block storage, move uploads then I’ll have 22GB. I’m just worried the space will keep perpetually disappearing and I can’t move anything else over to get more
Specifically I’d recommend running du -kx / | sort -n | tail -49
or similar, on your server, as root. You need a tactic which looks at what’s actually going on, without assumptions.
While you’re at the command line, I’d suggest
df -mT
free
And even
docker system df
docker images
I’m concerned about what you’re saying about backups. There are two kinds of backups: the normal kind which includes attachments and the lightweight kind which excludes them. They should be different sizes. You will have some configured choice of automatic scheduled backups. It seems possible to me that the automatic backups are happening when there’s space, and that fills the space.
I recommend you have a good look at files before you delete them - what’s the date and timestamp, what’s the size. That will help to determine how they came into existence.
It worth noting that deleting an open file will cause it to become unlisted but it won’t be deleted until it’s closed. It’s even possible, therefore, that you’ll see a different picture after a server reboot.
and I was looking at the blockstorage guide here and it’s more than just “buy and connect it” so I don’t want to risk moving and changing stuff without one