Disk space cleanup

I run an old forum (~10 years, 200k posts) that has been running into some chronic disk issues. At the moment, we’re not in imminent danger of the lights going out, but we have reached the point where I’m getting daily messages about backups failing due to no disk space (the backup process itself seems to take like 10 gigabytes of disk, which fails if we already have a backup on-server).

Is there anything I can do to stem this tide a little?

I accept that disk space is one of those things I have to deal with as time goes on and do plan to upgrade eventually, but I’m wondering if there are other ways to buy time via compression, or just some cleanup of files that I haven’t thought to check.

I have already run ./launcher cleanup (0B saved), and have attached a snapshot of the db stats.

db_stats.txt (17.7 KB)

I’d start with a measurement:

du -kx / | sort -n | tail -499

Probably also worth

df

and perhaps

find / -xdev -ls|sort -n -k 2|tail -199

which might take a few minutes

You might see one or more swapfiles so it’s worth checking if they are in use:

swapon

After running a du, I don’t see anything out of the ordinary (the bulk of the data is in /var/docker and /var/lib, which is about what I’d expect). Do you have any advice for what might be worth checking out in those folders?

My swap also looks normal (and not really enough to move the needle).

I can help if you show the data!

Of course, how silly of me.

The output of du -cha --max-depth=2 /var | grep -E "M|G"

2.1M    /var/backups
12K     /var/discourse/README.md
2.2M    /var/discourse/.git
2.7M    /var/discourse
54M     /var/lib/mlocate
36M     /var/lib/dpkg
190M    /var/lib/apt
18G     /var/lib/docker
18G     /var/lib
3.4M    /var/log/btmp
1.2M    /var/log/auth.log.3.gz
4.1G    /var/log/journal
1.2M    /var/log/auth.log.2.gz
42M     /var/log/btmp.1
1.6M    /var/log/auth.log.4.gz
7.9M    /var/log/auth.log.1
6.5M    /var/log/auth.log
4.1G    /var/log
1.2M    /var/cache/man
312M    /var/cache/apt
2.8M    /var/cache/snapd
4.9M    /var/cache/debconf
320M    /var/cache
46G     /var/docker/shared
46G     /var/docker
68G     /var
68G     total

DIgging deeper into /var/docker/shared and /var/lib/docker:

15G     /var/docker/shared/standalone/postgres_data
6.8G    /var/docker/shared/standalone/backups
14G     /var/docker/shared/standalone/uploads
43M     /var/docker/shared/standalone/redis_data
69M     /var/docker/shared/standalone/log
12G     /var/docker/shared/standalone/postgres_data_old
14M     /var/docker/shared/standalone/letsencrypt
46G     /var/docker/shared/standalone
46G     /var/docker/shared
46G     total
17M     /var/lib/docker/image/overlay2
17M     /var/lib/docker/image
14G     /var/lib/docker/overlay2/ed229eed209ffa6339adc9de9033c12487732c74572a3dc608eb32d720d1837c
1.2G    /var/lib/docker/overlay2/7448f4eb6c5a9e09b0a5537aa454c30221ab95314418eac9078c7c774de784e2
2.4G    /var/lib/docker/overlay2/8a7ca976e2c8b362302c2abb95d306520206bf3e6125672b160b19e150d1f914
88M     /var/lib/docker/overlay2/c6cd5a3006efb2457f3bba70450a85c42f53bfc7cdc3416fb6cfb5990e2eed72
1.1G    /var/lib/docker/overlay2/9f8a463bdc03f518d6f25a7c11873122bf858a266fd6ca40ea19dfb4a78e1f8d
18G     /var/lib/docker/overlay2
18G     /var/lib/docker
18G     total

Offhand, postgres_data_old looks suspicious. Is that safe to clean up?

Yes, it’s fine to delete postgres_data_old.

I find it difficult to read the output you pasted: I suggested particular forms of commands because I’ve found them useful.