/var/lib/docker/overlay2 took all the disk space


(Josh) #1

Hi, I installed discourse today (my first time using discourse :grinning:) and I’ve installed 3 sites in 3 different containers. I had some troubles and had to rebuild a site several times. My disk is full now. I’ve run this to clean space:
apt autoclean
apt autoremove
./launcher cleanup
docker system prune

But it’s still full. Almost all the disk space is take by this folder: /var/lib/docker/overlay2

As I only have 3 sites and in the mentioned folder and there are 8 folders taking more than 1GB: are there folders in /var/lib/docker/overlay2 that are not needed anymore? Can I delete them?

Thanks!

--- /var/lib/docker/overlay2 -----------------------------------------------------------------------------------------------------------------------------------------------------------------
    2.6 GiB [##########] /ec2fd5ec97616ecf81beadaa2054db614d023a137ae7a7356cf045164bdba296                                                                                                    
    2.6 GiB [######### ] /89dedb46c2b9c52d3fef241c5e1f2dba8e01af5ec5ddaa3fdca386b1ae2985d6
    2.0 GiB [#######   ] /474cf98c95c5c8532bacc11bec063fa0d51b6924e08a2375e30db79705f60e50
    1.8 GiB [######    ] /b9d8e6b857365ebbe0c0515f85f5961568563bbbdf9585a83ec4d4a65d06d489
    1.7 GiB [######    ] /486137a6ffef2ee5a0425a82dc637c0c60fc30eda7e577f2ba1cadb188163d4c
    1.3 GiB [####      ] /8daad3bdd055d98d5b91df87f1f7480de14b94be1834a383539dabc1a205abe8
    1.3 GiB [####      ] /73a33569960048127de37171666689d5514e31fdbf5d8b6d049d288c377d5de3
    1.0 GiB [###       ] /23b849361f6cd3b8fc2c41fb9fd11d1a203af41263f67a6d55ca241e142ef0b5

#2

You should do a cleanup

cd var/discourse
./launcher cleanup app

It should be fixed after that


(Josh) #3

Thanks, I’ve done it for all my 3 apps and disk usage is still the same


(Jay Pfaffman) #4

How much space do you have?


(Josh) #5
> df -h                                                                                                                                                                                52.490s
Filesystem      Size  Used Avail Use% Mounted on
udev            967M     0  967M   0% /dev
tmpfs           196M  8.2M  188M   5% /run
/dev/sda1        19G   13G  5.9G  68% /
tmpfs           977M     0  977M   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           977M     0  977M   0% /sys/fs/cgroup
overlay          19G   13G  5.9G  68% /var/lib/docker/overlay2/ec2fd5ec97616ecf81beadaa2054db614d023a137ae7a7356cf045164bdba296/merged
overlay          19G   13G  5.9G  68% /var/lib/docker/overlay2/89dedb46c2b9c52d3fef241c5e1f2dba8e01af5ec5ddaa3fdca386b1ae2985d6/merged
shm              64M  4.0K   64M   1% /var/lib/docker/containers/547fd63b76977441b23ef35348de4759e7267e8557d5d942b9388596889f0fea/mounts/shm
shm              64M  4.0K   64M   1% /var/lib/docker/containers/7615c950924a5db70655a1e6df89d2033ed6adb05471eaecbc95460205701765/mounts/shm
tmpfs           196M     0  196M   0% /run/user/0
overlay          19G   13G  5.9G  68% /var/lib/docker/overlay2/1e74eb75dca7856b264533a6f27dc7a7da4b1fe8b7c577f5a0670da09d8329c3/merged
shm              64M  8.0K   64M   1% /var/lib/docker/containers/9f28c71129366e5614ef57c2942a428f32aa3861aa752c3e20b7b8bdfe3840e3/mounts/shm

(Michael Friedrich) #6

Which OS and Docker version?

/var/lib/docker/overlay2 isn’t really a mount point on its own, / is running full. See this issue for details: overlay2 infinitely eats server disk space · Issue #33775 · moby/moby · GitHub

I’d use ncdu which allows to deeply analyze directories and figure out which of them take disk space. Sometimes it is application data, temporary files, databases or logs even which grow over time. Note: ncdu / might take some time.


(Jay Pfaffman) #7

The 3 ~2GB containers are Discourse. I’m not quite sure what the others are.

That said, 20GB is tight for even a single instance.


(Josh) #8

Thanks,

Jay: It’s a new server and I only installed discourse (3 different sites), and they are not live, so, all containers must be Discourse containers.
Michael: I will look into it


(Jay Pfaffman) #9

Yes, they are.

You can use docker images to list the images. But, really, you don’t have enough disk space to host 3 discourses.