Disk space usage of /lib/docker is high

Unterstützung Installation
1

Ran into a disk space issue on one of my droplets this morning. Discourse has almost filled the 30GB

Using ncdu, I’m getting

/discourse      6.4GB
/lib/docker    18.2GB

There are a fair number of posts & uploads, so I think the /discourse number is as expected. Drilling down into /lib/docker:

/lib/docker/overlay       9.7GB
/lib/docker/devicemapper  8.1GB

So, two questions:

  • Is 9.7GB an “normal” level of usage for the overlay folder?
  • Now that I moved from devicemapper to overlay, does anyone know if it’s safe to delete the devicemapper folder? Or do I need to run a docker command to clean it up somehow?
2

If you are on Docker 17.x a docker system prune should be safe. (Take a backup before, just in case)

3 
sudo docker system prune

returns

Total reclaimed space: 0B

sudo docker system df

gives me

TYPE                TOTAL               ACTIVE              SIZE                RECLAIMABLE
Images              4                   2                   2.181GB             1.718GB (78%)
Containers          2                   2                   3.933GB             0B (0%)
Local Volumes       0                   0                   0B                  0B

So looks like it’s lying about “reclaimable”… But even if it could cleanup 1.7GB, that’s not the whole of devicemapper.

4

I did an rm -rf on /var/lib/docker/devicemapper and nothing exploded, even with a reboot. So hopefully that’ll be ok.

Still interested to know whether 9.7GB is a ‘normal’ level of overlay usage?

5

Are you certain that is actual usage and not counting layers multiple time despite small delta?

6

Ooh I hadn’t thought of that. I just used du - what’s the best way to determine whether it’s counting things twice or not?

7

No idea, this is just me guessing :vampire:t4:

8

Looking at the man page, straight du is clever about hard links

$ du -sch overlay
9.7G	overlay
9.7G	total

With -l it counts every occurrence of the hard link as a unique file, so that gives an even higher reading:

$ du -schl overlay
15G	overlay
15G	total

So I don’t think the 9.7GB is caused by the layers - unless there’s some method other than hard links that could be tripping me up.

9

I encountered a similar issue; perhaps the same, it’s hard to tell. This is what fixed it (after a fashion) for me:

10

I was low on disk space and could not upgrade discourse nor free up space. Turns out /var/lib/docker/aufs/diff was the culprit. This is what worked for me:

  • upgrade system software, autoremove, autoclean and reboot to new kernel;
  • reinstall docker via get.docker.io
  • /var/discourse/launcher cleanup (recover 4GB space)
  • docker system prune (recover 1.4 GB)

I was on an old 17.x docker version and after several ./launcher rebuild app I had less than 5GB free on a 20 GB disk. Same issue as discussed here (thread closed):

11

Sometimes what I do is just junk /var/lib/docker altogether and start from scratch (stop docker and all containers first, we store nothing important there), perhaps try that.

12

Gibt es ein Plugin, das die Festplattengröße prüft und mich per E-Mail benachrichtigt, wenn sie unter einen bestimmten Wert fällt?