# Disk space usage of /lib/docker is high

**URL:** https://meta.discourse.org/t/disk-space-usage-of-lib-docker-is-high/67375
**Category:** Self-hosting
**Tags:** server-resources
**Created:** [August 3, 2017, 12:12pm UTC](https://meta.discourse.org/t/disk-space-usage-of-lib-docker-is-high/67375 "2017-08-03T12:12:59Z")
**Posts on this page:** 12
**Page:** 1

<div class="post-metadata">

### Author: ![david](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/david/32/157490_2.png) [@david](https://meta.discourse.org/u/david)
#### Post date: [August 3, 2017, 12:12pm UTC](https://meta.discourse.org/t/disk-space-usage-of-lib-docker-is-high/67375/1 "2017-08-03T12:12:59Z")

</div>

Ran into a disk space issue on one of my droplets this morning. Discourse has almost filled the 30GB

> [@Can't install Discourse with only 10 GB disk, run out of space](https://meta.discourse.org/t/cant-install-discourse-with-only-10-gb-disk-run-out-of-space/29149/7):
>
> it looks like a Discourse install takes 5,424,088 or 5.4 GB.

Using `ncdu`, I’m getting

```plaintext
/discourse 6.4GB
/lib/docker 18.2GB

```

There are a fair number of posts & uploads, so I think the `/discourse` number is as expected. Drilling down into `/lib/docker`:

```plaintext
/lib/docker/overlay 9.7GB
/lib/docker/devicemapper 8.1GB

```

So, two questions:

- Is `9.7GB` an “normal” level of usage for the overlay folder?
- Now that I moved from devicemapper to overlay, does anyone know if it’s safe to delete the devicemapper folder? Or do I need to run a docker command to clean it up somehow?

---

<div class="post-metadata">

### Author: ![Falco](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/falco/32/179432_2.png) [@Falco](https://meta.discourse.org/u/Falco)
#### Post date: [August 3, 2017, 12:15pm UTC](https://meta.discourse.org/t/disk-space-usage-of-lib-docker-is-high/67375/2 "2017-08-03T12:15:54Z")

</div>

> [@david](#):
>
> Or do I need to run a docker command to clean it up somehow?

If you are on Docker 17.x a `docker system prune` should be safe. (Take a backup before, just in case)

---

<div class="post-metadata">

### Author: ![david](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/david/32/157490_2.png) [@david](https://meta.discourse.org/u/david)
#### Post date: [August 3, 2017, 12:18pm UTC](https://meta.discourse.org/t/disk-space-usage-of-lib-docker-is-high/67375/3 "2017-08-03T12:18:21Z")

</div>

```plaintext
sudo docker system prune

```

returns

```plaintext
Total reclaimed space: 0B

```

* * *

```plaintext
sudo docker system df

```

gives me

```plaintext
TYPE TOTAL ACTIVE SIZE RECLAIMABLE
Images 4 2 2.181GB 1.718GB (78%)
Containers 2 2 3.933GB 0B (0%)
Local Volumes 0 0 0B 0B

```

So looks like it’s lying about “reclaimable”… But even if it could cleanup 1.7GB, that’s not the whole of devicemapper.

---

<div class="post-metadata">

### Author: ![david](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/david/32/157490_2.png) [@david](https://meta.discourse.org/u/david)
#### Post date: [August 3, 2017, 1:04pm UTC](https://meta.discourse.org/t/disk-space-usage-of-lib-docker-is-high/67375/4 "2017-08-03T13:04:51Z")

</div>

I did an `rm -rf` on `/var/lib/docker/devicemapper` and nothing exploded, even with a reboot. So hopefully that’ll be ok.

Still interested to know whether `9.7GB` is a ‘normal’ level of overlay usage?

---

<div class="post-metadata">

### Author: ![sam](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/sam/32/102149_2.png) [@sam](https://meta.discourse.org/u/sam)
#### Post date: [August 11, 2017, 5:37pm UTC](https://meta.discourse.org/t/disk-space-usage-of-lib-docker-is-high/67375/5 "2017-08-11T17:37:00Z")

</div>

> [@david](#):
>
> 9.7GB is a ‘normal’ level of overlay usage?

Are you certain that is actual usage and not counting layers multiple time despite small delta?

---

<div class="post-metadata">

### Author: ![david](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/david/32/157490_2.png) [@david](https://meta.discourse.org/u/david)
#### Post date: [August 11, 2017, 5:38pm UTC](https://meta.discourse.org/t/disk-space-usage-of-lib-docker-is-high/67375/6 "2017-08-11T17:38:44Z")

</div>

Ooh I hadn’t thought of that. I just used `du` - what’s the best way to determine whether it’s counting things twice or not?

---

<div class="post-metadata">

### Author: ![sam](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/sam/32/102149_2.png) [@sam](https://meta.discourse.org/u/sam)
#### Post date: [August 11, 2017, 6:00pm UTC](https://meta.discourse.org/t/disk-space-usage-of-lib-docker-is-high/67375/7 "2017-08-11T18:00:39Z")

</div>

No idea, this is just me guessing 🧛🏽

---

<div class="post-metadata">

### Author: ![david](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/david/32/157490_2.png) [@david](https://meta.discourse.org/u/david)
#### Post date: [August 11, 2017, 7:30pm UTC](https://meta.discourse.org/t/disk-space-usage-of-lib-docker-is-high/67375/8 "2017-08-11T19:30:06Z")

</div>

Looking at the `man` page, straight `du` is clever about hard links

```plaintext
$ du -sch overlay
9.7G	overlay
9.7G	total

```

With `-l` it counts every occurrence of the hard link as a unique file, so that gives an even higher reading:

```plaintext
$ du -schl overlay
15G	overlay
15G	total

```

So I don’t think the `9.7GB` is caused by the layers - unless there’s some method other than hard links that could be tripping me up.

---

<div class="post-metadata">

### Author: ![angus](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/angus/32/341715_2.png) [@angus](https://meta.discourse.org/u/angus)
#### Post date: [September 1, 2017, 11:32am UTC](https://meta.discourse.org/t/disk-space-usage-of-lib-docker-is-high/67375/9 "2017-09-01T11:32:02Z")

</div>

I encountered a similar issue; perhaps the same, it’s hard to tell. This is what fixed it (after a fashion) for me:

> [@Docker disk usage issue c. Sept 1 2017](https://meta.discourse.org/t/docker-disk-usage-issue-c-sept-1-2017/69194):
>
> If you do a fresh install, as I did recently for my [sandbox](https://discourse.angusmcleod.com.au/), you may, despite all the improvements in the cleanup scripts and docker itself, still encounter the dreaded disk usage creep issue; i.e. after rebuilding a number of times you see your disk usage go steadily upwards until the instance is no longer usable. This issue has had a number of iterations (hence ‘circa Sept 1 2017’). The current iteration seems to be un-removed aufs/diff files. Specifically, after a fresh Discourse install on…

---

<div class="post-metadata">

### Author: ![blau](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/blau/32/34145_2.png) [@blau](https://meta.discourse.org/u/blau)
#### Post date: [May 10, 2018, 12:48pm UTC](https://meta.discourse.org/t/disk-space-usage-of-lib-docker-is-high/67375/10 "2018-05-10T12:48:29Z")

</div>

I was low on disk space and could not upgrade discourse nor free up space. Turns out `/var/lib/docker/aufs/diff` was the culprit. This is what worked for me:

- upgrade system software, autoremove, autoclean and reboot to new kernel;
- reinstall docker via [get.docker.io](http://get.docker.io)
- `/var/discourse/launcher cleanup` (recover 4GB space)
- `docker system prune` (recover 1.4 GB)

I was on an old 17.x docker version and after several `./launcher rebuild app` I had less than 5GB free on a 20 GB disk. Same issue as discussed here (thread closed):

> [@Docker disk usage issue c. Sept 1 2017](https://meta.discourse.org/t/docker-disk-usage-issue-c-sept-1-2017/69194/8):
>
> I just did some tests and found that to get an accurate count of disk usage for /var/lib/docker, you need to pass the -x option to du so it doesn’t count files in mounted overlay file systems. Stopping docker, which unmounts the overlay filesystems will also work but of course is less convenient then just using the -x option for du. At least this is the case when using the overlay2 docker backend, I would assume it’s the same for most of the docker backends. Edit: Strangely the sizes reported…

---

<div class="post-metadata">

### Author: ![sam](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/sam/32/102149_2.png) [@sam](https://meta.discourse.org/u/sam)
#### Post date: [May 10, 2018, 10:00pm UTC](https://meta.discourse.org/t/disk-space-usage-of-lib-docker-is-high/67375/11 "2018-05-10T22:00:21Z")

</div>

Sometimes what I do is just junk `/var/lib/docker` altogether and start from scratch (stop docker and all containers first, we store nothing important there), perhaps try that.

---

<div class="post-metadata">

### Author: ![Dev\_Work](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/dev_work/32/124200_2.png) [@Dev\_Work](https://meta.discourse.org/u/Dev_Work)
#### Post date: [October 14, 2020, 8:19am UTC](https://meta.discourse.org/t/disk-space-usage-of-lib-docker-is-high/67375/12 "2020-10-14T08:19:11Z")

</div>

Is there a plugin that will check the disk size and, if less than the specified value, notify me by email?
