Page loading extremely slow and /docker/loverlay2 using large disk space

Installation
1

Running discourse under docker and recently finding discourse loading became extremely slow, df -h shows that

# df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            1.9G     0  1.9G   0% /dev
tmpfs           500M   41M  460M   9% /run
/dev/sda1       117G   60G   51G  55% /
tmpfs           2.0G   18M  2.0G   1% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           2.0G     0  2.0G   0% /sys/fs/cgroup
tmpfs           394M     0  394M   0% /run/user/0
overlay         117G   60G   51G  55% /usr/bin/wbin/data/overlay2/dfbf19c2da072a3a52c868284237d52aee235766d694ab0c5ed1da211aef7970/merged
overlay         117G   60G   51G  55% /usr/bin/wbin/data/overlay2/d41d8ad7053ece6fc41f3a00932ab6b700cec2f290a246bcfbdcd13e4e222bd7/merged
overlay         117G   60G   51G  55% /usr/bin/wbin/data/overlay2/cff77a73b3d615a01fb175ae6a0e1c0d5edc5d5b9ac9fe5ac9f67169632185ed/merged
overlay         117G   60G   51G  55% /usr/bin/wbin/data/overlay2/e0b89f758a0cc4bf2cb076204816366de559c021971099aac4ea1ce29c49364c/merged
overlay         117G   60G   51G  55% /usr/bin/wbin/data/overlay2/a370679848430d996e58b7d515a4fba887b2065adf660cfd176e100c19eb6eaa/merged
overlay         117G   60G   51G  55% /usr/bin/wbin/data/overlay2/f64c527ff68cc67bc0b2d0070e8da3cbd07679e57c51b47f66d438416003e15a/merged
overlay         117G   60G   51G  55% /usr/bin/wbin/data/overlay2/d1c4b8fe2752b999cb116d0c82d00adf1956dc940fce1eb27f01db05875e295a/merged
overlay         117G   60G   51G  55% /var/lib/docker/overlay2/8fe53ea9157ba3ae2ae42674dd13bcdab5cd50d70d220356698dd52cbf13e967/merged
tmpfs           394M     0  394M   0% /run/user/1000

and /overlay2/*/diff is also very large

# du -sh /var/lib/docker/overlay2

12G	/var/lib/docker/overlay2

# du -shc /var/lib/docker/overlay2/*/diff
7.4M	/var/lib/docker/overlay2/016bf588971d95bf43ef81c16ef4153f08cceded358bd91367551244465001cf/diff
12K	/var/lib/docker/overlay2/06d01901014ac62536c329d5779d2bfea29a5258b1c8142e56e18c7a396c981e/diff
12K	/var/lib/docker/overlay2/072a59e6d900b0a4588ae1d0dc8a776254ec314b57394691667a565c35df93fe/diff
12K	/var/lib/docker/overlay2/1825b4f8dcf2422c625663644c21962b5b45d52c1a0241e6823a6999d72954c9/diff
284K	/var/lib/docker/overlay2/1e2679b47851aa21bf68394f6dff36ae591fc4d67d72ae0ac94ccbbd73008227/diff
332K	/var/lib/docker/overlay2/1ff99ef050fa8b5047f698261a4bc69f945890b64f345afb298c8bace1adc444/diff
88M	/var/lib/docker/overlay2/20f2a30ae594f7310b0f7a094555967ae17555581b8b51a7b14ac7fe85f3645b/diff
12K	/var/lib/docker/overlay2/29da7c333dd4b760bf344e0eb581e7643a5bcdd24ecc6e53cd8d2e424b5cdfb3/diff
575M	/var/lib/docker/overlay2/33af770bb548e08a8a3c347268910065ed2f1dd4bf3b224c9ad0b5fe0412a4ff/diff
30M	/var/lib/docker/overlay2/3b50de4a8a53280b1ee6368655ece448c3a1db0d02d3326b856c1a1e53fdedbe/diff
12K	/var/lib/docker/overlay2/3c36e0bf023604784223d0acfffde3dc01daebd9fbf99e3c03aa1ff0b8fec964/diff
7.7M	/var/lib/docker/overlay2/3ccfebc03f1457e2088becc5ae6347ce473e383006982ba36884e4ef8c9f4909/diff
12K	/var/lib/docker/overlay2/3d66a1ace242982d6ba5dc202262658723efa398b626821413376d04117784cc/diff
16K	/var/lib/docker/overlay2/465dcaacbb755d2bfff282c374898d50cb67eaad28f0102744c26ec7c3395533/diff
30M	/var/lib/docker/overlay2/46d2409f27fa087c2d9b8e645af9dd6ed1447d3db94c589f53db14f5017d9504/diff
202M	/var/lib/docker/overlay2/4c67a4021ba2ec84586302df10f351f8667c2b734f47eb4b3eb28abd97d1fc7c/diff
1.4G	/var/lib/docker/overlay2/5073c622adb6762a95514c19a1affe00e0a58321d1ce68eeec60fc617f996264/diff
68K	/var/lib/docker/overlay2/59eaffea8fe6d7353ecbf0ef9c3c0e172193bbc93bcfa8a33530ffc9d6d438a3/diff
800K	/var/lib/docker/overlay2/5a67e03203f8573eab56db3b8496656391f35d6054ad69e7c8e64309f2ad192e/diff
8.0K	/var/lib/docker/overlay2/5d419ae4d604954af86ae0e2ada33633d6036dbb23778f775af3753265b7d9c5/diff
20K	/var/lib/docker/overlay2/5d9bd9427b012004f59ec9538ee8af2982a435f16a83933f2b7a1839d2447ef7/diff
8.0K	/var/lib/docker/overlay2/5fd27e0796e83b9a12a6ef81a36e1af06cabd8944ff3581a537a81042f817a8f/diff
16K	/var/lib/docker/overlay2/62235a7c903e965446abc07b4e35ab3b8f2bbc094fcb1e1ff6c4774f33c1ea92/diff
4.0K	/var/lib/docker/overlay2/6dba93306c140cce13a657942f6c9320f45b46da0be8b0fb7cf784d3ea3c2fdd/diff
179M	/var/lib/docker/overlay2/6e79620c6a5a8e183f41d06e8e71776cfe3d827f6ab644452e8f2a397d536113/diff
37M	/var/lib/docker/overlay2/738b42e633dfaaf10d250d535b3e81187cd2ece7834f3c8ad0e8b822515763f2/diff
1.8G	/var/lib/docker/overlay2/77cb1343e1f64990e5f5990729f101b917ce95e92e89d753d3a3e594196341dd/diff
242M	/var/lib/docker/overlay2/78f29c2713c65382ebf30f4aa4c324cede389a8bc27f9dd9478acdf7836c7c83/diff
222M	/var/lib/docker/overlay2/8ba232f250e7e1cb0e85523606858aac4c5f653136b80831dd986d3eec52d3a8/diff
164M	/var/lib/docker/overlay2/8fe53ea9157ba3ae2ae42674dd13bcdab5cd50d70d220356698dd52cbf13e967/diff
20K	/var/lib/docker/overlay2/8fe53ea9157ba3ae2ae42674dd13bcdab5cd50d70d220356698dd52cbf13e967-init/diff
44K	/var/lib/docker/overlay2/910816bc0e720e928e2b0d0cf365e4fe0b8a98769667dec5cc648095ea7e0fa2/diff
32K	/var/lib/docker/overlay2/9c0221ef6821d4d69b9f657260b354f1991530ec87f0eebe63d95cbb3fb55cec/diff
30M	/var/lib/docker/overlay2/a336ca295e58fb9228db6e19a591c735e0c2da553056ebc2f2e66a63a4627330/diff
8.0K	/var/lib/docker/overlay2/a6faab942fa10a878918a2c00a69e63e6f65dc114834302acefd86807cbe407c/diff
47M	/var/lib/docker/overlay2/ae6d4be8d39a6ef77113443eff63b21e956ecee745c36f70280048f5bbb37109/diff
300M	/var/lib/docker/overlay2/b3592e37ba5b6eee777e2665c75444f5198eeb26c05d9b0b42e14a8c51d84d4a/diff
125M	/var/lib/docker/overlay2/c31e2fdec1d444a99b6d95c03b7f18f68e8782211da895d669c7eb839c343f63/diff
16K	/var/lib/docker/overlay2/c5075366bc3a68a112e010c41fec051238593845352db0ce97c83038c6bfda8d/diff
16K	/var/lib/docker/overlay2/c59842c114123d6ea40499d870f0b1ee1e9d527cf3cb7b48fea8b6d4e866e929/diff
80K	/var/lib/docker/overlay2/c5d534cbe4470c3141e9b664a9fa5b0284ff990c79d5d01657a6d65910fa6ddd/diff
40K	/var/lib/docker/overlay2/cd4c3c7059bffc4736f8fdf0026ecff8e7e171347aeb546cbf136a314f2e6681/diff
12K	/var/lib/docker/overlay2/ceeb6c12f226564611a46446aa0aef9632521582693ae3de4eb240605339b5e9/diff
20K	/var/lib/docker/overlay2/d2f46cb5002369b6450979586b4bbd1cd12743c61bb12cee740a94e3a04bfa06/diff
82M	/var/lib/docker/overlay2/e04f3f7d18db0307a36ba418bace6ff26acb13cc89894deb930703b5358b236b/diff
24K	/var/lib/docker/overlay2/e0df3bff1e78c6c653ada3947373036196abe131f34da32d5f4b0653546675dc/diff
20K	/var/lib/docker/overlay2/e111194861895b0ba075166851412ceeecdd9f20dad8ba144bc5f02cd4571024/diff
1.8G	/var/lib/docker/overlay2/e2e277c7d2bc941b42c9d06ea23f7caa3bef7cf9c1fdfc63f48586d5adfafda6/diff
12K	/var/lib/docker/overlay2/ee3b5b992d7b851371a09614fc44a3bd26f4f4e5dc6645c44cb47737e6bb22c2/diff
12K	/var/lib/docker/overlay2/fcb6f4112dade14e1803d182f49117a8807675a938684e520190bed363672b90/diff
7.1G	total

the biggest inside is /discourse

/var/lib/docker/overlay2/77cb1343e1f64990e5f5990729f101b917ce95e92e89d753d3a3e594196341dd/diff/var/www# ls | xargs -I {} du -shx {}
1.7G	discourse

How to fix that?

2

That’s how big it’s supposed to be.

Speed is more likely about RAM.

1 Like
3

thank you, checking it out and finding there is only a small space of the mem left

# free -h -w 1
              total        used        free      shared     buffers       cache   available
Mem:          3.8Gi       3.6Gi       100Mi       3.0Mi        15Mi       157Mi        63Mi
Swap:         6.0Gi       4.3Gi       1.7Gi

but when i use top to view cpu usage, only kswapd0 is sus taking a lot of cpu

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND  
    147 root      20   0       0      0      0 S  22.1   0.0 466:10.47 kswapd0  
 838038 qisimao   25   5 6108156 159292   3332 S  14.9   4.0  14:47.47 ruby     
1128378 qisimao   20   0 5517068 147132   4732 R   7.9   3.7   0:49.06 ruby     
1131209 qisimao   20   0 5341676  84752   3664 S   6.9   2.1   0:16.10 ruby     
1131433 qisimao   20   0 5337708  87168   3860 S   6.6   2.2   0:13.29 ruby     
1131155 qisimao   20   0 5515340 105200   4208 D   6.3   2.6   0:19.56 ruby     
1128425 qisimao   20   0 5510732 128916   3876 S   5.9   3.2   0:44.48 ruby     
1131330 qisimao   20   0 5518444  80516   4556 S   5.0   2.0   0:16.18 ruby     
1131354 qisimao   20   0 5518252  84400   4400 S   4.0   2.1   0:15.52 ruby     
    386 root       0 -20       0      0      0 I   3.0   0.0  54:33.38 kworker+ 
    388 root       0 -20       0      0      0 I   2.6   0.0  54:11.81 kworker+ 
    285 root       0 -20       0      0      0 I   2.3   0.0  55:05.82 kworker+ 
    354 root       0 -20       0      0      0 I   2.3   0.0  51:55.05 kworker+ 
1131545 systemd+  20   0 1175616   8708   3912 S   1.7   0.2   0:00.99 postmas+ 
1131902 systemd+  20   0 1184152  13764   4496 R   1.7   0.3   0:01.59 postmas+ 
1127789 systemd+  20   0 1177432  11008   3144 S   1.3   0.3   0:02.86 postmas+ 
1132448 systemd+  20   0 1176508   9156   4252 S   1.3   0.2   0:00.27 postmas+ 
 285221 www-data  20   0   61904   4356   2024 S   1.0   0.1   4:19.51 nginx    
1131347 systemd+  20   0 1184764   8096   3724 D   1.0   0.2   0:01.38 postmas+ 
1131463 systemd+  20   0 1174200   8448   3900 S   1.0   0.2   0:00.17 postmas+ 
     30 root      20   0       0      0      0 S   0.7   0.0   8:01.81 ksoftir+ 
 285210 message+  20   0  133444   8476   1100 S   0.7   0.2  13:06.39 redis-s+ 
1121070 systemd+  20   0 1177628   4712   2652 S   0.7   0.1   0:04.65 postmas+ 
1123838 systemd+  20   0 1177556   5780   2916 S   0.7   0.1   0:03.40 postmas+ 
1123880 systemd+  20   0 1179840   6788   2992 S   0.7   0.2   0:03.06 postmas+ 
1131296 systemd+  20   0 1184712   7816   3732 S   0.7   0.2   0:01.62 postmas+ 
1131744 root      20   0   38636   3280   2660 R   0.7   0.1   0:01.07 top      
     10 root      20   0       0      0      0 S   0.3   0.0   4:11.79 ksoftir+ 
     24 root      20   0       0      0      0 S   0.3   0.0   4:52.74 ksoftir+ 
   1022 root      20   0   30.4g   6412   3416 S   0.3   0.2   7:22.68 BT-Task  
 285222 www-data  20   0   63016   3008   1492 S   0.3   0.1   4:06.79 nginx    
 285223 www-data  20   0   62940   3576   1832 S   0.3   0.1   4:03.03 nginx

is that normal?

4

It is if you need more ram because the cpu is spending all of it’s time swapping.

You need more ram.

1 Like