The user Vikash_Kumar is experiencing excessive disk usage from the redis_data and postgres_data folders, consuming over 20GB and causing performance issues on their DigitalOcean server. The issue appears to stem from Redis generating large dump.rdb files, which are not being properly cleaned up, likely due to a failed SMTP service causing queue buildup and Redis crashes. Community members Ed_S, Jagster, and pfaffman suggest investigating Redis memory usage, checking for system resource constraints, and cleaning up temporary files. After fixing the SMTP service and deleting temporary files, the immediate disk usage issue was resolved, but the postgres_data folder remains a concern. pfaffman recommends running ./launcher cleanup to address lingering data.
What I can do about this? redis_date & Postgres_date are eating my 20GB+ Space
Due to this, I am paying more on the digital ocean.
My forum: https://onehourforum.com/
Everytime these two folder are taking much of my space, and my forum is getting down.
Please help.
Screenshot attached.
For me redis_data is very small indeed. What do you get from
ls -haltrR /var/discourse/shared/standalone/redis_data
I have
root@foorumi-hel:/var/discourse# ls -haltrR /var/discourse/shared/standalone/redis_data
/var/discourse/shared/standalone/redis_data:
total 9.3M
drwxr-xr-x 13 root root 4.0K Jan 10 2023 ..
-rw-r--r-- 1 messagebus input 9.3M Dec 6 20:07 dump.rdb
drwxr-xr-x 2 messagebus input 4.0K Dec 6 20:07 .
So Vikash must have an issue indeed.
Agreed. I can’t imagine how there could be 16GB of redis. If you move to a new server, what’s in redis (typically) gets left behind, so one thing to do would be to just delete it and rebuild.
You can see what this says:
cd /var/discourse
./launcher enter app
redis-cli
info memory
exit
exit
exit
Also, for interest, at the bottom of your forum’s sidekiq status page (forum.url/sidekiq) there’s a summary of Redis memory usage, including peak usage. (Also, uptime - might be interesting if uptime is very short, because maybe redis is crashing and restarting and leaving some disk clutter.)
(However, that’s memory usage, and the problem here seems to be disk usage.)
Presumably you can delete all the temp files.
Is this a standard install? Does it have a history of being migrated from another machine? A quick search indicates that some kind of permissions problem, or other reason for redis to fail while saving/renaming, could be the cause.
This previous thread was, for me, inconclusive:
Discouse takes up a lot of disk space, mainly the redis_data folder
I think that they get left behind if redis shuts down before it can write it out. (But had no memory of writing that, so maybe I have it left in a temp file solfege/somewhere)
Those dates in the screenshot are in clumps at roughly 5 minute intervals - something isn’t working as it should!
I didn’t notice that!
There’s certainly something wrong. It’s this an underpowered machine? It’s it running a bunch of AI stuff? (I don’t think so, but those are my best guesses)
If safe to delete them, which solves the immediate disk space issue.
As a general checkup, which includes a check for processes being killed due to shortage of memory:
The smtp service was stopped,
and the queue were there,
is that an issue?
What I did?
fixed smtp, and deleted temp files.
That could explain it.
It will be interesting to see if those temp files are still being created.
You can try
cd /var/discourse
./launcher cleanup