Discouse takes up a lot of disk space, mainly the redis_data folder

[Alienazk]

Hello!
I’ve been running the community on discouse for almost a year now and my backup files are now up to 2.5G.
But we are running out of disk space (120G).
After investigation, the folder redis_data is taking up 59G. Is there any way to fix this?

[Alienazk]

Some fragments of the redis file are as follows:
Actually, I don’t quite understand what this represents.

{"data":{"message_type":"@3","topic_id":45450,"paylo  1last_@'_po 	number":14,"notification_level ,"highe *?/	29}},"user gs":[`?],"group?
                                                                                   null,"client?l}r@?@?@?15673327|288|/unread/45852$$$123 default|{"data":{"message_type":"@3","topic_id":1893,"paylo  0last_@&_po 	number ',"notification_level`highe )?2}},"user ds":[`?],"group?
                                     null,"client?l}r@?@?@?15673290|287|/unread/45852$$$123 default|{"data":{"message_type":"@3","topic_id":2249,"paylo  0last_@&_po 	number":5,"notification_level >,"highe )?.	12}},"user es":[`?],"group?
 null,"client?l}?q@?@?@?15673285|286|/unread/45852$$$123 default|{"data":{"message_type":"@3","topic_id":2249,"paylo  0last_@&_po 	number":3,"notification_level >,"highe )?.	12}},"user es":[`?],"group?
                                                                                           null,"client?l}?q@?@?@?15672031|285|/unread/45852$$$123 default

[merefield (Robert)]

Take a look at:

Is secure storage for Redis necessary?

Can you define “history” here? Losing the Redis DB (equivalent of redis-cli flushall) is something the Discourse app will recover from without major problems, but you will lose some stuff, like emails that were in a queue, etc. So while it’s not something catastrophic, if you can avoid it without much hassle I’d recommend you to.

[Alienazk]

Thank u! I’ll take a look at it!

[Alienazk]

-rw-r--r-- 1 messagebus messagebus 335833264 Feb 16 09:41 dump.rdb
-rw-r--r-- 1 messagebus messagebus         0 Jan 31 15:09 temp-1015520.rdb
-rw-r--r-- 1 messagebus messagebus   4919296 Jan 31 16:29 temp-1022742.rdb
-rw-r--r-- 1 messagebus messagebus 273073065 Jan 18 16:12 temp-1024589.rdb
-rw-r--r-- 1 messagebus messagebus 380357223 Jan 31 17:12 temp-1026744.rdb
-rw-r--r-- 1 messagebus messagebus 130723840 Jan 18 16:45 temp-1027939.rdb
-rw-r--r-- 1 messagebus messagebus 319178051 Jan 18 16:48 temp-1028173.rdb
-rw-r--r-- 1 messagebus messagebus 203666791 Jan 31 17:27 temp-1028295.rdb
-rw-r--r-- 1 messagebus messagebus 352884828 Jan 18 17:27 temp-1032450.rdb
-rw-r--r-- 1 messagebus messagebus 307757056 Jan 18 17:40 temp-1034133.rdb
-rw-r--r-- 1 messagebus messagebus 366613731 Jan 31 18:42 temp-1034836.rdb
-rw-r--r-- 1 messagebus messagebus   2502656 Jan 31 18:47 temp-1035328.rdb
-rw-r--r-- 1 messagebus messagebus 151254653 Jan 18 18:00 temp-1037002.rdb
-rw-r--r-- 1 messagebus messagebus 244047872 Jan 18 18:08 temp-1037784.rdb
-rw-r--r-- 1 messagebus messagebus    532480 Jan 18 18:12 temp-1038150.rdb
-rw-r--r-- 1 messagebus messagebus 128897024 Jan 18 19:14 temp-1043997.rdb
-rw-r--r-- 1 messagebus messagebus  28532736 Jan 18 19:19 temp-1044418.rdb
-rw-r--r-- 1 messagebus messagebus 326288698 Jan 18 20:16 temp-1049000.rdb
-rw-r--r-- 1 messagebus messagebus  34062520 Feb  1 02:03 temp-1066798.rdb
-rw-r--r-- 1 messagebus messagebus    442368 Jan 19 01:55 temp-1072983.rdb
-rw-r--r-- 1 messagebus messagebus   3805184 Feb  1 11:14 temp-1111400.rdb
-rw-r--r-- 1 messagebus messagebus         0 Jan 19 15:54 temp-1138306.rdb
-rw-r--r-- 1 messagebus messagebus   3158016 Feb  1 17:00 temp-1139656.rdb
-rw-r--r-- 1 messagebus messagebus         0 Feb  1 17:02 temp-1139836.rdb
-rw-r--r-- 1 messagebus messagebus 152346624 Feb  1 17:04 temp-1140054.rdb
-rw-r--r-- 1 messagebus messagebus 305575386 Feb  1 17:06 temp-1140297.rdb
-rw-r--r-- 1 messagebus messagebus         0 Feb  1 17:11 temp-1140838.rdb

The redis files look like …
they all seem to be messagebus files, is it safe to delete them?

[merefield (Robert)]

I just looked in that folder on my healthy production site and there are no files called temp-*, only dump.rdb

Given they are named “temp” you would have thought they are safe to delete, however, you might want to use the official command from within the container and then check they are deleted. (ie redis-cli flushall )

If you are less brave I’d hang on until one of the experienced sys admins on here can weigh in.

[Alienazk]

Thank u Robert
I am trying to google it!

[pfaffman (Jay Pfaffman)]

Looks safe to delete the temp files.

Remove temporary files from disk caused by Redis command BGREWRITEAOF

[merefield (Robert)]

It is strange they aren’t being house kept. I assume this machine is perpetually on?

[pfaffman (Jay Pfaffman)]

From my nascent understanding, those temp files get left behind if redis stops before the entire file is written out. The database stays in memory, so the dump is just to keep data between runs (e.g. The container is rebuilt, or machine reboots).

[merefield (Robert)]

@pfaffman

if redis stops before the entire file is written out.

That makes sense. But why would it stop? Would a ./launcher rebuild app do that? (redis is running inside the container so could do?) Or does it need a VPS halt or power cycle?

[pfaffman (Jay Pfaffman)]

Rebuild tried to do a clean shutdown, but I think it forces it after a certain amount of time. Reboots are required to apply kernel and library updates.

[Alienazk]

Thank you! I’ll delete them