Disk space too full to update Discourse instance

Support Self-hosting
1

Hello,

I have a small Discourse instance on a server with 25GB of space. Presently it only has 3.4 GB free, and I’m not sure why. This is preventing me from running updates.

There are no services running on this instance besides Discourse, nor have there ever been

So far to free up space I’ve:

  • deleted every log I could find both from Discourse and from /var/log/journal
  • deleted all but one of my Discourse backups
  • pruned all old docker images
  • ran apt-get autoclean and apt-get autoremove
  • ensured I didn’t have old kernals

I considered moving my postgres data to an external drive, but it’s less then a GB right now so I don’t think that will help enough.

/dev/vda1 is using 20 GB of space.

Is there anything else I can do? Is 25GB no longer a reasonable amount of disc space to run Discourse on?

2

It depends on mostly your uploads but also backups and postgres if your site is very large (which it sounds like is not the problem). How large are your uploads? The admin dashboard should say at the bottom.

3

It’s less then 50MB, this isn’t a big or old instance at all. As I mentioned, I considered moving it off the main drive but I don’t think that’s the major issue here.

4

If your uploads are ~25MB and your postgres is ~1GB I would suspect that Discourse is making backups you don’t need. I would check your backup retention settings (Admin > Settings > Backups) and look at your current backups (Admin > Backups) and delete any you don’t need.

Nevermind. Would help if I remembered you already cleared them.

You can check what’s using the space in the common places:

du -sh /shared/*
du -sh /var/www/discourse/*
du -sh /var/www/discourse/public/*
5

I deleted the one backup remaining after copying it off disk. I still only have 3.5 GB left.

6

Yes apologies, I just updated my post with updated instructions.

7

This server doesn’t have a /shared/ directory.

Here’s what I got from du -sh /var/discourse/*, only 658M in total:

4.0K    /var/discourse/LICENSE
12K     /var/discourse/README.md
4.0K    /var/discourse/bin
4.0K    /var/discourse/cids
8.0K    /var/discourse/containers
12K     /var/discourse/discourse-doctor
8.0K    /var/discourse/discourse-setup
368K    /var/discourse/image
8.0K    /var/discourse/install-discourse
28K     /var/discourse/launcher
32K     /var/discourse/samples
8.0K    /var/discourse/scripts
657M    /var/discourse/shared
180K    /var/discourse/templates

The biggest use of space in the file system are /var/lib/containerd/io.containerd.content.v1.content at 2.4 GB and /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs at a whooping 12 GB

8

Sorry, forgot to mention you needed to be in the container. The file you mentioned is from Docker. You can check what it’s being used by with docker system df and do a global prune with docker system prune if required, just make sure Discourse is running when you do so.

9

Ah, that’d make sense.

In the container, the only thing is larger then a GB is /var/www/discourse/vendor/bundle/ruby/ at 2.1 GB

I will add if I prune the image the update pulls, I have enough space, but when I attempt to update and it pulls it again, I lose it. So almost 5GB of space is the new image.

10

Are you able to show the output of that docker command? That directory is all of the Ruby gems and that size doesn’t seem alarming to me.

11

Indeed, please show all your outputs. I wouldn’t recommend looking around inside the container - your problem is at system level, and that’s where to start looking. And don’t look where you think there might be a problem, but look at the whole picture.

First, reboot. You want a clean filesystem and a fresh system.

Next, see this previous thread for detailed advice:

12

Have you run a

./launcher cleanup

Or otherwise pruned docker?

It is difficult to get by with 25gb except for the smallest forums.

13

my forum is a small forum (really mostly just me) and I use a VPS with 40gb just to be safe and not have to worry about updates causing heart ache

It is the only one out of my 3 VPSs that I don’t use 20gb - the others are an http server with MySQL and PHP back end and an rtmp server. The one running discourse also runs docker-mailserver but I have digest mail turned off and dms has a tiny footprint

14

This will likely solve your problem.

15

To gain a little space, I use these commands, it can help going through the rebuild

sudo apt -y autoremove && apt -y autoclean
sudo apt clean
sudo journalctl --vacuum-time=1s

1 and 2 clean ubuntu
3 clean some logs, it usually cleans around 3gb

16

This was in the OP but it’s obviously better to run the Discourse command

17

Thank you for your responses! Sorry for vanishing, life came up and I didn’t have time yesterday to do more troubleshooting.

The TLDR is I noticed how many threads there are on this exact problem and figured that even if I resolve it this time it will come up again, so I moved my instance to a slightly larger server with 35GB of space on it.

For anyone looking here for help with this exact issue who doesn’t want to upgrade:

./launcher cleanup was something I had already tried, it didn’t clear up enough space. I had also already done

sudo apt -y autoremove && apt -y autoclean
sudo apt clean
sudo journalctl --vacuum-time=1s

And again, it wasn’t enough. There was a lot (for a 25GB machine) of logs in journalctl, so absolutely check if that’s true for you, but for me it didn’t help enough.