I notice on a new (<1 mth) install of discourse the backup size increasing by a steady 0.2Mb / day. [Annotation%202018-12-20%20161313] Any thoughts on what grows day by day if new posts are not added? Backups are to S3, but I’m wondering if it’s backing up backups. I haven’t torn apart the gz’s yet.

We’re not backing up the backups. Is your forum active? Do you users upload images?

Just starting. No images yet. It’s the DB growing by about 2MB / day which is about 0.2mb compressed. Not a big problem, but interesting if the growth is continuous.

I think the backup logs are causing this. The system user creates a PM for every automatic backup, but it doesn’t invite the admins unless there was an error. I think it’s safe to create the PM only when there actually was an error or if it was a manual backup. I’m going to change that. https://github.com/discourse/discourse/commit/0bc1fa8aa4ea5393193cc5c44adbf7eee24cd395

But one PM hardly generates 2MB of data each day. If the forum is being read by users, could it be the table holding the user activity logs?

I can upload the gz backups files somewhere if you like / want a look at them. They aren’t particularly large yet. It’s not a complaint. it’s a “that’s a strange steady growth rate” for a new nearly empty site. I can’t attach files here or PM I don’t think.

In my tests the backup size increased roughly 75kB because of the backup logs. There are a bunch of log tables that could increase in size even though there are no new posts. Feel free to look at a diff of the database dump if you are curious, but I don’t think there’s anything else we can improve here.

After migrating to Discourse, I was concerned that our backups seemed to be increasing far more than expected. I was seeing backups increase by up to 30MB/week (after accounting for uploads) for only ~150 posts/week. But after about three months since migrating, the backups have stabilised to just 1-3MB/week. My guess would be that it took some logs some time to reach an equilibrium for new data vs expiring data. So if anyone else is in a similar situation after migrating, from my experience I’d say not to worry, it will eventually stabilise.

Backup Growth rate

Product Support

codinghorror (Jeff Atwood) December 21, 2018, 3:18am 5

Very nice change, thank you for bringing this up @me3 and excellent improvement @gerhard!

Topic		Replies	Views
How big is your Discourse backup? Product Support	15	4064	April 25, 2016
Recommended backup process for very large forum? Product Support	6	1483	July 11, 2017
Is there any reason why daily backups would see a reduction in size? Product Support	25	1857	July 23, 2019
Backups are no longer happening Product Support	4	674	April 9, 2021
Forum is disappearing 😳 Backup size consistently decreasing over time Product Support	5	260	December 17, 2024

Backup Growth rate

Related topics