Backup Growth rate

I notice on a new (<1 mth) install of discourse the backup size increasing by a steady 0.2Mb / day.

Any thoughts on what grows day by day if new posts are not added?
Backups are to S3, but I’m wondering if it’s backing up backups.
I haven’t torn apart the gz’s yet.

1 Like

We’re not backing up the backups.

Is your forum active? Do you users upload images?

1 Like

Just starting. No images yet.
It’s the DB growing by about 2MB / day which is about 0.2mb compressed.
Not a big problem, but interesting if the growth is continuous.

1 Like

I think the backup logs are causing this. The system user creates a PM for every automatic backup, but it doesn’t invite the admins unless there was an error. I think it’s safe to create the PM only when there actually was an error or if it was a manual backup. I’m going to change that.

https://github.com/discourse/discourse/commit/0bc1fa8aa4ea5393193cc5c44adbf7eee24cd395

9 Likes

Very nice change, thank you for bringing this up @me3 and excellent improvement @gerhard!

5 Likes

But one PM hardly generates 2MB of data each day. If the forum is being read by users, could it be the table holding the user activity logs?

2 Likes

I can upload the gz backups files somewhere if you like / want a look at them.
They aren’t particularly large yet.
It’s not a complaint. it’s a “that’s a strange steady growth rate” for a new nearly empty site.
I can’t attach files here or PM I don’t think.

3 Likes

In my tests the backup size increased roughly 75kB because of the backup logs. There are a bunch of log tables that could increase in size even though there are no new posts. Feel free to look at a diff of the database dump if you are curious, but I don’t think there’s anything else we can improve here.

1 Like

After migrating to Discourse, I was concerned that our backups seemed to be increasing far more than expected. I was seeing backups increase by up to 30MB/week (after accounting for uploads) for only ~150 posts/week. But after about three months since migrating, the backups have stabilised to just 1-3MB/week. My guess would be that it took some logs some time to reach an equilibrium for new data vs expiring data. So if anyone else is in a similar situation after migrating, from my experience I’d say not to worry, it will eventually stabilise.

6 Likes