Reduce local disk space needs by not (redundantly) gzipping backups

I am with it. It is a modern common standard.

Compressing a file twice just takes up time and resources.

It would be good if someone could do this unglamorous work though! It’s not shiny but it would make life better for self-hosters.

1 Like

Add me to the list of people who would appreciate this, since the doubled backup size causes backups to fail every so often for me, forcing a manual cleanup just to keep backups running.

1 Like

Maybe it could be pr-welcome ?

It’s a small fix, but breaking backups is also very, very bad.

3 Likes

That might be the answer - how to do that? It’s not a tag that’s available in this category

Only someone like Sam can decide what he will accept as a PR. Of course not every pr is one that was explicitly requested.

1 Like

We are tinkering with some related stuff at the moment. Not sure yet if it will touch this directly, but we’ll share more when we know a bit more.

6 Likes

This change should remove the ‘2x disk space’ requirement for backups. It also spreads the gz CPU cost over the whole of the backup operation, rather than it being a bottleneck at the end:

We’re also investigating the idea of dropping the wrapper .gz compression entirely. But that would be a change to the format, so we need to be especially cautious with that kind of change.

People who use S3 for uploads may also be interested in this performance improvement which landed yesterday.

9 Likes

Thanks… I hope we see extensive testing of backup restores, as this is quite a new way to do it.

Of course!

If any community members are keen to help with testing, you can use that branch name in the version: field of your site’s YAML, redeploy, and then try it out.

(I recommend doing that with a staging site rather than a production site)

3 Likes

This has been merged for some time, and seems to be working well. If anyone runs into any problems, please feel free to open a fresh topic.

4 Likes