Reduce local disk space needs by not (redundantly) gzipping backups

Feature
1

The backup process creates a tar file and then applies gzip to it. There are two types of things in the tar file: an already gzipped sql dump and the contents of uploads (if requested.) In my case every upload file is already compressed: gz, gzip, gif, jpeg, png, zip. So the final gzipping gains only 1% of size.

I believe it would be better to demand less free space.

A previous topic from 2016 mentions disabling backup compression, but it looks like the sql dump was at that time not compressed, which shifted the tradeoffs.

Add option to disable backup compression

8 Likes
2

I’m already working on a new backup format that removes the double compression. My hope is that it will be ready within a month or two.

12 Likes
3

Sounds great @gerhard!

2 Likes
4

Any update on this? Thanks

1 Like
5

Not to bug you too much, but how is this progressing?

6

Development of that feature is currently paused and it isn’t on our current roadmap. I hope we will get to it in 2024.

4 Likes
7

If I wrote a patch to accept a 0 in the compression rate to disable gzip, would that be something that you would accept?

1 Like
8

(I’m guessing that you’d save CPU time that way, but not space, because the gzipped tar file would still be created.)

9

I’m aiming to save cpu time. Actually, I was thinking of using the 0 as a flag that would change the code path so that it doesn’t gzip (sadly, zero is not a valid compression level supported across all gzip versions, afaik).

10

Hmm that wouldn’t help me at all! (Likewise others who’ve had the same problem with limited disk space.)

If tar were being used, it could be used with z or j options. If a subshell were being used, the output of tar could be piped into gzip. But I think in fact some higher level ruby functions may be in use.

1 Like
11

cough

2 Likes
12

Maybe it shouldn’t be too difficult… I appreciate that making changes to backup and restore must be made with great care, but I think just inlining the compression would save a lot of space requirement without any compatibility question.

From tar --help

-a, --auto-compress use archive suffix to determine the compression
-z, --gzip, --gunzip, --ungzip filter the archive through gzip

1 Like
13

Does -z actually do an in-place compression? I always assumed that it just ran gzip after the tar file is completed.

14

Unwisely, in this case! The bytes which represent the uncompressed tar file never hit the disk.

1 Like