The “(do not include files)” option when creating a backup manually refers to uploaded files and images, which are stored in the “uploads” folder. This folder contains images, avatars, user cards, and profile backgrounds, along with optimized thumbnails.
When creating a manual backup with this option, the resulting file is significantly smaller, as seen by dandv who experienced a 20x reduction in size. The difference in size is due to the exclusion of the “uploads” folder contents.
dandv wonders if the last backup was incremental or if linked images from remote URLs were downloaded, but zogstrip clarifies that the size difference is mainly due to the contents of the “uploads” folder.
The discussion also touches on the idea of file deduplication, which could potentially reduce storage needs, especially for forums with many reaction gifs. However, zogstrip advises against modifying files in the “uploads” folder.
It’s also noted that using a CDN for avatars, like avatars.discourse.org, can reduce storage needs on the local server. Additionally, sam mentions that user-uploaded avatars are stored in multiple sizes, resulting in 6 or more copies in backups.
It means uploaded files and images.
I’ve just triggered a manual backup with “Yes (do not include files)” and it’s ~20x as small as the automatic backup that occurred yesterday:
I’ve just migrated the forum from MyBB and there are no attachments (the migration doesn’t support them). What causes the difference in size? Is the last backup incremental? Are linked images from remote URLs downloaded in the first one?
Basically, everything that is inside the “uploads” folder.
This folder contains all the images (the ones included in posts, avatars, user cards and profile backgrounds) as well as optimized thumbnails.
If you have lots of duplicated images, there are tools that identify and replace duplicated files with symbolic links on Linux, in theory it should work well (never tested).
If you have a forum with lots of reaction gifs it would be a good case for file deduplication, I think.
Be carefull if you try this, backup everything and etc
I highly recommend that you do not mess with files that are in the “uploads” folders or you are in for a world of pain.
Hmm @sam is this another benefit of avatars.discourse.org the local server does not have to store a zillion copies of the generic avatars?
Yes, our letter avatars only live on our CDN. It does reduce storage.
That said, user uploaded avatars are “optimized” multiple times for various sizes so user uploaded avatars are effectively stored 6 times or so in different sizes in backups.
