Backup on external drive fails for no obvious reason

We have set up automatic backups and have also moved /var/discourse/shared/standalone/uploads and /var/discourse/shared/standalone/backups according to this guide to an external drive, i.e. we have the following app.yml config:

volumes:
  - volume:
      host: /var/discourse/shared/standalone
      guest: /shared
  - volume:
      host: /var/discourse/shared/standalone/log/var-log
      guest: /var/log
  - volume:
      host: /path/to/external/uploads
      guest: /shared/uploads
  - volume:
      host: /path/to/external/backups
      guest: /shared/backups

The admin receives the message “Backup failed” with the following log, available for one month here.

This error message appears for no obvious reason, because the backup seems to be created. Output of ls -la /path/to/external/backups/default/:

total 322618
drwxrwxrwx 2 root root         0 Sep 27  2019 .
drwxrwxrwx 2 root root         0 Jun 27  2019 ..
-rwxrwxrwx 1 root root 327798879 Mai  1 04:21 xxx-yyyy-2021-05-01-020805-v20210420015635.tar.gz

Do you have any idea what might be going on here? Our Discourse version is 2.7.0.beta8 (656b0ae39e). Our backup settings are the following:

It might be a permission issue. Try making the directory write by all.

Which directory do you mean? The one that contains the backup (/path/to/external/backups/default) already is writable by everybody (see output of ls -la above).

It fails only after 44 seconds.

gzip creates a second file and removes the uncompressed file upon completion. This means that you need to have at least 327 MB of disk space available. Error messages might be obscured due to the fact that this is an external drive. Theory: you might be running out of disk space?

4 Likes

We’re mounting gocryptfs on a SAMBA share, and there should be plenty of space available. Output of df -h:

Filesystem                            Size  Used Avail Use% Mounted on
udev                                  1,9G     0  1,9G   0% /dev
tmpfs                                 385M  1,5M  384M   1% /run
/dev/mapper/hermes--vg-root            36G   21G   14G  60% /
tmpfs                                 1,9G     0  1,9G   0% /dev/shm
tmpfs                                 5,0M     0  5,0M   0% /run/lock
tmpfs                                 1,9G     0  1,9G   0% /sys/fs/cgroup
/dev/sda1                             704M  215M  439M  33% /boot
//xxx.file.core.windows.net/storage2  5,0T  491M  5,0T   1% /mnt/storage2/cipher
/mnt/storage2/cipher                  5,0T  491M  5,0T   1% /mnt/storage2/plain
tmpfs                                 385M     0  385M   0% /run/user/1000

Notably, a manual backup succeeded after I chose “Yes (do not include uploads)” – log here. So the failure might indeed be related to the amount of data compressed.

I then moved the backups to the internal drive again, and now manual backups also work with uploads included – log here.

It seems the failure is specific to our gocryptfs / SAMBA setup. Still, if anybody has ideas to investigate this further, I’m happy to hear them. Like, what exactly makes gzip say Operation not permitted.

2 Likes