Cron task to sync local backups to DigitalOcean Spaces

It doesn’t seem I can get backups to DigitalOcean Spaces to work. However the server has the bucket mounted over s3fs with the same credentials. I only need the backups to be stored on Spaces, not uploads etc. Logs say EXCEPTION: Aws::S3::Errors::Forbidden Production log says Failed to list backups from S3: Aws::S3::Errors::SignatureDoesNotMatch

Is there a way to cron task a move of the local backup to the bucket via a bash script? Thanks.

For anyone else running into this and wants to keep rolling, you can issue this command at the bash terminal. The user needs to be part of the docker group
docker cp app:"/var/www/discourse/public/backups/default" /whatever-host-path/

2 Likes

Do you mean discourse backups? If so, why is this in WordPress?

If you want them to go to spaces why not just configure S3 backups?

2 Likes

The main site is WordPress. Discourse is in subdirectory.
Main site: main.com
Discourse: main.com/forums
But configuring S3 credentials for DigitalOcean Spaces gives me the above errors. I have a deadline to meet so just got around this by the commands I mentioned earlier. But it would be handy to see someone’s working settings and / or troubleshooting steps to make this work.

Above you mention s3fs. You don’t need that. The errors above are coming from s3fs, which is not something that Discourse supports. You want to follow something like Setting up file and image uploads to S3 or Using Object Storage for Uploads (S3 & Clones), but configure just the backup bucket.

1 Like

To make it more explicit, the “host” has the DigitalOcean Spaces bucket mounted, not the container. The two issues are unrelated. Backups throw the errors if Discouse is configured for S3. Backups work if I manually copy the local backups to the host machine and then move them to DigitalOcean Spaces.

If the Forbidden logs are from Discourse and not s3fs, trying to save to the Spaces bucket then it’s something like you have the wrong credentials or wrong bucket name.

Ah!

   Failed to list backups from S3:

Ah. I think that’s the problem. It’s trying to get a bucket list and can’t. I’m not quite sure how to fix it, but that’s the problem that you want to solve, so you might edit the subject. If you want to fix your S3 backups, you can get help here; if you want to configure s3fs you’ll llkely need to seek help elsewhere.

Try this ENV variable in your app.yml

  FORCE_S3_UPLOADS: 1

IS your endpoint correct? Have a look at the Digital Ocean section in Using Object Storage for Uploads (S3 & Clones).

1 Like

Finally this worked for DigitalOcean Spaces a) Uploading backups to Object Storage b) Not uploading file uploads to Object Storage. Kindly mark this as resolved. Thanks everyone!

2 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.