Large Backup Upload to S3 Failing

(Nigel Fish) #1

Backups are working, but my upload to S3 is failing. It is definitely not an authorization issue, as it is only large backups. When I backup without files (about 42MB) the backup uploads to S3 as expected.

Including files is about 4GB, and I get the following error log:

[2016-08-24 17:19:14] Gzipping archive, this may take a while...
[2016-08-24 17:23:17] Executing the after_create_hook for the backup...
[2016-08-24 17:24:07] **EXCEPTION: multipart upload failed: The provided 'x-amz-content-sha256' header does not match what was computed.**
[2016-08-24 17:24:07] /var/www/discourse/vendor/bundle/ruby/2.3.0/gems/aws-sdk-resources-2.5.3/lib/aws-sdk-resources/services/s3/multipart_file_uploader.rb:81:in `abort_upload'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/aws-sdk-resources-2.5.3/lib/aws-sdk-resources/services/s3/multipart_file_uploader.rb:70:in `upload_parts'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/aws-sdk-resources-2.5.3/lib/aws-sdk-resources/services/s3/multipart_file_uploader.rb:44:in `upload'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/aws-sdk-resources-2.5.3/lib/aws-sdk-resources/services/s3/file_uploader.rb:32:in `upload'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/aws-sdk-resources-2.5.3/lib/aws-sdk-resources/services/s3/object.rb:251:in `upload_file'
/var/www/discourse/lib/s3_helper.rb:30:in `upload'

Any ideas what the issue might be? I’m on version v1.7.0.beta2 +137


(Jay Pfaffman) #2

Most web servers have a limit on how big a file can be uploaded. It would probably be easiest to upload your backup file to wherever your backup files are located via ssh/scp. If you did a standard install, it would be /var/discourse/shared/standalone/backups/.

Edit: Oops, but that wasn’t what you were asking.

(Nigel Fish) #3

I’m not uploading to the discourse server, it’s discourse uploading to S3.

(Alan Tan) #4

I’ll have a look at this next week.

(Alan Tan) #5

@dphrag Can you updating to latest and try again to see if this is still happening? I tested locally and couldn’t get it to fail.

(Nigel Fish) #6

I updated to version: v1.7.0.beta3 +36 which I assume is as latest as I can get.

Still failed with the error message:

[2016-08-30 16:17:02] EXCEPTION: multipart upload failed: The provided 'x-amz-content-sha256' header does not match what was computed.
[2016-08-30 16:17:02] /var/www/discourse/vendor/bundle/ruby/2.3.0/gems/aws-sdk-resources-2.5.3/lib/aws-sdk-resources/services/s3/multipart_file_uploader.rb:81:in `abort_upload'

And the smaller non-files backup still works.

(Nigel Fish) #7

Still failing on large backups for me on v1.7.0.beta4 +59.

Any ideas at all?

(Alan Tan) #8

Can you try changing the region to see if it helps? Let me know what region you’re using and I’ll try to repro again. I’m using US-WEST (N. California)

(Nigel Fish) #9

Am using US-WEST (N. California) as well.