Users markersocial and Freez report failures when uploading Discourse backups to Scaleway S3 using multipart uploads, with errors like “The specified multipart upload does not exist” or “Your request was not formed properly.” Falco confirms multipart uploads are unavoidable for large files (>15MB) and suggests contacting Scaleway support. Freez reaches out to Scaleway, who request HTTP-level logs to debug — Falco notes Discourse’s open-source nature allows Scaleway to inspect the exact SDK calls. Later, Freez reports intermittent success (7/10 backups working), then full success Freez, while markersocial still experiences failures, suggesting Scaleway’s fixes may not be fully rolled out. Falco notes the advantage of S3 compatibility: fixes on Scaleway’s side require no Discourse changes.
Running on 2.6.1 stable with a s3 clone (Scaleway). Here is the log:
[2021-01-30 09:52:59] Uploading archive...
[2021-01-30 09:55:36] EXCEPTION: failed to abort multipart upload: The specified multipart upload does not exist. The upload ID might be invalid, or the multipart upload might have been aborted or completed.
[2021-01-30 09:55:36] /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/aws-sdk-s3-1.83.2/lib/aws-sdk-s3/multipart_file_uploader.rb:94:in `rescue in abort_upload'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/aws-sdk-s3-1.83.2/lib/aws-sdk-s3/multipart_file_uploader.rb:82:in `abort_upload'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/aws-sdk-s3-1.83.2/lib/aws-sdk-s3/multipart_file_uploader.rb:78:in `upload_parts'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/aws-sdk-s3-1.83.2/lib/aws-sdk-s3/multipart_file_uploader.rb:51:in `upload'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/aws-sdk-s3-1.83.2/lib/aws-sdk-s3/file_uploader.rb:38:in `upload'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/aws-sdk-s3-1.83.2/lib/aws-sdk-s3/customizations/object.rb:342:in `upload_file'
/var/www/discourse/lib/backup_restore/s3_backup_store.rb:39:in `upload_file'
/var/www/discourse/lib/backup_restore/backuper.rb:301:in `upload_archive'
/var/www/discourse/lib/backup_restore/backuper.rb:41:in `run'
/var/www/discourse/script/spawn_backup_restore.rb:9:in `backup'
/var/www/discourse/script/spawn_backup_restore.rb:34:in `block in <main>'
/var/www/discourse/script/spawn_backup_restore.rb:4:in `fork'
/var/www/discourse/script/spawn_backup_restore.rb:4:in `<main>'
[2021-01-30 09:55:36] Deleting old backups...
[2021-01-30 09:55:36] Cleaning stuff up...
[2021-01-30 09:55:36] Removing archive from local storage...
[2021-01-30 09:55:36] Removing '.tar' leftovers...
[2021-01-30 09:55:36] Marking backup as finished...
[2021-01-30 09:55:36] Refreshing disk stats...
I have the same configuration on a couple of other servers and backups are working fine (though the backup sizes are significantly smaller, so multipart upload likely isn’t a factor). It looks like they support multipart uploads also: Managing multipart uploads | Scaleway Documentation - so not sure what the problem is.
EXCEPTION: multipart upload failed:
Your request was not formed properly;
Your request was not formed properly
Then, every one or two days:
EXCEPTION: failed to abort multipart upload:
The specified multipart upload does not exist.
The upload ID might be invalid, or
the multipart upload might have been aborted or completed.
@Falco what can we do to understand what is going on?
Thank you