Uploading local files to DigitalOcean Spaces (S3) with
rake uploads:migrate_to_s3 starts and migrates some files, then suddenly fails for one file with error
When this file is manually uploaded using the scipt mentioned in Extend S3 configuration for other S3 API compatible services and the script is rerun, it uploads more files but fails again for some other file.
Can someone from the team please have a look at this? I have nearly 50GB of files that i need to move from local storage to S3 in the near future (configuration and upload for new files work without issues).
I would be happy even if it’s just a workaround for now: I have looked at the code
to_s3_migration.rb and can identify the problematic files, but when I upload them to S3 using
Aws::S3::Resource from the script mentioned above, the file metadata is missing and the original name of the file is lost on download.
Search on the web returns lots of issues for SignatureDoesNotMatch: one of them mentions that version of the signing algorithm that the gem is using might be wrong, but this is beyond my current skill to check.
This is configuration for rake:
DISCOURSE_S3_ACCESS_KEY_ID="id" \ DISCOURSE_S3_SECRET_ACCESS_KEY="secret" \ DISCOURSE_S3_REGION="sfo2" \ DISCOURSE_S3_BUCKET="mybucket/uploads" \ DISCOURSE_S3_ENDPOINT="https://sfo2.digitaloceanspaces.com" \ DISCOURSE_S3_CDN_URL="https://mybucket.sfo2.cdn.digitaloceanspaces.com" \ rake uploads:migrate_to_s3 --trace