Cannot upload or backup to Amazon S3

bartv · October 12, 2015, 12:07pm

Hi,

we’ve been trying to set up backup to S3 in the 1.4 stable version, hosted on Communiteq (formerly DiscourseHosting). Our access keys are correct and we can upload through their CLI. However, Discourse throws the following errors (see dump below).

We found an issue that seems to be related:

github.com/aws/aws-sdk-ruby

"The request signature we calculated does not match the signature you provided." when uploading with `Aws::S3::Object#upload_file`

opened 02:49AM - 11 Sep 15 UTC

closed 09:14PM - 24 Sep 15 UTC

BanzaiMan

Travis CI has recently switched to use AWS SDK v2 for uploading artifacts to S3.… The up-to-date code is found at https://github.com/travis-ci/dpl/blob/master/lib/dpl/provider/s3.rb This works well for the most part, but a few isolated cases have been reported where the request signature does not match what's expected. 1. https://travis-ci.org/rebar/rebar3/jobs/79769044#L5696 (`Aws::S3::Errors::SignatureDoesNotMatch`) 2. https://travis-ci.org/octoblu/gateblu-forever/jobs/79543777#L409 (`Aws::S3::MultipartUploadError`) The code is quite simple: 1. Initialize `Aws::S3::Resource` with appropriate options (https://github.com/travis-ci/dpl/blob/75f10f342cc3e5d31c6cd45a2997966233d1deb6/lib/dpl/provider/s3.rb#L10) 2. And upload with `Aws::S3::Object#upload_file` (https://github.com/travis-ci/dpl/blob/75f10f342cc3e5d31c6cd45a2997966233d1deb6/lib/dpl/provider/s3.rb#L56) There is no custom signature generation from our code, so I am not sure what explains this exception. I appreciate any help. ref: https://github.com/travis-ci/travis-ci/issues/4776

Any ideas?

[2015-10-12 07:49:16] Executing the after_create_hook for the backup
[2015-10-12 07:49:18] EXCEPTION: multipart upload failed: The request signature we calculated does not match the signature you provided. Check your key and signing method.; The request signature we calculated does not match the signature you provided. Check your key and signing method.; The request signature we calculated does not match the signature you provided. Check your key and signing method.; The request signature we calculated does not match the signature you provided. Check your key and signing method.; The request signature we calculated does not match the signature you provided. Check your key and signing method.; The request signature we calculated does not match the signature you provided. Check your key and signing method.; The request signature we calculated does not match the signature you provided. Check your key and signing method.; The request signature we calculated does not match the signature you provided. Check your key and signing method.; The request signature we calculated does not match the signature you provided. Check your key and signing method.; The request signature we calculated does not match the signature you provided. Check your key and signing method.
[2015-10-12 07:49:18] /var/www/discourse/vendor/bundle/ruby/2.2.0/gems/aws-sdk-resources-2.0.45/lib/aws-sdk-resources/services/s3/multipart_file_uploader.rb:71:in `abort_upload'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/aws-sdk-resources-2.0.45/lib/aws-sdk-resources/services/s3/multipart_file_uploader.rb:60:in `upload_parts'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/aws-sdk-resources-2.0.45/lib/aws-sdk-resources/services/s3/multipart_file_uploader.rb:34:in `upload'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/aws-sdk-resources-2.0.45/lib/aws-sdk-resources/services/s3/file_uploader.rb:32:in `upload'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/aws-sdk-resources-2.0.45/lib/aws-sdk-resources/services/s3/object.rb:102:in `upload_file'
/var/www/discourse/lib/s3_helper.rb:16:in `upload'
/var/www/discourse/app/models/backup.rb:54:in `block in upload_to_s3'
/var/www/discourse/app/models/backup.rb:53:in `open'
/var/www/discourse/app/models/backup.rb:53:in `upload_to_s3'
/var/www/discourse/app/models/backup.rb:33:in `after_create_hook'
/var/www/discourse/lib/backup_restore/backuper.rb:267:in `after_create_hook'
/var/www/discourse/lib/backup_restore/backuper.rb:49:in `run'
/var/www/discourse/lib/backup_restore/backup_restore.rb:161:in `block in start!'
/var/www/discourse/lib/backup_restore/backup_restore.rb:158:in `fork'
/var/www/discourse/lib/backup_restore/backup_restore.rb:158:in `start!'
/var/www/discourse/lib/backup_restore/backup_restore.rb:13:in `backup!'
/var/www/discourse/app/controllers/admin/backups_controller.rb:31:in `create'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.10/lib/action_controller/metal/implicit_render.rb:4:in `send_action'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.10/lib/abstract_controller/base.rb:189:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.10/lib/action_controller/metal/rendering.rb:10:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.10/lib/abstract_controller/callbacks.rb:20:in `block in process_action'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.10/lib/active_support/callbacks.rb:113:in `call'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.10/lib/active_support/callbacks.rb:113:in `call'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.10/lib/active_support/callbacks.rb:552:in `block (2 levels) in compile'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.10/lib/active_support/callbacks.rb:502:in `call'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.10/lib/active_support/callbacks.rb:502:in `call'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.10/lib/active_support/callbacks.rb:86:in `run_callbacks'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.10/lib/abstract_controller/callbacks.rb:19:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.10/lib/action_controller/metal/rescue.rb:29:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.10/lib/action_controller/metal/instrumentation.rb:32:in `block in process_action'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.10/lib/active_support/notifications.rb:159:in `block in instrument'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.10/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.10/lib/active_support/notifications.rb:159:in `instrument'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.10/lib/action_controller/metal/instrumentation.rb:30:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.10/lib/action_controller/metal/params_wrapper.rb:250:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/activerecord-4.1.10/lib/active_record/railties/controller_runtime.rb:18:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.10/lib/abstract_controller/base.rb:136:in `process'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/actionview-4.1.10/lib/action_view/rendering.rb:30:in `process'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/rack-mini-profiler-0.9.6/lib/mini_profiler/profiling_methods.rb:77:in `block in profile_method'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.10/lib/action_controller/metal.rb:196:in `dispatch'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.10/lib/action_controller/metal/rack_delegation.rb:13:in `dispatch'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.10/lib/action_controller/metal.rb:232:in `block in action'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.10/lib/action_dispatch/routing/route_set.rb:82:in `call'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.10/lib/action_dispatch/routing/route_set.rb:82:in `dispatch'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.10/lib/action_dispatch/routing/route_set.rb:50:in `call'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.10/lib/action_dispatch/routing/mapper.rb:45:in `call'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.10/lib/action_dispatch/journey/router.rb:73:in `block in call'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.10/lib/action_dispatch/journey/router.rb:59:in `each'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.10/lib/action_dispatch/journey/router.rb:59:in `call'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.10/lib/action_dispatch/routing/route_set.rb:692:in `call'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/rack-protection-1.5.3/lib/rack/protection/frame_options.rb:31:in `call'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/omniauth-1.2.2/lib/omniauth/strategy.rb:186:in `call!'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/omniauth-1.2.2/lib/omniauth/strategy.rb:164:in `call'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/omniauth-1.2.2/lib/omniauth/strategy.rb:186:in `call!'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/omniauth-1.2.2/lib/omniauth/strategy.rb:164:in `call'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/omniauth-1.2.2/lib/omniauth/strategy.rb:186:in `call!'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/omniauth-1.2.2/lib/omniauth/strategy.rb:164:in `call'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/omniauth-1.2.2/lib/omniauth/strategy.rb:186:in `call!'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/omniauth-1.2.2/lib/omniauth/strategy.rb:164:in `call'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/omniauth-1.2.2/lib/omniauth/strategy.rb:186:in `call!'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/omniauth-1.2.2/lib/omniauth/strategy.rb:164:in `call'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/omniauth-1.2.2/lib/omniauth/builder.rb:59:in `call'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/rack-1.5.5/lib/rack/conditionalget.rb:35:in `call'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/rack-1.5.5/lib/rack/head.rb:11:in `call'
/var/www/discourse/lib/middleware/anonymous_cache.rb:124:in `call'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.10/lib/action_dispatch/middleware/params_parser.rb:27:in `call'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.10/lib/action_dispatch/middleware/flash.rb:254:in `call'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/message_bus-1.0.16/lib/message_bus/rack/middleware.rb:60:in `call'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/rack-1.5.5/lib/rack/session/abstract/id.rb:225:in `context'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/rack-1.5.5/lib/rack/session/abstract/id.rb:220:in `call'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.10/lib/action_dispatch/middleware/cookies.rb:562:in `call'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/activerecord-4.1.10/lib/active_record/query_cache.rb:36:in `call'
/var/www/discourse/vendor/gems/rails_multisite/lib/rails_multisite/connection_management.rb:175:in `call'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.10/lib/action_dispatch/middleware/callbacks.rb:29:in `block in call'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/activesupport-4.1.10/lib/active_support/callbacks.rb:82:in `run_callbacks'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.10/lib/action_dispatch/middleware/callbacks.rb:27:in `call'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.10/lib/action_dispatch/middleware/remote_ip.rb:76:in `call'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.10/lib/action_dispatch/middleware/debug_exceptions.rb:17:in `call'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.10/lib/action_dispatch/middleware/show_exceptions.rb:30:in `call'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/logster-1.0.0.3.pre/lib/logster/middleware/reporter.rb:31:in `call'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/railties-4.1.10/lib/rails/rack/logger.rb:38:in `call_app'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/railties-4.1.10/lib/rails/rack/logger.rb:22:in `call'
/var/www/discourse/config/initializers/quiet_logger.rb:10:in `call_with_quiet_assets'
/var/www/discourse/config/initializers/silence_logger.rb:26:in `call'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/actionpack-4.1.10/lib/action_dispatch/middleware/request_id.rb:21:in `call'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/rack-1.5.5/lib/rack/methodoverride.rb:21:in `call'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/rack-1.5.5/lib/rack/runtime.rb:17:in `call'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/rack-1.5.5/lib/rack/sendfile.rb:112:in `call'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/rack-mini-profiler-0.9.6/lib/mini_profiler/profiler.rb:170:in `call'
/var/www/discourse/lib/middleware/request_tracker.rb:73:in `call'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/railties-4.1.10/lib/rails/engine.rb:514:in `call'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/railties-4.1.10/lib/rails/application.rb:144:in `call'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/railties-4.1.10/lib/rails/railtie.rb:194:in `public_send'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/railties-4.1.10/lib/rails/railtie.rb:194:in `method_missing'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/rack-1.5.5/lib/rack/builder.rb:138:in `call'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/rack-1.5.5/lib/rack/urlmap.rb:65:in `block in call'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/rack-1.5.5/lib/rack/urlmap.rb:50:in `each'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/rack-1.5.5/lib/rack/urlmap.rb:50:in `call'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/thin-1.6.3/lib/thin/connection.rb:86:in `block in pre_process'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/thin-1.6.3/lib/thin/connection.rb:84:in `catch'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/thin-1.6.3/lib/thin/connection.rb:84:in `pre_process'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/thin-1.6.3/lib/thin/connection.rb:53:in `process'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/thin-1.6.3/lib/thin/connection.rb:39:in `receive_data'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/eventmachine-1.0.7/lib/eventmachine.rb:187:in `run_machine'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/eventmachine-1.0.7/lib/eventmachine.rb:187:in `run'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/thin-1.6.3/lib/thin/backends/base.rb:73:in `start'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/thin-1.6.3/lib/thin/server.rb:162:in `start'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/thin-1.6.3/lib/thin/controllers/controller.rb:87:in `start'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/thin-1.6.3/lib/thin/runner.rb:200:in `run_command'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/thin-1.6.3/lib/thin/runner.rb:156:in `run!'
/var/www/discourse/vendor/bundle/ruby/2.2.0/gems/thin-1.6.3/bin/thin:6:in `<top (required)>'
/var/www/discourse/vendor/bundle/ruby/2.2.0/bin/thin:23:in `load'
/var/www/discourse/vendor/bundle/ruby/2.2.0/bin/thin:23:in `<main>'
[2015-10-12 07:49:18] Notifying 'pap' of the end of the backup...
[2015-10-12 07:49:18] Removing old backups...
[2015-10-12 07:49:18] Cleaning stuff up...
[2015-10-12 07:49:18] Removing '.tar' leftovers...
[2015-10-12 07:49:18] Removing tmp '/var/www/discourse/tmp/backups/db4890/2015-10-12-074605' directory...
[2015-10-12 07:49:18] Unpausing sidekiq...
[2015-10-12 07:49:18] Marking backup as finished...
[2015-10-12 07:49:18] Finished!

sam · October 12, 2015, 10:03pm

tests-passed and beta are not affected by this bug. Our hosted customers have this fix.

I am torn on updating this gem on stable, it feels like a bit of a risky move.

bartv · October 14, 2015, 12:33pm

Sorry, I don’t entirely understand what you’re saying. It’s correct that this is currently broken in 1.4 stable?

codinghorror · October 14, 2015, 6:30pm

Requires a gem update, is how I read it. Not our bug, technically, but requires recompiling against a newer gem version that handles this particular task.

Topic		Replies	Views
I can't get a good backup - it fails after after_create_hook Bug	17	2728	September 21, 2016
Intended path to migrate S3 to local Installation uploads , s3	5	748	May 13, 2023
Issues backing up to s3 Installation backups , s3	5	935	January 9, 2024
Backup Failure after Upgrade to 2.3.0.beta9 Support	9	618	June 22, 2019
S3 migration issues- some images got pointed to S3, but with the wrong URL format Support	7	1468	December 29, 2018

Cannot upload or backup to Amazon S3

Related topics