Issues backing up to s3

Product Support Self-hosting
,

AI-generated summary

The user jerdog encountered an “Access Denied” error while trying to back up to an S3 bucket, despite entering correct credentials. sam suggested the issue might be related to insufficient IAM permissions, a theory Falco supported. jerdog confirmed the bucket was private but made it public temporarily, which did not resolve the issue. Ultimately, jerdog resolved the problem by ensuring proper IAM setup, matching AWS regions, and confirming that the instances were on the same AWS account. The user credited sam for guiding them toward the solution.

1

So having some issues in backing up to S3 - I have put in my s3 credentials (it’s a private bucket) and each time I backup I get the following error:

[2018-01-29 21:41:45] Executing the after_create_hook for the backup...
[2018-01-29 21:41:45] EXCEPTION: Access Denied
[2018-01-29 21:41:45] /var/www/discourse/vendor/bundle/ruby/2.4.0/gems/aws-sdk-core-3.6.0/lib/seahorse/client/plugins/raise_response_errors.rb:15:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/aws-sdk-s3-1.4.0/lib/aws-sdk-s3/plugins/sse_cpk.rb:22:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/aws-sdk-s3-1.4.0/lib/aws-sdk-s3/plugins/dualstack.rb:26:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/aws-sdk-s3-1.4.0/lib/aws-sdk-s3/plugins/accelerate.rb:35:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/aws-sdk-core-3.6.0/lib/aws-sdk-core/plugins/jsonvalue_converter.rb:20:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/aws-sdk-core-3.6.0/lib/aws-sdk-core/plugins/idempotency_token.rb:17:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/aws-sdk-core-3.6.0/lib/aws-sdk-core/plugins/param_converter.rb:24:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/aws-sdk-core-3.6.0/lib/aws-sdk-core/plugins/response_paging.rb:10:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/aws-sdk-core-3.6.0/lib/seahorse/client/plugins/response_target.rb:23:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/aws-sdk-core-3.6.0/lib/seahorse/client/request.rb:70:in `send_request'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/aws-sdk-s3-1.4.0/lib/aws-sdk-s3/client.rb:849:in `create_multipart_upload'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/aws-sdk-s3-1.4.0/lib/aws-sdk-s3/multipart_file_uploader.rb:52:in `initiate_upload'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/aws-sdk-s3-1.4.0/lib/aws-sdk-s3/multipart_file_uploader.rb:43:in `upload'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/aws-sdk-s3-1.4.0/lib/aws-sdk-s3/file_uploader.rb:32:in `upload'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/aws-sdk-s3-1.4.0/lib/aws-sdk-s3/customizations/object.rb:252:in `upload_file'
/var/www/discourse/lib/s3_helper.rb:28:in `upload'
/var/www/discourse/app/models/backup.rb:55:in `block in upload_to_s3'
/var/www/discourse/app/models/backup.rb:54:in `open'
/var/www/discourse/app/models/backup.rb:54:in `upload_to_s3'
/var/www/discourse/app/models/backup.rb:33:in `after_create_hook'
/var/www/discourse/lib/backup_restore/backuper.rb:250:in `after_create_hook'
/var/www/discourse/lib/backup_restore/backuper.rb:44:in `run'
/var/www/discourse/lib/backup_restore/backup_restore.rb:167:in `block in start!'
/var/www/discourse/lib/backup_restore/backup_restore.rb:164:in `fork'
/var/www/discourse/lib/backup_restore/backup_restore.rb:164:in `start!'
/var/www/discourse/lib/backup_restore/backup_restore.rb:18:in `backup!'
/var/www/discourse/app/controllers/admin/backups_controller.rb:32:in `create'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_controller/metal/basic_implicit_render.rb:4:in `send_action'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/abstract_controller/base.rb:186:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_controller/metal/rendering.rb:30:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/abstract_controller/callbacks.rb:20:in `block in process_action'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.4/lib/active_support/callbacks.rb:131:in `run_callbacks'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/abstract_controller/callbacks.rb:19:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_controller/metal/rescue.rb:20:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_controller/metal/instrumentation.rb:32:in `block in process_action'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.4/lib/active_support/notifications.rb:166:in `block in instrument'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.4/lib/active_support/notifications/instrumenter.rb:21:in `instrument'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.4/lib/active_support/notifications.rb:166:in `instrument'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_controller/metal/instrumentation.rb:30:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_controller/metal/params_wrapper.rb:252:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.4/lib/active_record/railties/controller_runtime.rb:22:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/abstract_controller/base.rb:124:in `process'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionview-5.1.4/lib/action_view/rendering.rb:30:in `process'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-mini-profiler-0.10.7/lib/mini_profiler/profiling_methods.rb:76:in `block in profile_method'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_controller/metal.rb:189:in `dispatch'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_controller/metal.rb:253:in `dispatch'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/routing/route_set.rb:49:in `dispatch'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/routing/route_set.rb:31:in `serve'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/routing/mapper.rb:16:in `block in <class:Constraints>'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/routing/mapper.rb:46:in `serve'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/journey/router.rb:50:in `block in serve'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/journey/router.rb:33:in `each'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/journey/router.rb:33:in `serve'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/routing/route_set.rb:834:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-protection-2.0.0/lib/rack/protection/frame_options.rb:31:in `call'
/var/www/discourse/lib/middleware/omniauth_bypass_middleware.rb:24:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-2.0.3/lib/rack/conditional_get.rb:38:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-2.0.3/lib/rack/head.rb:12:in `call'
/var/www/discourse/lib/middleware/anonymous_cache.rb:149:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-2.0.3/lib/rack/session/abstract/id.rb:232:in `context'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-2.0.3/lib/rack/session/abstract/id.rb:226:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/middleware/cookies.rb:613:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/middleware/callbacks.rb:26:in `block in call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.4/lib/active_support/callbacks.rb:97:in `run_callbacks'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/middleware/callbacks.rb:24:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/middleware/debug_exceptions.rb:59:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/middleware/show_exceptions.rb:31:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/logster-1.2.9/lib/logster/middleware/reporter.rb:31:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/railties-5.1.4/lib/rails/rack/logger.rb:36:in `call_app'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/railties-5.1.4/lib/rails/rack/logger.rb:26:in `call'
/var/www/discourse/config/initializers/100-quiet_logger.rb:16:in `call'
/var/www/discourse/config/initializers/100-silence_logger.rb:29:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/middleware/remote_ip.rb:79:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/middleware/request_id.rb:25:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-2.0.3/lib/rack/method_override.rb:22:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/middleware/executor.rb:12:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-2.0.3/lib/rack/sendfile.rb:111:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-mini-profiler-0.10.7/lib/mini_profiler/profiler.rb:171:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/message_bus-2.1.2/lib/message_bus/rack/middleware.rb:63:in `call'
/var/www/discourse/lib/middleware/request_tracker.rb:138:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/railties-5.1.4/lib/rails/engine.rb:522:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/railties-5.1.4/lib/rails/railtie.rb:185:in `public_send'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/railties-5.1.4/lib/rails/railtie.rb:185:in `method_missing'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-2.0.3/lib/rack/urlmap.rb:68:in `block in call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-2.0.3/lib/rack/urlmap.rb:53:in `each'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-2.0.3/lib/rack/urlmap.rb:53:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/unicorn-5.4.0/lib/unicorn/http_server.rb:606:in `process_client'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/unicorn-5.4.0/lib/unicorn/http_server.rb:701:in `worker_loop'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/unicorn-5.4.0/lib/unicorn/http_server.rb:549:in `spawn_missing_workers'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/unicorn-5.4.0/lib/unicorn/http_server.rb:142:in `start'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/unicorn-5.4.0/bin/unicorn:126:in `<top (required)>'
/var/www/discourse/vendor/bundle/ruby/2.4.0/bin/unicorn:23:in `load'
/var/www/discourse/vendor/bundle/ruby/2.4.0/bin/unicorn:23:in `<main>'
Notifying '' of the end of the backup... 
[2018-01-29 21:41:51] Removing old backups...

It is saying “access denied” but seeing as I am giving it the correct credentials I am at a loss.

2

Perhaps the credentials are correct but the account does not have the permissions required to create new buckets or something like that.

3

What @sam said, IAM permission is quite extensive, and getting it 100% usually involves some back and forth.

4

So the settings I have for backup are:

image
image631×305 30.4 KB

and the settings for files are:

image
image636×529 56.5 KB

After first having issues I turned off IAM profile. I have it as a private bucket.

I then made it fully public in the Permissions tab (ignoring AWS warnings telling me not to do that) and still getting the above error.

5

And I finally got this setup and working correctly. Ultimately it had to do with getting IAM setup correctly, being in the same region, and then making sure the instances were on the same account in AWS. Once that was done things started working just fine.

For reference, this guide was extremely helpful:

http://crunchify.com/how-to-setup-amazon-s3-correct-way-for-your-discourse-forum-setup-image-upload-and-backup-both-to-aws-s3/

I marked the last reply by @sam as the solution as it pointed me down at least the right rabbit hole.