S3 存储桶的策略和权限问题？

Shane_Liebling · 2018 年12 月 23 日 16:59

我有两个 S3 存储桶，一个用于备份，一个用于上传。我为用户/策略配置了针对这两个存储桶的完整 S3 访问权限。我已将备份和上传都设置为发送到 S3。奇怪的是，我可以成功创建备份并将其保存到 S3，但任何图像上传或附件操作都会返回“访问被拒绝”（Access Denied）错误。

我已经尝试过重建系统，也创建过新的存储桶和策略，但都没有帮助。

这似乎还导致无法添加 Gravatar 头像。Sidekiq 中有一批待处理的任务报错：

Jobs::HandledExceptionWrapper: Wrapped Aws::S3::Errors::AccessDenied: Access Denied

我可以在 AWS 控制台上手动向“uploads”存储桶添加内容，使用 AWS CLI 工具也可以完成同样的操作。

有什么建议吗？

Shane_Liebling · 2018 年12 月 23 日 17:07

Output from the logs:

/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.27.0/lib/seahorse/client/plugins/raise_response_errors.rb:15:in 
call' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/aws-sdk-s3-1.19.0/lib/aws-sdk-s3/plugins/sse_cpk.rb:22:in 
call' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/aws-sdk-s3-1.19.0/lib/aws-sdk-s3/plugins/dualstack.rb:26:in 
call' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/aws-sdk-s3-1.19.0/lib/aws-sdk-s3/plugins/accelerate.rb:35:in 
call' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.27.0/lib/aws-sdk-core/plugins/jsonvalue_converter.rb:20:in 
call' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.27.0/lib/aws-sdk-core/plugins/idempotency_token.rb:17:in 
call' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.27.0/lib/aws-sdk-core/plugins/param_converter.rb:24:in 
call' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.27.0/lib/aws-sdk-core/plugins/response_paging.rb:10:in 
call' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.27.0/lib/seahorse/client/plugins/response_target.rb:23:in 
call' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.27.0/lib/seahorse/client/request.rb:70:in 
send_request' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/aws-sdk-s3-1.19.0/lib/aws-sdk-s3/client.rb:5498:in 
put_object' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/aws-sdk-s3-1.19.0/lib/aws-sdk-s3/file_uploader.rb:42:in 
block in put_object' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/aws-sdk-s3-1.19.0/lib/aws-sdk-s3/file_uploader.rb:50:in 
open_file' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/aws-sdk-s3-1.19.0/lib/aws-sdk-s3/file_uploader.rb:41:in 
put_object' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/aws-sdk-s3-1.19.0/lib/aws-sdk-s3/file_uploader.rb:34:in 
upload' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/aws-sdk-s3-1.19.0/lib/aws-sdk-s3/customizations/object.rb:308:in 
upload_file' /var/www/discourse/lib/s3_helper.rb:28:in 
upload' /var/www/discourse/lib/file_store/s3_store.rb:48:in 
store_file' /var/www/discourse/lib/file_store/s3_store.rb:22:in 
store_upload' /var/www/discourse/lib/upload_creator.rb:126:in 
block (2 levels) in create_for' /var/www/discourse/lib/upload_creator.rb:125:in 
open' /var/www/discourse/lib/upload_creator.rb:125:in 
block in create_for' /var/www/discourse/lib/distributed_mutex.rb:34:in 
synchronize' /var/www/discourse/lib/distributed_mutex.rb:5:in 
synchronize' /var/www/discourse/lib/upload_creator.rb:36:in 
create_for' /var/www/discourse/app/models/user_avatar.rb:41:in 
block in update_gravatar!' /var/www/discourse/lib/distributed_mutex.rb:34:in 
synchronize' /var/www/discourse/lib/distributed_mutex.rb:5:in 
synchronize' /var/www/discourse/app/models/user_avatar.rb:14:in 
update_gravatar!' /var/www/discourse/app/jobs/regular/update_gravatar.rb:12:in 
execute' /var/www/discourse/app/jobs/base.rb:137:in 
block (2 levels) in perform' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/rails_multisite-2.0.4/lib/rails_multisite/connection_management.rb:63:in 
with_connection' /var/www/discourse/app/jobs/base.rb:127:in 
block in perform' /var/www/discourse/app/jobs/base.rb:123:in 
each' /var/www/discourse/app/jobs/base.rb:123:in 
perform' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:187:in 
execute_job' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:169:in 
block (2 levels) in process' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/middleware/chain.rb:128:in 
block in invoke' /var/www/discourse/lib/sidekiq/pausable.rb:81:in 
call' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/middleware/chain.rb:130:in 
block in invoke' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/middleware/chain.rb:133:in 
invoke' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:168:in 
block in process' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:139:in 
block (6 levels) in dispatch' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/job_retry.rb:98:in 
local' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:138:in 
block (5 levels) in dispatch' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq.rb:36:in 
block in &lt;module:Sidekiq&gt;' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:134:in 
block (4 levels) in dispatch' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:199:in 
stats' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:129:in 
block (3 levels) in dispatch' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/job_logger.rb:8:in 
call' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:128:in 
block (2 levels) in dispatch' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/job_retry.rb:73:in 
global' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:127:in 
block in dispatch' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/logging.rb:48:in 
with_context' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/logging.rb:42:in 
with_job_hash_context' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:126:in 
dispatch' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:167:in 
process' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:85:in 
process_one' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:73:in 
run' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/util.rb:16:in 
watchdog' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/util.rb:25:in 
block in safe_thread'

merefield · 2018 年12 月 23 日 17:14

If you see Access Denied, it may well be that it is indeed, denied.

I had a simliar issue recently, you need to loosen access restriction on AWS just enough to resolve it.

Shane_Liebling · 2018 年12 月 23 日 17:15

The S3 user has all access to all buckets. (Uncertain why backups work but not upload.)

merefield · 2018 年12 月 23 日 17:17

Mine was working fine for ages then broke, but modifying the access rights resolved it.

Somewhere there was a change.

Shane_Liebling · 2018 年12 月 23 日 17:36

I can try, but this is my current policy:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "s3:*",
      "Resource": "*"
    }
  ]
}

So the user should be able to do anything.

Shane_Liebling · 2018 年12 月 23 日 18:29

Tried again with this, same issue - backups work but uploads do not:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "s3:PutAccountPublicAccessBlock",
                "s3:GetAccountPublicAccessBlock",
                "s3:ListAllMyBuckets",
                "s3:ListBucket",
                "s3:HeadBucket",
                "s3:GetBucketLocation"
            ],
            "Resource": "*"
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::gauntlet-forums-backup",
                "arn:aws:s3:::gauntlet-forums-storage",
                "arn:aws:s3:::gauntlet-forums-backup/*",
                "arn:aws:s3:::gantlet-forums-storage/*"
            ]
        },
        {
            "Sid": "VisualEditor2",
            "Effect": "Allow",
            "Action": "s3:*",
            "Resource": "arn:aws:s3:::*/*"
        }
    ]
}

Shane_Liebling · 2018 年12 月 23 日 19:13

Tried it with the policy in your post and in Setting up file and image uploads to S3 - still no dice.

gerhard · 2018 年12 月 23 日 19:18

Looks like a typo to me.

Shane_Liebling · 2018 年12 月 23 日 19:23

Fixed typo. No change. Still not working.

gerhard · 2018 年12 月 23 日 19:25

Is your uploads bucket empty? Delete it and let Discourse create the bucket. Also, make sure you follow the instructions in Setting up file and image uploads to S3.

Shane_Liebling · 2018 年12 月 23 日 19:28

Huzzah! Deleting the bucket fixed it! Thanks for that!

gerhard · 2018 年12 月 23 日 19:33

That’s so weird. I’ve seen that outcome a couple of times in the last few weeks. I’m not sure what’s going on with manually created buckets yet.

Shane_Liebling · 2018 年12 月 23 日 19:34

Strange to be sure… I swapped out that highly-permissive policy for one made from the two posts referenced and things seem good now too.

jidetheblogger · 2018 年12 月 23 日 19:40

deleted the bucket from where? I have not been able to upload images on my forum too…

pfaffman · 2018 年12 月 23 日 19:55

I had trouble getting uploads to work with digitalocean spaces yesterday, also with an error about the bucket already existing. I ran out of time before I could fully diagnose its. 'tis the season.

Shane_Liebling · 2018 年12 月 23 日 23:16

I deleted the bucket from S3 and let Discourse create it itself.

zogstrip · 2018 年12 月 24 日 10:48

I think that’s because S3 changed their default permission settings.

The settings when I create it manually

The settings when Discourse creates it

jidetheblogger · 2018 年12 月 24 日 12:57

My settings are same with the discourse permission settings, yet images won’t upload.

Inspect elements shows something like "failed to load resource…

gerhard · 2019 年1 月 4 日 13:40

Are you sure? You should see an “Access Denied” error if the upload fails. I thinks it’s a problem with a policy or CORS configuration. According to your screenshot the bucket permissions seem correct (both checkboxes under the “Manage public access control lists (ACLs)” are unchecked) and you “failed to load resource” looks like the browser simply isn’t allowed to load the image.

I’ve updated the instructions in Set up file and image uploads to S3. Maybe it helps you find the difference in your S3 bucket configuration.

话题		回复	浏览量
S3 image upload access denied while backups upload working fine Self-hosting s3	22	2725	2020 年3 月 23 日
Set up file and image uploads to S3 Self-Hosting configuring , how-to	35	120053	2025 年10 月 25 日
S3 Backup ... suspect access issue Support	4	1887	2018 年10 月 16 日
Issues backing up to s3 Self-hosting backups , s3	5	994	2024 年1 月 9 日
The bucket does not allow ACLs Support s3	5	2708	2022 年12 月 3 日

S3 存储桶的策略和权限问题？

相关话题