Tengo dos buckets de S3: uno para copias de seguridad y otro para subidas. Tengo un usuario y una política para ambos con acceso completo a S3. Tanto las copias de seguridad como las subidas están configuradas para ir a S3. Extrañamente, puedo realizar una copia de seguridad y esta termina en S3, pero cualquier intento de subir imágenes o adjuntos genera errores de “Acceso denegado”.
He intentado reconstruir el sistema. He creado nuevos buckets y políticas. Nada de eso ha ayudado.
Esto también parece impedir la adición de imágenes de Gravatar. Sidekiq tiene varias tareas pendientes que indican:
Jobs::HandledExceptionWrapper: Wrapped Aws::S3::Errors::AccessDenied: Acceso denegado
Puedo agregar elementos manualmente en el sitio de AWS al bucket de subidas. También puedo hacer lo mismo usando la herramienta de CLI de AWS.
¿Alguna idea?
1 me gusta
Output from the logs:
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.27.0/lib/seahorse/client/plugins/raise_response_errors.rb:15:in
call' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/aws-sdk-s3-1.19.0/lib/aws-sdk-s3/plugins/sse_cpk.rb:22:in
call' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/aws-sdk-s3-1.19.0/lib/aws-sdk-s3/plugins/dualstack.rb:26:in
call' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/aws-sdk-s3-1.19.0/lib/aws-sdk-s3/plugins/accelerate.rb:35:in
call' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.27.0/lib/aws-sdk-core/plugins/jsonvalue_converter.rb:20:in
call' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.27.0/lib/aws-sdk-core/plugins/idempotency_token.rb:17:in
call' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.27.0/lib/aws-sdk-core/plugins/param_converter.rb:24:in
call' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.27.0/lib/aws-sdk-core/plugins/response_paging.rb:10:in
call' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.27.0/lib/seahorse/client/plugins/response_target.rb:23:in
call' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/aws-sdk-core-3.27.0/lib/seahorse/client/request.rb:70:in
send_request' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/aws-sdk-s3-1.19.0/lib/aws-sdk-s3/client.rb:5498:in
put_object' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/aws-sdk-s3-1.19.0/lib/aws-sdk-s3/file_uploader.rb:42:in
block in put_object' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/aws-sdk-s3-1.19.0/lib/aws-sdk-s3/file_uploader.rb:50:in
open_file' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/aws-sdk-s3-1.19.0/lib/aws-sdk-s3/file_uploader.rb:41:in
put_object' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/aws-sdk-s3-1.19.0/lib/aws-sdk-s3/file_uploader.rb:34:in
upload' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/aws-sdk-s3-1.19.0/lib/aws-sdk-s3/customizations/object.rb:308:in
upload_file' /var/www/discourse/lib/s3_helper.rb:28:in
upload' /var/www/discourse/lib/file_store/s3_store.rb:48:in
store_file' /var/www/discourse/lib/file_store/s3_store.rb:22:in
store_upload' /var/www/discourse/lib/upload_creator.rb:126:in
block (2 levels) in create_for' /var/www/discourse/lib/upload_creator.rb:125:in
open' /var/www/discourse/lib/upload_creator.rb:125:in
block in create_for' /var/www/discourse/lib/distributed_mutex.rb:34:in
synchronize' /var/www/discourse/lib/distributed_mutex.rb:5:in
synchronize' /var/www/discourse/lib/upload_creator.rb:36:in
create_for' /var/www/discourse/app/models/user_avatar.rb:41:in
block in update_gravatar!' /var/www/discourse/lib/distributed_mutex.rb:34:in
synchronize' /var/www/discourse/lib/distributed_mutex.rb:5:in
synchronize' /var/www/discourse/app/models/user_avatar.rb:14:in
update_gravatar!' /var/www/discourse/app/jobs/regular/update_gravatar.rb:12:in
execute' /var/www/discourse/app/jobs/base.rb:137:in
block (2 levels) in perform' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/rails_multisite-2.0.4/lib/rails_multisite/connection_management.rb:63:in
with_connection' /var/www/discourse/app/jobs/base.rb:127:in
block in perform' /var/www/discourse/app/jobs/base.rb:123:in
each' /var/www/discourse/app/jobs/base.rb:123:in
perform' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:187:in
execute_job' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:169:in
block (2 levels) in process' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/middleware/chain.rb:128:in
block in invoke' /var/www/discourse/lib/sidekiq/pausable.rb:81:in
call' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/middleware/chain.rb:130:in
block in invoke' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/middleware/chain.rb:133:in
invoke' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:168:in
block in process' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:139:in
block (6 levels) in dispatch' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/job_retry.rb:98:in
local' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:138:in
block (5 levels) in dispatch' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq.rb:36:in
block in <module:Sidekiq>' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:134:in
block (4 levels) in dispatch' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:199:in
stats' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:129:in
block (3 levels) in dispatch' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/job_logger.rb:8:in
call' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:128:in
block (2 levels) in dispatch' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/job_retry.rb:73:in
global' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:127:in
block in dispatch' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/logging.rb:48:in
with_context' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/logging.rb:42:in
with_job_hash_context' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:126:in
dispatch' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:167:in
process' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:85:in
process_one' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:73:in
run' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/util.rb:16:in
watchdog' /var/www/discourse/vendor/bundle/ruby/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/util.rb:25:in
block in safe_thread'
merefield
23 Diciembre, 2018 17:14
3
If you see Access Denied, it may well be that it is indeed, denied.
I had a simliar issue recently, you need to loosen access restriction on AWS just enough to resolve it.
2 Me gusta
The S3 user has all access to all buckets. (Uncertain why backups work but not upload.)
merefield
23 Diciembre, 2018 17:17
5
Mine was working fine for ages then broke, but modifying the access rights resolved it.
Somewhere there was a change.
Getting an Access Denied issue in error log … which seems to be causing Sidekiq to stay paused after backups!! (which is interesting on its own) - is my access policy on IAM not generous enough?:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:GetBucketLocation",
"s3:ListAllMyBuckets"
],
"Resource": "*"
},
{
"Effect": "Allow",
"…
1 me gusta
I can try, but this is my current policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "s3:*",
"Resource": "*"
}
]
}
So the user should be able to do anything.
2 Me gusta
Tried again with this, same issue - backups work but uploads do not:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"s3:PutAccountPublicAccessBlock",
"s3:GetAccountPublicAccessBlock",
"s3:ListAllMyBuckets",
"s3:ListBucket",
"s3:HeadBucket",
"s3:GetBucketLocation"
],
"Resource": "*"
},
{
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::gauntlet-forums-backup",
"arn:aws:s3:::gauntlet-forums-storage",
"arn:aws:s3:::gauntlet-forums-backup/*",
"arn:aws:s3:::gantlet-forums-storage/*"
]
},
{
"Sid": "VisualEditor2",
"Effect": "Allow",
"Action": "s3:*",
"Resource": "arn:aws:s3:::*/*"
}
]
}
Tried it with the policy in your post and in Setting up file and image uploads to S3 - still no dice.
Fixed typo. No change. Still not working.
gerhard
23 Diciembre, 2018 19:25
11
Is your uploads bucket empty? Delete it and let Discourse create the bucket. Also, make sure you follow the instructions in Setting up file and image uploads to S3 .
4 Me gusta
Huzzah! Deleting the bucket fixed it! Thanks for that!
gerhard
23 Diciembre, 2018 19:33
13
That’s so weird. I’ve seen that outcome a couple of times in the last few weeks. I’m not sure what’s going on with manually created buckets yet.
3 Me gusta
Strange to be sure… I swapped out that highly-permissive policy for one made from the two posts referenced and things seem good now too.
1 me gusta
deleted the bucket from where? I have not been able to upload images on my forum too…
pfaffman
23 Diciembre, 2018 19:55
16
I had trouble getting uploads to work with digitalocean spaces yesterday, also with an error about the bucket already existing. I ran out of time before I could fully diagnose its. 'tis the season.
2 Me gusta
I deleted the bucket from S3 and let Discourse create it itself.
3 Me gusta
I think that’s because S3 changed their default permission settings.
The settings when I create it manually
The settings when Discourse creates it
4 Me gusta
My settings are same with the discourse permission settings, yet images won’t upload.
Inspect elements shows something like "failed to load resource …
gerhard
4 Enero, 2019 13:40
20
jidetheblogger:
images won’t upload
Are you sure? You should see an “Access Denied” error if the upload fails. I thinks it’s a problem with a policy or CORS configuration. According to your screenshot the bucket permissions seem correct (both checkboxes under the “Manage public access control lists (ACLs)” are unchecked) and you “failed to load resource” looks like the browser simply isn’t allowed to load the image.
I’ve updated the instructions in Set up file and image uploads to S3 . Maybe it helps you find the difference in your S3 bucket configuration.
2 Me gusta
