Then it should be fixed.
I am getting
Job exception: Access Denied in the Discourse logs when attempting to upload a file to S3. Some details:
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/aws-sdk-core-2.5.3/lib/seahorse/client/plugins/raise_response_errors.rb:15:in `call' /var/www/discourse/vendor/bundle/ruby/2.3.0/gems/aws-sdk-core-2.5.3/lib/aws-sdk-core/plugins/s3_sse_cpk.rb:19:in `call' /var/www/discourse/vendor/bundle/ruby/2.3.0/gems/aws-sdk-core-2.5.3/lib/aws-sdk-core/plugins/s3_dualstack.rb:23:in `call' /var/www/discourse/vendor/bundle/ruby/2.3.0/gems/aws-sdk-core-2.5.3/lib/aws-sdk-core/plugins/s3_accelerate.rb:33:in `call' /var/www/discourse/vendor/bundle/ruby/2.3.0/gems/aws-sdk-core-2.5.3/lib/aws-sdk-core/plugins/param_converter.rb:20:in `call' /var/www/discourse/vendor/bundle/ruby/2.3.0/gems/aws-sdk-core-2.5.3/lib/seahorse/client/plugins/response_target.rb:21:in `call' /var/www/discourse/vendor/bundle/ruby/2.3.0/gems/aws-sdk-core-2.5.3/lib/seahorse/client/request.rb:70:in `send_request' /var/www/discourse/vendor/bundle/ruby/2.3.0/gems/aws-sdk-core-2.5.3/lib/seahorse/client/base.rb:207:in `block (2 levels) in define_operation_methods' /var/www/discourse/vendor/bundle/ruby/2.3.0/gems/aws-sdk-resources-2.5.3/lib/aws-sdk-resources/services/s3/file_uploader.rb:42:in `block in put_object' /var/www/discourse/vendor/bundle/ruby/2.3.0/gems/aws-sdk-resources-2.5.3/lib/aws-sdk-resources/services/s3/file_uploader.rb:52:in `open_file' /var/www/discourse/vendor/bundle/ruby/2.3.0/gems/aws-sdk-resources-2.5.3/lib/aws-sdk-resources/services/s3/file_uploader.rb:41:in `put_object' /var/www/discourse/vendor/bundle/ruby/2.3.0/gems/aws-sdk-resources-2.5.3/lib/aws-sdk-resources/services/s3/file_uploader.rb:34:in `upload' /var/www/discourse/vendor/bundle/ruby/2.3.0/gems/aws-sdk-resources-2.5.3/lib/aws-sdk-resources/services/s3/object.rb:251:in `upload_file' /var/www/discourse/lib/s3_helper.rb:30:in `upload' /var/www/discourse/lib/file_store/s3_store.rb:41:in `store_file' /var/www/discourse/lib/file_store/s3_store.rb:17:in `store_upload' /var/www/discourse/app/models/upload.rb:205:in `block (2 levels) in create_for' /var/www/discourse/app/models/upload.rb:204:in `open' /var/www/discourse/app/models/upload.rb:204:in `block in create_for' /var/www/discourse/lib/distributed_mutex.rb:21:in `synchronize' /var/www/discourse/lib/distributed_mutex.rb:5:in `synchronize' /var/www/discourse/app/models/upload.rb:105:in `create_for' /var/www/discourse/app/models/user_avatar.rb:23:in `block in update_gravatar!' /var/www/discourse/lib/distributed_mutex.rb:21:in `synchronize' /var/www/discourse/lib/distributed_mutex.rb:5:in `synchronize' /var/www/discourse/app/models/user_avatar.rb:13:in `update_gravatar!' /var/www/discourse/app/jobs/regular/update_gravatar.rb:12:in `execute' /var/www/discourse/app/jobs/base.rb:154:in `block (2 levels) in perform'
hostname ip-172-31-48-109-app process_id 22178 application_version 65081a91939f9ed85a74a2d0023c864444195854 current_db default current_hostname [REDACTED] job Jobs::UpdateGravatar problem_db default opts user_id [12, 19, 10, 43] avatar_id [14, 21, 12, 45] current_site_id default
[HASH] REDACTED-BUCKET-NAME [23/Mar/2017:21:48:58 +0000] 184.108.40.206 arn:aws:iam::REDACTED:user/discourse-uploads REDACTED REST.HEAD.BUCKET - "HEAD / HTTP/1.1" 200 - - - 26 25 "-" "aws-sdk-ruby2/2.5.3 ruby/2.3.3 x86_64-linux resources" - [HASH] REDACTED-BUCKET-NAME [23/Mar/2017:21:48:58 +0000] 220.127.116.11 arn:aws:iam::REDACTED:user/discourse-uploads REDACTED REST.PUT.OBJECT original/2X/1/1e03e179f46682a0049e5f80e0ab43cde9f3613e.png "PUT /original/2X/1/1e03e179f46682a0049e5f80e0ab43cde9f3613e.png HTTP/1.1" 403 AccessDenied 243 201977 5 - "-" "aws-sdk-ruby2/2.5.3 ruby/2.3.3 x86_64-linux resources" -
I successfully used AWS CLI to verify the credentials I am using are capable of uploading a file to the bucket.
Perhaps I’ve just been looking at it for too long, can anyone see if I’m overlooking something obvious?
I’ve set up S3 with this guide and it works fine for new uploads. Thanks for that!
Now the problem:
After successful running Rake
I have a lot (several hundred) images with the following URL:
Some pictures were displayed correctly, but most of them were not.
The original URL was
https://www.mydomain.com/uploads/default/optimized/2X/4/8985bf4653fafbbd255643394dd2b9877fda0ac4.jpg and it was set to
I had to restore the database because many images were broken. Running latest, default-docker setup and nginx reverse-proxy.
You have to rebake all posts to fix this issue - its a bug in the migrate script. See e.g.
for a discussion of the issue.
In my migration I got maybe half the old images to migrate over via
uploads:migrate_to_s3 and subsequent rebake, but half didn’t make it. So, I have had to keep all the old images on the server since I don’t know which ones made it over and which ones did not.
Nice share. It works flawlessly. Although, you will only get your keys after you create a user and you must have created your policy before you can complete the creating of a “user”.
I’m having trouble getting ours to work. If you follow these steps and create an IAM user, are you supposed to check off the box “s3 use iam profile” in Discourse > Admin > Settings?
And if so, checking that box says NOTE: enabling will override “s3 access key id” and “s3 secret access key” settings. so does that mean you don’t enter the 2 keys? And how does it know which IAM user to use?
I was able to get S3 working briefly on my site, but as soon as I did, the media player stopped working correctly. Is this just the way it is when using S3?
And this is still the case in 2018? Sorry, just want to make sure?
Well, s3 support is FAR more polished today and even used on meta. I can highly recommend using it.
I would say, it depends, if you expect lots of uploads going with s3 + an s3 CDN is probably way easier to cope with but costs will be a tiny bit higher.
That is my question as S3 doesn’t seem to be an easy setup on Discourse. I am having trouble getting it setup
That is a bit unfair, using IAM is very fiddly, the general hobbyist installers would just go with Digital Ocean meaning that IAM is not even an option for them. I would recommend going without IAM first and only after you get the standard way going, go for ultra ninja mode.
My apologies - wasn’t saying the problem was Discourse - it just is a bit confusing of what to put where and what’s needed and such. I don’t think it’s easy to setup, either with IAM or without - but that doesn’t point the blame at Discourse.
Oh it is far more complicated than built-in cause multiple services are involved, Amazon do a spectacular job at making the web ux as complicated as possible for mere humans.
What we need to focus on is improving the OP here with better screenshots and better notes.