Glad it helped @deepmehtait
4 Likes
Shouldnt we add this ? :
arn:aws:s3:::accesspoint/*"
1 Like
What this error mean ?
NameError: uninitialized constant FileStore::ToS3Migration::Aws
1 Like
The problem has been fixed:
8 Likes
Hi there guys,
I’m moving a Discourse installation to a new server. The domain stays the same, we have moved all old image uploads to a S3 digital ocean spaces with RCLONE. Did a clean discourse install, uploaded a backup without images. So far so good, all posts are there.
It is working with new uploads, but i am having a hard time rerouting old uploads in posts to this new location.
The old location was domain[dot]com/uploads/
The new location with all uploads in place is a space.ams3.digitaloceanspaces[dot]com/
I have tried remapping:
discourse remap [olddomain.com/uploads/](http://olddomain.com/uploads/) [space.ams3.digitaloceanspaces.com/](http://space.ams3.digitaloceanspaces.com/)
and i have tried:
rake posts:rebake
rake posts:rebake_match[“uploads”]
The paths that old images are linking to is still the old domain path. Seems like the remap and the rebake had no effect. Am i missing something obvious here?
- edit i just learned that migrate_to_s3 first checks if a file is already there which is great. Downside is that i see that migrate_to_s3 does not like to work with Digital Ocean spaces.
1 Like
If you could tell, If we had more than 1 bucket, can we assign all those buckets in this section of the (single) policy (two lines for each/one bucket)?
1 Like
Updated guide to include placeholder forms, as it’s really easy to get confused with all the different bucket names!
8 Likes
What is the risk involved while keeping all public access always unblocked??
1 Like
Hmm, yeah, we seem to be missing instructions for setting up CloudFront which would allow for public access to be blocked.
3 Likes
Hello
I need help. My issue with S3 configuration :
When I add an image into a post the image was not show. I have this text instead :
My site : Les animaux dans Coin Master - Le jeu en détail - Coin Master Stratégies
But When I go to the console I can see the file into my bucket.
The backup is working and was send to S3.
So why I can see the image into my post ?
Thanks for help
1 Like
I’ve got some additional AWS information that might be useful.
The IAM policy in @zogstrip’s post isn’t perfectly ideal. If you apply it, the IAM console will complain about issues, though it may still work to get you the permissions that you need.
By using asterisked Get* and, the policy would attempt to assign a couple of permissions that need defined resources beyond just the bucket name or object name: GetAccessPointPolicy, and GetJobTagging.
Also, List* includes ListAllMyBuckets and HeadBucket, which is redundant since it’s already in the second statement (and invalid in the first).
I think, ideally, we’d have an exact list of required permissions instead of using asterisked Get* and List*. I am not confident that this example policy is the “minimum required permissions.”
But it’s also entirely possible that I’m worried about nothing.
This is the policy that I’ve used. I haven’t tested extensively. I know that I could easily just apply s3:*, but also, I’m not sure why I’d want to give Discourse permission to delete the bucket. I just like going with minimum permissions whenever possible:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Action": [
"s3:PutObjectVersionAcl",
"s3:PutObjectAcl",
"s3:PutObject",
"s3:PutLifecycleConfiguration",
"s3:PutBucketCORS",
"s3:ListBucket",
"s3:GetReplicationConfiguration",
"s3:GetObjectVersionTorrent",
"s3:GetObjectVersionTagging",
"s3:GetObjectVersionForReplication",
"s3:GetObjectVersionAcl",
"s3:GetObjectVersion",
"s3:GetObjectTorrent",
"s3:GetObjectTagging",
"s3:GetObjectRetention",
"s3:GetObjectLegalHold",
"s3:GetObjectAcl",
"s3:GetObject",
"s3:GetMetricsConfiguration",
"s3:GetLifecycleConfiguration",
"s3:GetInventoryConfiguration",
"s3:GetEncryptionConfiguration",
"s3:GetBucketWebsite",
"s3:GetBucketVersioning",
"s3:GetBucketTagging",
"s3:GetBucketRequestPayment",
"s3:GetBucketPublicAccessBlock",
"s3:GetBucketPolicyStatus",
"s3:GetBucketPolicy",
"s3:GetBucketObjectLockConfiguration",
"s3:GetBucketNotification",
"s3:GetBucketLogging",
"s3:GetBucketLocation",
"s3:GetBucketCORS",
"s3:GetBucketAcl",
"s3:DeleteObject",
"s3:CreateBucket",
"s3:AbortMultipartUpload"
],
"Resource": [
"arn:aws:s3:::my-content-bucket/*",
"arn:aws:s3:::my-content-bucket",
"arn:aws:s3:::my-backup-bucket/*",
"arn:aws:s3:::my-backup-bucket"
]
},
{
"Sid": "",
"Effect": "Allow",
"Action": [
"s3:ListAllMyBuckets",
"s3:HeadBucket"
],
"Resource": "*"
}
]
}
Some more AWS notes!
- If you’re new to AWS and are looking to move more of your Discourse infrastructure over there, I can’t recommend learning Terraform enough.
- Use AWS IAM Instance Profiles instead of AWS secret keys to manage S3 permissions from your EC2 instance. There’s a convenient checkbox in the admin panel in Discourse to enable this option. This way, there’s no key/secret management. You do this by:
- Create an IAM role
- Create an IAM policy with the S3 permissions as described in this thread.
- Attach the IAM policy to the role (
aws_iam_role_policy_attachmentin Terraform) - Create an IAM Instance Profile with the IAM role (
aws_iam_instance_profilein Terraform) - Create your EC2 instance using this IAM Instance Profile (
iam_instance_profilein youraws_instanceoraws_launch_configurationresources in Terraform)
- Discourse manages its own lifecycle rules in the content bucket if you have the “s3 configure tombstone policy” option checked. This is done so that S3 will remove old content that has been deleted. If you are using this option, your Terraform code would benefit from ignoring this change with a
lifecycleblock withignore_changes = [ "lifecycle_rule" ]
By the way, if you’re using S3 for content and backups, adding in RDS for your database and an AWS Application Load Balancer puts you pretty much on the path to a full enterprise-grade Discourse setup with totally replaceable instances (at additional cost, but if you’re doing this for a company that’s probably the way to go). You just need some launch-time provisioning to install Docker and Discourse and bootstrap/run. This could be simple user data scripts, Chef, Ansible, whatever you want! The web_only template is what you go off of for that sort of setup. I you do all that, your instances will be Cattle instead of Pets.
2 Likes
Feel free to try it out and provide a better/tighter policy 
That policy is 7 years old, which is an eternity in Computer World, so things might have changed in S3.
2 Likes
Oh yes! I planned on it 
but I accidentally posted my comment before I was done with it. I’m about to add in the policy I used.
I don’t really intend to shrink the permissions much because I can see the Discourse does some things that other applications don’t tend to do, like managing its own S3 lifecycle rules (which is actually pretty cool!)
I also suspect that AWS added additional S3 actions, I’m not sure that things like GetJobTagging existed back then - which is probably why IAM now complains.
3 Likes
Two quick notes for @zogstrip’s original post.
-
In the Permissions instructions, it would be helpful if you would say whether or not to leave the two remaining public access checkboxes (below the two you show) checked or not.
-
The instructions say to click Create Policy in the IAM screen, but as far as I can tell at this point, you need to click Policies in the sidebar first.
Otherwise, the instructions worked perfectly for me to offload backups to S3.
2 Likes
Thanks for the feedback. Looks like the UI changed a bit since I initially wrote this topic.
Feel free to update OP since it’s a wiki topic 
2 Likes