How to manually migrate all files from S3 back to EC2 Server?

Product Support

AI-generated summary

Manually Migrating Files from S3 to EC2 Server

The discussion revolves around manually migrating files from S3 to local storage on an EC2 server. Pravi initially reported broken images on their site after rebaking, despite the images being stored in S3.

Pravi followed the steps outlined in Migrating uploads from S3 to local but only some files were migrated. They decided to manually migrate all files and sought advice on the process.

Pravi eventually found a solution by:

  1. Entering the app container using ./launcher enter app
  2. Installing awscli and configuring AWS with their S3 credentials
  3. Syncing the S3 bucket with their local public/uploads/default/ directory using aws s3 sync s3://mybucketname/
  4. Running rake posts:rebake and rebuilding the app using ./launcher rebuild app

However, they still encountered a broken image link issue, which may be a separate problem.

1

After restoring my site in a new server all the images are broken. The images are broken even after rebake. Is it possible to migrate manually? if so, How can I manually migrate all files from S3 to local storage?

2

Do the images on the new site point to the S3 bucket? If so, they should work.

Maybe have a look at Migrating uploads from S3 to local - #2 by Stephen, though I think there should be a howto.

3

Hi, thanks for responding.

I followed the same steps mentioned by you earlier, but nothing happened. I ended up having a site with broken images. see my previous topic,

Only some files are migrated back from s3, not all.

root@ubuntu-blr-app:/var/www/discourse# rake uploads:migrate_from_s3
Migrating uploads from S3 to local storage for 'default'...
..Done!

So, I was thinking of manually migrating all the files and I don’t know if it’s possible..?

4

s3
s3484×285 7.76 KB

I have the above three folders in my s3 bucket. I don’t know the location in my server where I have to copy each of the above folders from S3. After copying will a rake posts:rebake would do the final job??

5

Guys, Any thoughts on this issue??

6

Finally, I think I find a solution to my problem. I think I have solved 95% of the problem. I am not sure if its the best way to do it.

./launcher enter app
sudo apt install awscli
aws configure

Configured aws with my S3 ID and password. then

cd public/uploads/default/
aws s3 sync s3://mybucketname/

This command downloaded all the three directories with all the files from S3. Since I can’t find the folder named inventory anywhere in my previous server, I deleted it (Not sure about deleting this folder). Then

rake posts:rebake
exit
./launcer rebuild app

Bam, everything started working again as normal. But I find image links in one of the post is broken, but if edited I can view the image in the editor. I download the same image and tried to upload it again, but still the broken image. Not sure if its a separate problem or it’s related to the current problem.

s55
s551589×823 228 KB