Hi there! We are migrating all our uploads/images between two different s3 compatible services (both are digital ocean spaces if it matters) and I have decided that we are stuck in a quite bad state. I’ll start with explaining how the migration was done: We cloned/synced the initial bucket to th…

After running the toombstone recovery rake task and then the fix_missing_uploads rake task, I finally got it to start “fix” the images. It seems like it’s downloading them and uploading them again, and it takes a lot of time and uses a lot of resources, but at the least the users will get their ima…

Might be worth mentioning that I have tried to use the CDN uri provided for the spaces bucket as well (with a rebake).

Output from missing uploads: rake posts:missing_uploads Looking for missing uploads on: default Fixing missing uploads: 🚫 17075 post uploads are missing. 16906 uploads are missing. 1 of 16906 are old scheme uploads. 14646 of 139801 posts are affected. post_uploads have 3448 entries optimized_ima…

You can see Moving from one S3 bucket to another I think that i have a draft of howto that I will try to post tomorrow.

That would be very helpful, tyvm!

Hey @Jite ! See if this works for you. If it does, I’ll go about creating a proper howto Old buckets This assumes that you can install and configure a tool to move your data from your old bucket to a local machine and then again do the same from local to the new bucket. See aws cli sync (which…

Thank you for the response. This is basically what I did the last time, but I tried this again. The issue is that posts.count returns 0. All posts have the transperent.png file in the cooked post and contains a hash in the uncooked. Is there any way to make it resolve the image correctly during ba…

Hmm. Right. That cooked change works only to avoid the remake. If a remake does not work then something else is wrong. Maybe the assets are not where discourse thinks?

Well, it’s possible, but the “move” was pretty much a 1:1 move of all files in the bucket, hehe…

So you can replace the old bucket url with the new and it works? Do the values in Uploads look right?

I’m a bit scared that changing back to old bucket might start of a job to move everything to tombstone due to them being “old” now, but yes, the database seems to point to the correct (new location) images, it’s basically just the old posts that don’t resolve to the correct image (I’m guessing…).

更改用于上传的 s3 存储桶

支持

Jite (Johannes Tegnér) 2021 年4 月 30 日 07:10 5

那会非常有帮助，非常感谢！

Migrate S3 bucket to new S3 bucket (compatible service)

话题		回复	浏览量
Moving from one S3 bucket to another Support	6	2050	2021 年8 月 8 日
Migrate S3 bucket to new S3 bucket (compatible service) Support	1	761	2022 年3 月 31 日
Stop using Amazon s3 for uploads Support	51	5143	2021 年8 月 22 日
Migrating old uploads to S3 Self-hosting s3	8	1657	2021 年5 月 11 日
Moving uploads to a different S3 location Support	2	457	2020 年10 月 1 日

更改用于上传的 s3 存储桶

相关话题