Download avatars locally from S3 (shutting down S3 in my site)

Product Support

AI-generated summary

The discussion revolves around downloading avatars locally from S3 before shutting it down due to high costs. skozz has a large community and wants to keep user avatars after disconnecting S3. They also aim to disable image uploads but allow avatar uploads only.

skozz has tried removing S3 credentials and found that some users still have their avatars, while others don’t. pfaffman suggested using a rake task for migrating uploads from S3 to local, but skozz is unsure how to scope it to avatars only, as downloading all images from S3 would be costly.

Falco questioned whether skozz is using S3 without a CDN, which could be contributing to the high costs. skozz confirmed they are not using a CDN and will explore alternative options, such as using a CDN to reduce costs.

1

Hola! I tried for days to get the info I need searching in the Meta and internet without success.

TD;DR

I´m shutting down S3 due to the costs. I would like to store the user avatars locally to avoid user losing their avatars before switching off S3.

Background (long story short)

  • I have a ~5 years old community. At the beginning storing images locally, and then switched to store them in S3.
  • Today I have the discourse app in forum.example.com, and I migrating it to example.com
  • In the process, I’m reducing the high costs (it is a huge community), and the main one is S3. We don’t care about losing the link to uploaded images to the posts, however, we would like to keep the user avatars.
  • After migrating the forum to example.com I tried just removing the S3 credentials to see what happens. I found that part of the user still having their avatars, and others don’t

So, now I’m struggling to meet the following goals:

  1. Keep user avatars after the S3 disconnection
  2. Disable the upload images to the forum, but allow ONLY upload avatars.

Any input, idea or help will be very appreciated :heart:

2

There is a rake task that you can search for Migrating uploads from s3 to local.

3

Are you by any chance using S3 without a CDN? S3 should not be one of the main costs, unless it’s a extremely outlier community, if properly configured with a CDN fronting all traffic.

4

Good point. Nope, I’m not using a CND. Apart from the AWS, is there any other one commonly used by the community? I will start checking them asap.

Thanks for sharing! however, I don’t know how to scope it only to avatars. If I run a download of all the images from S3, I’d need a mortgage :stuck_out_tongue: