# Set up file and image uploads to S3

**URL:** https://meta.discourse.org/t/set-up-file-and-image-uploads-to-s3/7229
**Category:** Self-Hosting
**Tags:** configuring, how-to
**Created:** [6 juni 2013 om 19:37 UTC](https://meta.discourse.org/t/set-up-file-and-image-uploads-to-s3/7229 "2013-06-06T19:37:43Z")
**Posts on this page:** 16
**Page:** 2

<div class="post-metadata">

### Author: ![Denis\_Didkovsky](https://avatars.discourse-cdn.com/v4/letter/d/53a042/32.png) [@Denis\_Didkovsky](https://meta.discourse.org/u/Denis_Didkovsky)
#### Post date: [17 oktober 2024 om 17:21 UTC](https://meta.discourse.org/t/set-up-file-and-image-uploads-to-s3/7229/267 "2024-10-17T17:21:47Z")

</div>

Is the fist post in the above  
[https://meta.discourse.org/t/set-up-file-and-image-uploads-to-s3](https://meta.discourse.org/t/set-up-file-and-image-uploads-to-s3)  
is still valid guide that allows admins to make it? If no, is there any other place where 2024 correct S3 guide is located? It is said that this is official and supported by Discource team.

we implemented it as in the above and and images are broken on discourse instance although they are being uploaded to S3.

**Our flow**

1. Set up the S3 bucket itself and the policy in S3 as in the above (all steps are accurately followed)

2. Set up the env in app.yml and in admin (identical) as in the above

3. launcher rebuild app

4. any CDN is NOT used at all as our understanding is that it is optional and it should work without CDN.

_ **Result:** _

1. Images are being uploaded to bucket by discourse
2. links in posts are correctly replaced by Discourse to amazon links like  
[xxx-bucket.s3.dualstack.us-east-1.amazonaws.com/original/1X/a1b21eb5de071799d4b5e5215619d11d28602dfe.jpeg](http://xxx-bucket.s3.dualstack.us-east-1.amazonaws.com/original/1X/a1b21eb5de071799d4b5e5215619d11d28602dfe.jpeg)
3. links **are not accessible** (assumed because of the S3 policy in official guide)

Assumed change for the official S3 policy in guide . **Principal** : This policy does not specify a `Principal`, which means it applies to whoever has been authenticated with the right permissions. If to allow anonymous (public) access, we need to specify `"Principal": "*"`. That also might not be good solution as it makes bucket public.  
**this is assumption as we appreciate if someone will update the guide to make it correct in 2024**

---

<div class="post-metadata">

### Author: ![aosus](https://avatars.discourse-cdn.com/v4/letter/a/ed8c4c/32.png) [@aosus](https://meta.discourse.org/u/aosus)
#### Post date: [24 oktober 2024 om 20:59 UTC](https://meta.discourse.org/t/set-up-file-and-image-uploads-to-s3/7229/268 "2024-10-24T20:59:28Z")

</div>

Hey,  
.we are using Cloudflare R2 for s3 storage(it works now) and i wanted to move discourse to another server.  
However profile pictures were missing when restoring from a backup even when checking the “backup uploads” option.

so i ran the s3 migrate command on the old server to migrate images that are still local, and it’s failing with this trace, it’s not clear what’s the reason.

```bash
rake uploads:migrate_to_s3 --trace
** Invoke uploads:migrate_to_s3 (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute uploads:migrate_to_s3
Please note that migrating to S3 is currently not reversible! 
[CTRL+c] to cancel, [ENTER] to continue

Migrating uploads to S3 for 'default'...
Uploading files to S3...
 - Listing local files
..... => 5123 files
 - Listing S3 files
....... => 6871 files
 - Syncing files to S3
.......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................^[[B............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
Updating the URLs in the database...
Removing old optimized images...
Flagging all posts containing lightboxes for rebake...
828 posts were flagged for a rebake
rake aborted!
FileStore::ToS3MigrationError: 4898 of 5838 uploads are not migrated to S3. S3 migration failed for db 'default'. (FileStore::ToS3MigrationError)
/var/www/discourse/lib/file_store/to_s3_migration.rb:132:in `raise_or_log'
/var/www/discourse/lib/file_store/to_s3_migration.rb:73:in `migration_successful?'
/var/www/discourse/lib/file_store/to_s3_migration.rb:383:in `migrate_to_s3'
/var/www/discourse/lib/file_store/to_s3_migration.rb:59:in `migrate'
/var/www/discourse/lib/tasks/uploads.rake:126:in `migrate_to_s3'
/var/www/discourse/lib/tasks/uploads.rake:106:in `block in migrate_to_s3_all_sites'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rails_multisite-6.1.0/lib/rails_multisite/connection_management/null_instance.rb:49:in `with_connection'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rails_multisite-6.1.0/lib/rails_multisite/connection_management/null_instance.rb:36:in `each_connection'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rails_multisite-6.1.0/lib/rails_multisite/connection_management.rb:21:in `each_connection'
/var/www/discourse/lib/tasks/uploads.rake:104:in `migrate_to_s3_all_sites'
/var/www/discourse/lib/tasks/uploads.rake:100:in `block in <main>'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/task.rb:281:in `block in execute'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/task.rb:281:in `each'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/task.rb:281:in `execute'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/task.rb:219:in `block in invoke_with_call_chain'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/task.rb:199:in `synchronize'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/task.rb:199:in `invoke_with_call_chain'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/task.rb:188:in `invoke'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/application.rb:188:in `invoke_task'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/application.rb:138:in `block (2 levels) in top_level'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/application.rb:138:in `each'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/application.rb:138:in `block in top_level'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/application.rb:147:in `run_with_threads'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/application.rb:132:in `top_level'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/application.rb:83:in `block in run'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/application.rb:214:in `standard_exception_handling'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/application.rb:80:in `run'
bin/rake:13:in `<top (required)>'
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.18/lib/bundler/cli/exec.rb:58:in `load'
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.18/lib/bundler/cli/exec.rb:58:in `kernel_load'
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.18/lib/bundler/cli/exec.rb:23:in `run'
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.18/lib/bundler/cli.rb:455:in `exec'
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.18/lib/bundler/vendor/thor/lib/thor/command.rb:28:in `run'
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.18/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.18/lib/bundler/vendor/thor/lib/thor.rb:527:in `dispatch'
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.18/lib/bundler/cli.rb:35:in `dispatch'
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.18/lib/bundler/vendor/thor/lib/thor/base.rb:584:in `start'
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.18/lib/bundler/cli.rb:29:in `start'
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.18/exe/bundle:28:in `block in <top (required)>'
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.18/lib/bundler/friendly_errors.rb:117:in `with_friendly_errors'
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.18/exe/bundle:20:in `<top (required)>'
/usr/local/bin/bundle:25:in `load'
/usr/local/bin/bundle:25:in `<main>'

```

it seems these pictures are still stored locally for some reason, even though i ran this migration script before.

---

<div class="post-metadata">

### Author: ![pfaffman](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/pfaffman/32/120154_2.png) [@pfaffman](https://meta.discourse.org/u/pfaffman)
#### Post date: [25 oktober 2024 om 22:16 UTC](https://meta.discourse.org/t/set-up-file-and-image-uploads-to-s3/7229/269 "2024-10-25T22:16:34Z")

</div>

My guess is that your changed something in your s3 setup and the missing images are on a different bucket or different path than the new ones.

You can look at

```
 Upload.pluck(:url)

```

To see what’s wrong, maybe

---

<div class="post-metadata">

### Author: ![Walter\_Jenkins](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/walter_jenkins/32/459298_2.png) [@Walter\_Jenkins](https://meta.discourse.org/u/Walter_Jenkins)
#### Post date: [4 november 2024 om 16:35 UTC](https://meta.discourse.org/t/set-up-file-and-image-uploads-to-s3/7229/270 "2024-11-04T16:35:44Z")

</div>

I recently had this same problem and was able to resolve the issue. I did so by updating the iam permissions with this role

```plaintext
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
               "s3:List*",
               "s3:Get*",
               "s3:AbortMultipartUpload",
               "s3:DeleteObject",
               "s3:PutObject",
               "s3:PutObjectAcl",
               "s3:PutObjectVersionAcl",
               "s3:PutLifecycleConfiguration",
               "s3:CreateBucket",
               "s3:PutBucketCORS"
      ],
      "Resource": [
        "arn:aws:s3:::my-bucket",
        "arn:aws:s3:::my-bucket/*"
      ]
    },
    {
       "Effect": "Allow",
       "Action": [
           "s3:ListAllMyBuckets",
           "s3:ListBucket"
       ],
       "Resource": "*"
    }
  ]
}

```

I also had to enable ACL object ownership

 ![image](https://global.discourse-cdn.com/meta/original/4X/2/1/4/21419ceddb1a0aeebd58c4a6246cf6e66ec1a005.jpeg)

Here are my full configs when I search s3

 ![This image shows a screenshot of a website with a blue border around it, containing various text and a list. (Captioned by AI)](https://global.discourse-cdn.com/meta/original/4X/4/7/0/47085e3e8a8a1811b4fa80a38cbe6574eed56adb.png)

---

<div class="post-metadata">

### Author: ![ChallaPradyumna](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/challapradyumna/32/484563_2.png) [@ChallaPradyumna](https://meta.discourse.org/u/ChallaPradyumna)
#### Post date: [24 januari 2025 om 22:18 UTC](https://meta.discourse.org/t/set-up-file-and-image-uploads-to-s3/7229/271 "2025-01-24T22:18:57Z")

</div>

Setup CORS on the bucket along with the above settings for multi-part uploads.

```plaintext
[
    {
        "AllowedHeaders": [
            "content-type",
            "x-amz-acl",
            "x-amz-meta-sha1-checksum"
        ],
        "AllowedMethods": [
            "GET",
            "HEAD",
            "PUT"
        ],
        "AllowedOrigins": [
            "*"
        ],
        "ExposeHeaders": [],
        "MaxAgeSeconds": 3000
    }
]

```

---

<div class="post-metadata">

### Author: ![Rhod](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/rhod/32/507923_2.png) [@Rhod](https://meta.discourse.org/u/Rhod)
#### Post date: [24 juni 2025 om 00:01 UTC](https://meta.discourse.org/t/set-up-file-and-image-uploads-to-s3/7229/272 "2025-06-24T00:01:04Z")

</div>

Is het mogelijk om S3-serviceproviders te wijzigen?

Ik zou graag willen weten of er problemen zullen zijn als ik probeer S3-serviceproviders te wijzigen, met uitzondering van reeds geüploade afbeeldingen. Ik heb hier geen grote problemen mee, aangezien de site nog niet operationeel is; ik ben er nog mee bezig.

Is het mogelijk om de afbeeldingen lokaal op te slaan en S3 vervolgens opnieuw te activeren met de nieuwe serviceprovider?

---

<div class="post-metadata">

### Author: ![pfaffman](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/pfaffman/32/120154_2.png) [@pfaffman](https://meta.discourse.org/u/pfaffman)
#### Post date: [24 juni 2025 om 16:34 UTC](https://meta.discourse.org/t/set-up-file-and-image-uploads-to-s3/7229/273 "2025-06-24T16:34:06Z")

</div>

U kunt `siteSettings.include_s3_uploads_in_backups=true` instellen in rails en vervolgens een back-up maken, de S3-instellingen wijzigen en de back-up herstellen.

---

<div class="post-metadata">

### Author: ![Rhod](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/rhod/32/507923_2.png) [@Rhod](https://meta.discourse.org/u/Rhod)
#### Post date: [28 juni 2025 om 06:50 UTC](https://meta.discourse.org/t/set-up-file-and-image-uploads-to-s3/7229/274 "2025-06-28T06:50:24Z")

</div>

Oké, ik snap het. Heel erg bedankt!

---

<div class="post-metadata">

### Author: ![ericomeehan](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/ericomeehan/32/542726_2.png) [@ericomeehan](https://meta.discourse.org/u/ericomeehan)
#### Post date: [1 juli 2025 om 22:02 UTC](https://meta.discourse.org/t/set-up-file-and-image-uploads-to-s3/7229/275 "2025-07-01T22:02:44Z")

</div>

> [@Discourse](#):
>
> ## One-way door
> 
> Unlike many configuration decisions in Discourse, note that using S3 is a “one-way door;” that is, a move that _cannot easily be reversed._ There is no safe or maintained way to move files out of S3 to be in the local uploads. In particular, the `migrate_to_s3` moves more than just post images to S3; files for which there is no reverse path. (For more details, see [Migrate\_from\_s3 problems](https://meta.discourse.org/t/migrate-from-s3-problems/119064)).

The linked thread mentions a PR to remove a `migrate_from_s3` task, and I have seen no reason to believe it has been added back later, but I wanted to check if this was still the case. I am considering migrating to MinIO and am a bit concerned with making it a dependency.

> [@pfaffman](#):
>
> You can set `siteSettings.include_s3_uploads_in_backups=true` in rails and then make a backup, change the S3 settings, and restore the backup.

I wonder if this could provide a workaround for what I mentioned above. Include that setting, make a backup, disable S3 settings, and restore the backup.

---

<div class="post-metadata">

### Author: ![pfaffman](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/pfaffman/32/120154_2.png) [@pfaffman](https://meta.discourse.org/u/pfaffman)
#### Post date: [1 juli 2025 om 22:21 UTC](https://meta.discourse.org/t/set-up-file-and-image-uploads-to-s3/7229/276 "2025-07-01T22:21:36Z")

</div>

Ja. Ik zou het herstellen naar een nieuwe server, zodat als er iets misgaat, je nog steeds een werkende server hebt.

---

<div class="post-metadata">

### Author: ![ericomeehan](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/ericomeehan/32/542726_2.png) [@ericomeehan](https://meta.discourse.org/u/ericomeehan)
#### Post date: [2 juli 2025 om 00:22 UTC](https://meta.discourse.org/t/set-up-file-and-image-uploads-to-s3/7229/277 "2025-07-02T00:22:29Z")

</div>

> [@Enable hidden setting to include S3 uploads in the backups](https://meta.discourse.org/t/enable-hidden-setting-to-include-s3-uploads-in-the-backups/276535/11):
>
> Ok. I didn’t know. But, what’s the diff between ‘Local Uploads’ and “Files Stored in S3 ‘Uploads’” folder? Also, though I haven’t changed the setting in web\_only or in rails console as told above YET, but only chose the one setting told [here](https://bathindahelper.com/admin/site_settings/category/backups). But still when I ran the backup, just 10 min ago, it showed as if it downloaded thousands of files, (which I presume could only be on my aws s3 storage bucket, and ‘Download’ word itself means it is downloading from remote S3 bucket folder). Also it is …

From this thread, it seems that this setting downloads the individual files from S3, but I wonder if there might be issues with posts referencing S3 URLs rather than local files. I’m not sure how that is handled in the database. I would be interested to know if your migration to another provider was successful, @Rhod.

---

<div class="post-metadata">

### Author: ![Rhod](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/rhod/32/507923_2.png) [@Rhod](https://meta.discourse.org/u/Rhod)
#### Post date: [2 juli 2025 om 14:41 UTC](https://meta.discourse.org/t/set-up-file-and-image-uploads-to-s3/7229/278 "2025-07-02T14:41:47Z")

</div>

> [@ericomeehan](#):
>
> … I would be interested to know if your migration to another provider was successful, @Rhod.

I made the transition smoothly. I emphasize that I didn’t have any major issues because the number of images or backups was minimal. The forum was entirely for testing purposes, not yet in “production”.

Finally, after several trials and errors, I reinstalled the forum on a new server (Hetzner) and configured Cloudflare S3 R2 (previously, I had Amazon S3), for backups and images with the server that finally convinced me.

I’m sorry I can’t be much help. I’ll see if I can test it on another server in the future.

---

<div class="post-metadata">

### Author: ![RoldanLT](https://avatars.discourse-cdn.com/v4/letter/r/da6949/32.png) [@RoldanLT](https://meta.discourse.org/u/RoldanLT)
#### Post date: [8 juli 2025 om 11:08 UTC](https://meta.discourse.org/t/set-up-file-and-image-uploads-to-s3/7229/279 "2025-07-08T11:08:00Z")

</div>

> [@Rhod](#):
>
> configured Cloudflare S3 R2

With CF R2, do your chat thumbnails work fine?

---

<div class="post-metadata">

### Author: ![Rhod](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/rhod/32/507923_2.png) [@Rhod](https://meta.discourse.org/u/Rhod)
#### Post date: [9 juli 2025 om 04:27 UTC](https://meta.discourse.org/t/set-up-file-and-image-uploads-to-s3/7229/280 "2025-07-09T04:27:31Z")

</div>

I hadn’t noticed that error; the truth is I have the chat disabled on my site.

I don’t think I’ll be using it, but I still have the same error you mentioned in your [post](https://meta.discourse.org/t/help-image-thumbnail-is-broken-on-chat-messages-when-using-cloudflare-r2/372078).

 ![2025-07-08](https://global.discourse-cdn.com/meta/original/4X/6/6/e/66e085a713afc19d0d8688f0ee918cc4e5e9daa8.png)

 ![La imagen muestra un error de autorización en un documento XML, con etiquetas que indican un argumento inválido. (Subtitulado por IA)](https://global.discourse-cdn.com/meta/original/4X/1/6/a/16a14e684f210ba6e7566cd7b695d7c394998668.png)

However, I’ll keep an eye on your [post](https://meta.discourse.org/t/help-image-thumbnail-is-broken-on-chat-messages-when-using-cloudflare-r2/372078)to see if they help you find a solution.

---

<div class="post-metadata">

### Author: ![nickrsan](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/nickrsan/32/509995_2.png) [@nickrsan](https://meta.discourse.org/u/nickrsan)
#### Post date: [8 augustus 2025 om 23:07 UTC](https://meta.discourse.org/t/set-up-file-and-image-uploads-to-s3/7229/281 "2025-08-08T23:07:20Z")

</div>

Bedankt Walter! Ik kwam hier om hetzelfde te posten, maar zag dat jij het al had gedaan.

@Discourse is er een kans dat de inhoud van Walters bericht wordt geïntegreerd in het oorspronkelijke bericht?

---

<div class="post-metadata">

### Author: ![mingyu\_zhao](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/mingyu_zhao/32/526876_2.png) [@mingyu\_zhao](https://meta.discourse.org/u/mingyu_zhao)
#### Post date: [25 oktober 2025 om 02:43 UTC](https://meta.discourse.org/t/set-up-file-and-image-uploads-to-s3/7229/282 "2025-10-25T02:43:17Z")

</div>

Can I upload videos to S3? Currently, when I upload videos, only the cover image is uploaded to S3, but the video itself cannot be uploaded.

[Previous page](https://meta.discourse.org/t/set-up-file-and-image-uploads-to-s3/7229.md?page=1)
