Configure an S3 compatible object storage provider for uploads


I’m a bit stuck and confused and hope somebody can help me.
I first had a bitnami install and realized how much trouble this would give me along the way, I reinstalled using the standard install.
I was able to restore my backup and everything was fine, even though I went from the 2.8 to the 2.9 beta.

I tested again my backup on my google bucket and it still worked like a charm.

Note that all the S3 config was done through the web interface and not via ENV variables.

For GDPR reasons, I created a new backup bucket in europe (lets call it discourse-backup-eu ), and now that I was able to change the ENV variable, I set DISCOURSE_S3_ENDPOINT:, rebuild the app, changed the backup bucket name in the web interface, reran the backup and I was very please to see the backup files appearing on my new backup bucket in europe.

Now I wanted the uploads to go to another bucket and avoid filling up my vm disk space.

So I configured a new bucket (lets call it discourse-uploads), made it public, added the Storage Legacy Bucket Owner role to my service account on that new bucket.
Then added a rule to my existing load balancer (lets call it to use a backend bucket with cloud CDN enabled as instructed here. The rule being /discourse-uploads/* points to the bucket discourse-uploads

I tested my CDN with a test.jpg in the root of the bucket but I couldn’t reach it via and had to create a subfolder called discourse-uploads inside the bucket, moved the test.jpg inside and now I can see my test picture via

In the web UI, I changed the dummy bucket name under “s3 upload bucket” (I was forced to set previously while setting up the backup) to discourse-uploads, filled the CDN URL with and ticked “enable s3 uploads”.

From there on, if I would try to upload a image, I would get a popup saying Invalid Argument in the browser window (coming from a 422 error with a json content saying basically the same).

I tried to rebake all post, but had no effect, I still had the error.

So I figured, I should try using the env variables instead of the Web UI.

and use the following config:

DISCOURSE_S3_BUCKET: discourse-uploads/discourse-uploads
DISCOURSE_S3_BACKUP_BUCKET: discourse-backup-eu

I rebuilt the app.
Then I cannot open discourse anymore because none of the assets were uploaded to the bucket and get a 404

I thought that trying to upload directly to a sub folder in the bucket directly was a bit of a stretch enven though the OP suggests it works (at least for the backup bucket)

changed the env variable to
DISCOURSE_S3_BUCKET: discourse-uploads
(Thinking that later I can play with the host rule instead to avoid having to upload to a sub folder)

and rebuilt to see if anything gets uploaded, but nothing gets uploaded to the bucket and discourse still fails to open because of 404s.

So my questions are :

  • Do the Web UI and ENV variable collide ?
  • When are the assets supposed to be uploaded to the bucket ?
  • How can I debug this ? I don’t see any error in the logs
  • Is it possible to set a subfolder of a bucket in the config ?
  • Once this works, Are the previously uploaded images transferred to the bucket ? If I rebake, what will the url of the previously uploaded images look like ?

Thank you !

1 Like