Unable to setup S3 bucket

Support
1

Hi,

I’m trying to setup S3 uploads and it seems that the uploads are working correctly but the GET requests are failing for a weird reason. It seems that discourse is adding a “.cn” to the URI of the uploaded object

After uploading the image in my S3 bucket the public URL is:

https://hobig...bucket-eu.s3.eu-central-1.amazonaws.com/original/1X/5e894113...48918.jpeg

but when I inspect the browser my discourse forum is trying to load (notice the .cn)

https://hobig...bucket-eu.s3.eu-central-1.amazonaws.com.cn/original/1X/5e894113...48918.jpeg

Does anyone know why is this happening?

Here are my admin settings:

The image shows a configuration settings page for uploading files to Amazon S3, including options for s3 access key id, secret access key, region, bucket, and endpoint. (Subtitulado por IA)
The image shows a configuration settings page for uploading files to Amazon S3, including options for s3 access key id, secret access key, region, bucket, and endpoint. (Subtitulado por IA)1478×1318 257 KB

2

You should follow the instructions in Configure an S3 compatible object storage provider for uploads and/or Set up file and image uploads to S3 and put those settings in your yml file rather than in the database/UX.

You don’t need to configure an endpoint for AWS. You do want to configure a CDN.

1 Like
3

Hi @pfaffman,

It seems like you might have unintentionally pasted the same guide twice.

When you mention “settings in your yml file,” could you clarify which YAML file you’re referring to? Are you perhaps talking about the app.yml file located in the /var/discourse/containers folder?

Additionally, if I configure my S3/CloudFront integration directly through the YAML file, would those settings override what’s configured in the admin section of Discourse?

Thank you!

4

Sorry. One of those should have been Set up file and image uploads to S3.

See the linked topics that describe how to configure S3, but yes, the app.yml is what you want to change (since you didn’t mention web_only.yml)

Yes. Entering them in the YML file hides them from the UX.

5

Hi @pfaffman,

I have my S3 bucket, CloudFront setup, and CloudFront’s origin set to my S3 bucket. Here’s my current app.yml configuration:

ENV:
  DISCOURSE_USE_S3: true
  DISCOURSE_S3_REGION: eu-central-1
  DISCOURSE_S3_ACCESS_KEY_ID: AKIAWPLPUxxxxxx
  DISCOURSE_S3_SECRET_ACCESS_KEY: PaXQu7pKxxxx
  DISCOURSE_S3_CDN_URL: https://dsuxxxhrz2qn.cloudfront.net
  DISCOURSE_S3_BUCKET: hobigxxxxbucket-eu

After rebuilding the app with ./launcher rebuild app, when I visit the website, it just shows the loader without loading anything. Upon inspecting the network tab, I found that it can’t fetch the precompiled static assets (mostly .js), which I assume is because they aren’t in my S3 bucket. You can check it here: forum.hobiguru.com.

I also tried running the migration rake task, but to no avail:

root@ubuntu-s-1vcpu-1gb-fra1-01-app:/var/www/discourse# rake uploads:migrate_to_s3 --trace



** Invoke uploads:migrate_to_s3 (first_time)
** Invoke environment (first_time)
** Execute environment



** Execute uploads:migrate_to_s3
Please note that migrating to S3 is currently not reversible!
[CTRL+c] to cancel, [ENTER] to continue
Migrating uploads to S3 for 'default'...
Some uploads were not migrated to the new scheme. Running the migration, this may take a while...
rake aborted!
FileStore::ToS3MigrationError: Some uploads could not be migrated to the new scheme. You need to fix this manually. (FileStore::ToS3MigrationError)
/var/www/discourse/lib/file_store/to_s3_migration.rb:156:in `migrate_to_s3'
/var/www/discourse/lib/file_store/to_s3_migration.rb:59:in `migrate'
/var/www/discourse/lib/tasks/uploads.rake:126:in `migrate_to_s3'
/var/www/discourse/lib/tasks/uploads.rake:106:in `block in migrate_to_s3_all_sites'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rails_multisite-6.1.0/lib/rails_multisite/connection_management/null_instance.rb:49:in `with_connection'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rails_multisite-6.1.0/lib/rails_multisite/connection_management/null_instance.rb:36:in `each_connection'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rails_multisite-6.1.0/lib/rails_multisite/connection_management.rb:21:in `each_connection'
/var/www/discourse/lib/tasks/uploads.rake:104:in `migrate_to_s3_all_sites'
/var/www/discourse/lib/tasks/uploads.rake:100:in `block in <main>'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/task.rb:281:in `block in execute'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/task.rb:281:in `each'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/task.rb:281:in `execute'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/task.rb:219:in `block in invoke_with_call_chain'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/task.rb:199:in `synchronize'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/task.rb:199:in `invoke_with_call_chain'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/task.rb:188:in `invoke'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/application.rb:188:in `invoke_task'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/application.rb:138:in `block (2 levels) in top_level'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/application.rb:138:in `each'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/application.rb:138:in `block in top_level'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/application.rb:147:in `run_with_threads'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/application.rb:132:in `top_level'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/application.rb:83:in `block in run'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/application.rb:214:in `standard_exception_handling'
/var/www/discourse/vendor/bundle/ruby/3.3.0/gems/rake-13.2.1/lib/rake/application.rb:80:in `run'
bin/rake:13:in `<top (required)>'
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.18/lib/bundler/cli/exec.rb:58:in `load'
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.18/lib/bundler/cli/exec.rb:58:in `kernel_load'
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.18/lib/bundler/cli/exec.rb:23:in `run'
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.18/lib/bundler/cli.rb:455:in `exec'
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.18/lib/bundler/vendor/thor/lib/thor/command.rb:28:in `run'
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.18/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.18/lib/bundler/vendor/thor/lib/thor.rb:527:in `dispatch'
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.18/lib/bundler/cli.rb:35:in `dispatch'
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.18/lib/bundler/vendor/thor/lib/thor/base.rb:584:in `start'
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.18/lib/bundler/cli.rb:29:in `start'
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.18/exe/bundle:28:in `block in <top (required)>'
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.18/lib/bundler/friendly_errors.rb:117:in `with_friendly_errors'
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.18/exe/bundle:20:in `<top (required)>'
/usr/local/bin/bundle:25:in `load'

My original intention was to use S3 only for storing user uploads, and I added the CDN just to handle that. However, now I’m facing a new problem — my app’s static assets are also being served through the CDN, which I didn’t intend. :confused:

Is there a way to upload all static assets on app launch to the S3 bucket and then have them served through the CDN, or maybe a way to only serve users’ uploads through CDN or is there a better solution for this?

Maybe I’m still missing something obvoius ? dunno
Thanks for your help!

6

I suspect that’s because you didn’t follow the instructions.

Yeah. I understand. That’s possible, but you couldn’t make that work. This is the recommended way; it’s well documented and hundreds of people do it this way.

1 Like