How to Configure Cloudflare R2 for your Discourse Community

Product Support Self-hosting
,
31

Some notes from my experience getting this working (not entirely)…

I made the mistake of putting all the config variables into the Discourse admin panel, which meant migrate_to_s3 wouldn’t work until I also added the variables to my app.yml

I didn’t realize at first this setup was also to serve static files from r2, so I had to do rake s3:upload_assets similar to joshhabka.

Because of this I also added this to my app.yml to ensure assets re-upload on rebuilds:

hooks:
  after_assets_precompile:
    - exec:
        cd: $home
        cmd:
          - sudo -E -u discourse bundle exec rake s3:upload_assets

I also encountered the You can only specify one checksum at a time error. This was fixed, as noted, by

  AWS_REQUEST_CHECKSUM_CALCULATION: "when_required"
  AWS_RESPONSE_CHECKSUM_VALIDATION: "when_required"

Then I ended up with the same 1 of x uploads are not migrated encountered by joshhabka.

Migrating uploads to S3 for 'default'...
Uploading files to S3...
 - Listing local files
 => 79 files
 - Listing S3 files
. => 81 files
 - Syncing files to S3
...............................................................................
Updating the URLs in the database...
Removing old optimized images...
Flagging all posts containing lightboxes for rebake...
18 posts were flagged for a rebake
rake aborted!
FileStore::ToS3MigrationError: 1 of 88 uploads are not migrated to S3. S3 migration failed for db 'default'. (FileStore::ToS3MigrationError)
/var/www/discourse/lib/file_store/to_s3_migration.rb:132:in 'FileStore::ToS3Migration#raise_or_log'
/var/www/discourse/lib/file_store/to_s3_migration.rb:73:in 'FileStore::ToS3Migration#migration_successful?'
/var/www/discourse/lib/file_store/to_s3_migration.rb:385:in 'FileStore::ToS3Migration#migrate_to_s3'
/var/www/discourse/lib/file_store/to_s3_migration.rb:59:in 'FileStore::ToS3Migration#migrate'
/var/www/discourse/lib/tasks/uploads.rake:126:in 'Object#migrate_to_s3'
/var/www/discourse/lib/tasks/uploads.rake:106:in 'block in Object#migrate_to_s3_all_sites'
/var/www/discourse/vendor/bundle/ruby/3.4.0/gems/rails_multisite-7.0.0/lib/rails_multisite/connection_management/null_instance.rb:49:in 'RailsMultisite::ConnectionManagement::NullInstance#with_connection'
/var/www/discourse/vendor/bundle/ruby/3.4.0/gems/rails_multisite-7.0.0/lib/rails_multisite/connection_management/null_instance.rb:36:in 'RailsMultisite::ConnectionManagement::NullInstance#each_connection'
/var/www/discourse/vendor/bundle/ruby/3.4.0/gems/rails_multisite-7.0.0/lib/rails_multisite/connection_management.rb:17:in 'RailsMultisite::ConnectionManagement.each_connection'
/var/www/discourse/lib/tasks/uploads.rake:104:in 'Object#migrate_to_s3_all_sites'
/var/www/discourse/lib/tasks/uploads.rake:100:in 'block in <main>'
/usr/local/bin/bundle:25:in 'Kernel#load'
/usr/local/bin/bundle:25:in '<main>'
Tasks: TOP => uploads:migrate_to_s3
(See full trace by running task with --trace)

I have not found a way to resolve this one myself. It seems like everything works until it gets to flagging posts for a rebake and then it fails. It did seem to migrate the content though, I wonder how I can figure out which content was not uploaded successfully.