Probleem met het herstellen van back-up--SiteSetting::Upload.s3_base_url mislukt--omdat enable_s3_uploads in de database is ingesteld

AI-generated summary

pfaffman encountered issues restoring a Discourse backup from a multisite instance with S3 uploads enabled. The restore failed with “Something went wrong while remapping uploads” and later revealed that enable_s3_uploads was set in the database, causing the system to expect S3 configuration even when incomplete — leading to errors like “s3_upload_bucket site setting has to be set.” On a clean install, uploads failed to display because path remapping (attempted to fix S3 references) incorrectly altered local upload paths — e.g., looking for /public/default/original/... instead of /public/uploads/default/original/.... pfaffman confirmed the root cause: manually setting enable_s3_uploads years ago without proper S3 setup breaks restores. They suggest hiding S3-related settings from the UI to prevent misconfiguration, though acknowledging that database-level S3 configuration can still be useful. The issue highlights the fragility of mixing partial S3 settings with local uploads during restore, especially when paths are remapped incorrectly.

I have a backup from the default site on a multisite instance. Uploads are on S3. I have successfully created a backup with include_s3_uploads_in_backups set to true.

When I try to restore this from the command line on either another multisite instance (with s3) or a fresh standard install, I get an error: Something went wrong while remapping uploads.

I have added a bunch of puts to lib/backup_restore/uploads_restorer.rb and have pinned down that this will cause the error:

  puts "base url #{SiteSetting::Upload.s3_base_url}"

the next, which seems to be the one that was failing before my debugging, line is:

       current_s3_base_url = SiteSetting::Upload.enable_s3_uploads ? SiteSetting::Upload.s3_base_url : nil

I’ve tried commenting out various s3 variables in discourse.conf to see if there is some bad value in the database that’s somehow causing the problem, but they are all empty.

Hmm. And on the clean install, the backup failed and now the site returns

{
errors: [
"`s3_upload_bucket` site setting has to be set."
]
}

So maybe some other s3 setting is set in the database that I misssed. Yes enable_s3_uploads is set in the database. I am confused about the relationship between use_s3 and enable_s3_uploads…

1 like

So it’s confirmed that the issue was that some years ago when I first tried to set up S3, I set enable_s3_uploads in the database, which causes the restore not to work because it tries to upload to s3 but doesn’t have enough information. (Sigh. But it fails restoring the the multisite/s3 instace because some files “are not migrated to s3”.)

I wonder whether it would make sense to make the s3 uploads settings all hidden. I’m pretty sure that no good can come from setting them in the UX. But it can be handy to set s3 uploads in the database. . . . maybe.

The one restored to the standard install is failing to show uploads because it’s doing this:

      Started GET "/uploads/short-url/puhaSNHeEy1S2knGFQIAZ8lprRy.pdf" for 68.11.35.109 at 2022-01-25 19:48:06 +0000
Processing by UploadsController#show_short as PDF
  Parameters: {"base62"=>"puhaSNHeEy1S2knGFQIAZ8lprRy", "extension"=>"pdf"}
Sent file /var/www/discourse/public/default/original/1X/b2a283b9381b837234e7d4830b.pdf (4.4ms)
Completed 500 Internal Server Error in 130ms (ActiveRecord: 0.0ms | Allocations: 10019)
ActionController::MissingFile (Cannot read file /var/www/discourse/public/default/original/1X/b2a283b9381b837234e7d4830.pdf)

The path that it’s looking for is /var/www/discourse/public/default/original/1X/ instead of /var/www/discourse/public/uploads/default/original/1X/

… . .

And that’s because I did a remap trying to fix the S3 bucket references and that broke the paths in the local urls.

I think maybe this is to much in the weeds to be useful to anyone. Though I still think that hiding the S3 upload variables from the UX is a good idea.

3 likes