Trouble restoring backup--SiteSetting::Upload.s3_base_url is failing--because enable_s3_uploads was set in database

General
1

I have a backup from the default site on a multisite instance. Uploads are on S3. I have successfully created a backup with include_s3_uploads_in_backups set to true.

When I try to restore this from the command line on either another multisite instance (with s3) or a fresh standard install, I get an error: Something went wrong while remapping uploads.

I have added a bunch of puts to lib/backup_restore/uploads_restorer.rb and have pinned down that this will cause the error:

  puts "base url #{SiteSetting::Upload.s3_base_url}"

the next, which seems to be the one that was failing before my debugging, line is:

       current_s3_base_url = SiteSetting::Upload.enable_s3_uploads ? SiteSetting::Upload.s3_base_url : nil

I’ve tried commenting out various s3 variables in discourse.conf to see if there is some bad value in the database that’s somehow causing the problem, but they are all empty.

Hmm. And on the clean install, the backup failed and now the site returns

{
errors: [
"`s3_upload_bucket` site setting has to be set."
]
}

So maybe some other s3 setting is set in the database that I misssed. Yes enable_s3_uploads is set in the database. I am confused about the relationship between use_s3 and enable_s3_uploads…

1 Like
2

So it’s confirmed that the issue was that some years ago when I first tried to set up S3, I set enable_s3_uploads in the database, which causes the restore not to work because it tries to upload to s3 but doesn’t have enough information. (Sigh. But it fails restoring the the multisite/s3 instace because some files “are not migrated to s3”.)

I wonder whether it would make sense to make the s3 uploads settings all hidden. I’m pretty sure that no good can come from setting them in the UX. But it can be handy to set s3 uploads in the database. . . . maybe.

The one restored to the standard install is failing to show uploads because it’s doing this:

      Started GET "/uploads/short-url/puhaSNHeEy1S2knGFQIAZ8lprRy.pdf" for 68.11.35.109 at 2022-01-25 19:48:06 +0000
Processing by UploadsController#show_short as PDF
  Parameters: {"base62"=>"puhaSNHeEy1S2knGFQIAZ8lprRy", "extension"=>"pdf"}
Sent file /var/www/discourse/public/default/original/1X/b2a283b9381b837234e7d4830b.pdf (4.4ms)
Completed 500 Internal Server Error in 130ms (ActiveRecord: 0.0ms | Allocations: 10019)
ActionController::MissingFile (Cannot read file /var/www/discourse/public/default/original/1X/b2a283b9381b837234e7d4830.pdf)

The path that it’s looking for is /var/www/discourse/public/default/original/1X/ instead of /var/www/discourse/public/uploads/default/original/1X/

… . .

And that’s because I did a remap trying to fix the S3 bucket references and that broke the paths in the local urls.

I think maybe this is to much in the weeds to be useful to anyone. Though I still think that hiding the S3 upload variables from the UX is a good idea.

3 Likes