The S3 configuration is in the app.yml. Not site settings.
Edit:
I realize I’m not being overly explanative and I’m not meaning to hide details.
We use OVH S3 and it’s configured in app.yml.
I backed up our test forum without uploads but still had S3 enabled at that point.
I then restored it on the new site with the exact same app.yml and that’s where the problem started. To be clear it’s fixed right now, but I’m unsure if it was me rebaking it several times or discourse caching aggressively. Which is why I need to know how to really do this properly and get it right first time. My fear is that if I ever need to restore a back up to our production instance and we run into this, I need to know exactly how to fix this asap before users notice.