So, I also wanted to put my uploads behind CloudfFlare, using Wasabi instead of S3, so no CloudFront available.
I ended up directly modifying the DB, and restarting discourse and it worked. Also, does https between CDN and S3 really matter if the bucket is public read only?
UPDATE site_settings SET value = 'uploads.domain.tld' WHERE name = 's3_upload_bucket';