You need to revert the changes you made for subfolder, and change the domain name on your install once the new subdomain is in DNS.
Hopefully you have a backup from before you attempted to downgrade - take another backup before you attempt the above.
Then you’ll need to reestablish SSO.