This issue has been resolved. I’m assuming it was fixed by adding the locale to the anonymous cache key.
The set locale from accept language header setting can safely be enabled on both private and public sites. This doesn’t meet the OP’s request for a language switcher, but it should solve this problem for most new users on the site:
If the allow user locale and set locale from accept language header site settings are enabled, anonymous users will have the site served to them in the locale they have set for their browser. New users also have their locale set to their browser’s language when they sign up on the site.
I wonder how much this has changed since 2016. My sense is that machine translations have improved a lot since then.