A default light and dark color palette per theme which is set when selecting one or the other sounds good.
You get a different experience if you switch themes when browsing anonymously - Default: white background, Dark: black background for instance. (as the user hasn’t changed the palette.)