Yes, it is.
You create two color palettes (or customize existing ones). I created my ‘Pink’ color palette based on the WCAG light palette and my ‘Dark Pink’ color palette by customizing the WCAG dark palette. I assume you don’t want your users to select the dark color palette also for light mode; then I would ensure that color palette can be selected by users is disabled. If that’s disabled on all palettes, the dropdown in the users’ interface preferences disappears.
Then your users will only see
Instead of the color selector you see in your preferences here at Meta.
You set your dark palette as the site-wide default color palette for dark mode in the default dark mode color scheme ID site setting.
And you put the light palette into the color palette setting of your theme.
The current limitation is that I cannot easily create two more palettes e.g., a light and a dark blue one and let the users choose between pink and blue. While the light color palette which is connected to the theme works fine, I cannot have a different default dark palette for that theme. But as long as you want only one light and one dark color palette, the current options are enough for you.