Prefers-color-scheme only considers the OS mode.
For the interface color selector to work properly for non-color-scheme colors and other CSS values, you must always use dark-light-choose instead like explained here Update themes and plugins to support automatic dark mode - #5 by pmusaraj