I see what you mean. I made a little screencast to demonstrate it.
The same thing happens on my site (and here on meta), even with the selector on the sidebar. It happens to be a bit less ugly because the colors are closer, and the button is not so prominent in the header as in your case.
It’s probably a fairly minor design fix to prevent it and I agree it’s a good idea.