That hasn’t been the case for a long time actually. It used to be that the flair was directly tied to the primary group, but nowadays the flair option in your preferences can be used to select any flair one of the groups you’re in has even if the ability to select your primary group is turned off for normal users.
My preferences for my standard test account on the NaNoWriMo instance which has the ability to select your primary group turned off for regular users.
(And in case anyone is curious about it, I have multiple test accounts there with different trust and access levels that are kept in a custom group whose name references my moderator account)