Flair either not showing or delayed

I believe you do need to mark both as “default to primary” but then the order of operation matters.

If you add UserA to GroupB first, then GroupA, it seems GroupA will be set as their primary group.
If you add UserA to GroupA first, then GroupB, GroupB should be their primary group.

All of that is based on this: