I completely agree!
The status quo doesn’t make sense to me, as I only want the current trust level to be apparent in a badge collection. Ideally I’d like it to be the most obvious badge on the user card too (i.e. always displayed in the same prominent location), but not as in-your-face as an avatar flair.
I’ve achieved close to this by replacing the default trust badges with some custom ones that use SQL and revert automatically. Can share them if you like.