Please see screenshots, as they speak for me.
Seems like an easy CSS fix @Johani can you take this?
Fixed via
https://github.com/discourse/discourse/commit/fb0b54f7f810ecfd425c4ed2ca8b6abbc8324c7b
A selector specific to the badge system being on was preventing some styles from applying when badges are turned off.
Thanks for reporting @nineofcups49