That setting only hides it from the page listing all the badges, for example: https://meta.discourse.org/badges. Badges are always announced to the recipient because it’s an award system – so there’s not a way to make certain badges only visible to admins.
The user notes plugin is indeed the way to go!
Would banning their IP work, or do you think they’d be angry/savvy enough to workaround it with a VPN?