The tag group visibility only affects who can see the tag once it’s applied, not who can see the topic itself. To restrict topic visibility you would need to place it in a category with group permissions: