It’s very likely to always be the case that topics do not have permissions and Categories do. If you want only people in THAT GROUP to be able to see something then you have to create a category just for THAT GROUP.
You could conceivably use tags and find a way (a plugin, maybe?) to have everyone who is not in THAT GROUP to mute that tag. This could keep them from seeing the tagged categories unless they chose to un-mute that tag (or your plugin coerced them to always mute it). If it would be merely annoying that people not in THAT GROUP not see the posts, this could be a solution.