I’ve been trying to repro this, no luck so far. Let me make sure I’m mirroring your config as closely as I can:
This category is reply/see to everyone, and create/reply/see to a single group (let’s call it foo)? Group foo is closed, you must be added by an admin or group owner? Users invited to the topic by admins are (sometimes) ending up in group foo?
Is all that correct?