Category-Specific Moderators, phase 1 RFC

The question of invitations relates to groups rather than categories. Group managers (as opposed to category moderators) have been implemented on the back-end but not exposed in the UI. But you could use that to set up specific users that can invite users into topics that are in restricted categories.

https://meta.discourse.org/t/invitations-to-group-restricted-topics/25895