Ability to EXCLUDE in security settings

Is there any chance that the Security settings for categories might enable an admin to say who CAN’T access it rather than just who can?

For example;

  • Group A is All Users, and
  • Group B is a small group of specialists (e.g. invited guests),

the only way to create a category viewable ONLY to those in A who are not in B is to create a new Group that includes all users minus the very small number of guests

It is frustrating to have to work around this issue and to maintain new groupings which only exist for this purpose.

Is there a simpler workaround?

I’ve tried searching for previous posts on this subject but the terms are fairly general.