Yeah, the fundamental dev xp thing here is that we should be disallowing this fiddling with automatic groups and force you down the route of using PUT {{site}}/admin/users/{{userId}}/trust_level only to transition stuff. Return a 422 or something if people attempt to use endpoints to manually add people to the groups or remove.
Unrolling trust level should always happen from here: