I did this for a client a while back. Trust levels weren’t going to work for us, so instead we created private groups per-category which each user was automatically provisioned into, and category moderators were owners of.
“Bans” were as simple as removing said memberships which meant zero work on behalf of the administration.
Slightly more effort to get started, but effectively zero thereafter.