This does seem to be expected behaviour. If you look at the Trust Level Permissions Reference guide you’ll see a lot of user actions are TL (trust level) dependant including those of category moderators.
are hardcoded to be only available for staff and leader (TL4).
I don’t see any way to modify this behavior (though it’s possible to do something with a TC).
I agree that it would be nice to have better granular control here.