Welcome to Meta 
What do you want your moderators to be able to do, and what exactly do you want to prevent them from doing?
Maybe making them category moderators instead is the option you are looking for? Category mods can only moderate content, but not users. Also, take a look at the permissions of users with trust level 4. Those can also, for example, close and pin topics. But they cannot handle flags.
Trust Level Permissions Reference is a great overview of the default permissions and also mentions the related settings. You can also read more about category moderators in Category Group Review/Moderation and Category Moderator Improvements