Instead of hiding the drop-down box from a moderator who is not an owner of any group, would it be possible to show the drop-down box with only a message of “You are not the owner of any groups”?
Such a message would give the moderator and their admin a better grasp of how to populate the field so that the moderator can accomplish their task of inviting a user while adding the user to a group.