How to limit invites to admins

Invites are allowed to trust level 2 and higher as I recall.