Well, you could achieve this by granting 0 votes to all trust levels below, say, TL3 or TL2. That would effectively disable voting for those users.
Speculating here, but I can understand the appeal of letting everyone see what’s being voted on, and how the votes are being cast, even if not everyone can vote. Kinda like how parliaments work.