100 ignores is a signal that someone is not appropriately engaging with the community and there is an antisocial pattern of behaviour. I recently dealt with a similar issue where one person would consistently derail topics and would repeatedly engage in bad faith arguments. It was never anything bannable but over time I received so many complaints that this individual was damaging their experience of participating on the forum.
So I would say, yes you definitely can make it a popularity contest of who can or can not stay. Or rather, someone extremely unpopular is probably engaging in some kind of behavior that is not compatible with “civil discussion”. And that should be addressed.
In my case I started with DM warnings about the type of behavior causing issues. “Don’t derail every conversation, avoid baiting other users”. It continued, so I escalated the punishment. Temporary suspension for 3 days. It was still a problem so I finally indefinitely silenced them. A person who is overall a net negative contributor to your forum should not be prioritized