Yes you would either need a dedicated category or tag to function best or a method to detect poll
But in my idea would be more to have a dedicated category.
Imho is why a feature request or plugin to add category security option for voting However a dedicated tag might be more flexible.