I think you can disable it using the rails console:
You can follow the steps provided in Using Hidden Site Settings and use
SiteSetting.pop3_polling_enabled = "f"
But I think it’s a bug that this doesn’t work in the interface
Theoretically, the validator checks whether the setting should be deactivated and then does not check the configuration any further. Obviously, this is not working in this case. But it works for the rails workaround 
I think the problem is that the frontend doesn’t pass "f" but "false". That’s why the check for passing the validation doesn’t work. After changing
return true if val == "f"
to
return true if val == "f" || val == "false"
I was able to disable pop3_polling_enabled in the interface
Edit: Actually there is already a bug report from February