If you search here for “reactions” you’ll find a few interesting topics. The first is about an official plugin which I’m sure you could use for downvotes - you could have a 
reaction icon (see Discourse Reactions - Beyond Likes).
If you just want downvotes you could change the 
to (see Change the Like icon to 👍)!