I’ve just tested this on my local development site and can confirm that votes are not being returned to the user when a topic is moved to a category that does not allow for votes. This doesn’t seem ideal to me, but I’m not sure if it’s the intended behaviour or not. The only alternative I can think of is that if a topic is moved to a category that does not allow voting, all votes should be removed from the topic. There would be potential issues with this approach as well.
If you know of any specific ways of reproducing the edge cases, let us know. I can look into this some more next week.