The slow load of that topic may have more to do with it having 4542 posts 
That’s a very long topic.
Nevertheless, I’ve made some performance improvements. The first time you load the topic you won’t notice a difference, but subsequent loads may be faster, if the ratings plugin is having any effect on the load time.
https://github.com/angusmcleod/discourse-ratings/commit/55dd33988d3fd9b7b4368d7618d19481176de13f
Let me know how it goes.