How does Discourse efficiently generate the topic list page? (The "Latest" page)


(KajMagnus) #1

I’m trying to understand how Discourse efficently generates the topic list page, i.e. the one shown if one clicks the “Latest” button. It lists the most recently replied-to topics, and lists topics sorted by number-of-likes, and by number of posts, among other things. I have not found any database indexes that would make possible e.g. [efficient lookups of topics sorted by number-of-likes, in a certain category]. For example, there is no index on the column topics.like_count. Yet the Latest page quickly finds topics sorted in all possible manners.

How is Discourse able to efficiently generate the Latest page, restricted to certain categories only, sorted in all different manners that Discourse supports?

Hmm, I noticed that on this page: “So, you want to help out with Discourse” (Discourse Development Contribution Guidelines) there’s a todo item “Improve performance of the topic list page” — you don’t happen to be doing full table scans right now? If so, then I’m wondering if something like ElasticSearch could be the “right thing” when optimizing the Latest page. ElasticSearch is kind of built for that particular purpose, I’m thinking, i.e. faceted search: restrict to this category, find things, sort by these properties.

(Background: I’m asking this because I’m developing forum sofware, as a hobby project, and I like Discourse and I’m copying some ideas. However I hope this question would also be useful for contributors that are into working on Discoure, perhaps people that want to help out with the above-mentioned todo item.)

(I hope I posted this in the correct category?)

Best regards, KajMagnus


(Sam Saffron) #2

Efficient query strategies and indexes, very little magic. You can see the queries we generate in rack-mini-profiler in the top left corner if you run in dev mode.