Preloading custom data and fields in the topic list

Development

AI-generated summary

The discussion revolves around preloading custom data and fields in the topic list for Discourse plugins to avoid N+1 queries. sam explains two mechanisms: TopicList.preloaded_custom_fields for loading custom fields and TopicList.on_preload for bulk loading additional data. codinghorror mentions potential N+1 issues in the Feature Voting plugin, which joebuhlig clarifies have largely been resolved. Falco highlights the benefits for plugins like signatures, and sam expresses openness to extending the feature to the post stream. angus provides a link to a 2025 update on the topic.

1

Certain plugins need the ability to “preload” data in the topic list.

Some specific examples:

  • Discourse solved adds an icon on the topic list when a topic is solved
  • Discourse assign will display the username the topic is assigned to on the topic list

To achieve that we have 2 mechanisms:

Preloading a custom field

If you have any custom fields you need for rendering of the topic list use:

TopicList.preloaded_custom_fields << "my_custom_field"

This will force the system to ensure that the Topic object model has the custom field loaded when serializing.

If you do not do that and try to access a custom field an error will be raised.

###Preloading custom data

In some cases you may need to preload custom data that goes beyond a custom field. To do so:

TopicList.on_preload do |topics, topic_list|
    # your custom code to load up all topic data in bulk goes here
end

This is useful when you need to reach out and preload other models.

By carefully using both these methods you can ensure your plugin does not cause N+1 queries.

2

I believe Feature Voting plugin currently has n+1 queries and perhaps for the same reasons?

3

very possible, I only added the ability to preload custom data today.

4

This will make a lot of plugins use way less code, like my signatures plugin.

5

Does signature show up in topic list? scary …

6

Oh no, misread as post list. :smile:

Will there be an analogous feature to the post stream?

7

I would be happy to add an analogous feature, makes sense there as well.

8

It used to. I think I’ve cleaned most of those up. I don’t know of any others.

https://github.com/discourse/discourse-feature-voting/commit/135eaf8a5ec0e37b64adf10971384ee88abec580

I had to preload the custom fields due to errors if the data was blank. It was tricky to debug at first because I had existing data so I couldn’t replicate the errors reported. Then someone mentioned this process in Babble. Once I implemented it, the errors were gone. I never looked into it further than that.

9

Just for anyone reading this in 2025 I’ve written an updated version on the same topic