Paginating API search results

@mbajur I assume it is hard limited for performance reasons. Because search allows you to get multiple types of data (posts, topics) matching numerous arbitrary constraints (has tag, is in category, contains string), search results can be quite expensive for the back end to generate. Paginated results usually mean the data is lying around in memory, paginating search results would involve storing previous search results or regenerating preceding pages (getting page 50 would involve either having pages 1-49 laying around or regenerating them). Stopping after 50 matches have been found keeps fruitful queries from being horrendously expensive. It would be convenient if the API would provide a way for say a client with a master key to get more than 50 results for a search in some way and let the developer deal with whatever performance costs are incurred, but I think the idea is that you should be able to find some more efficient way to get your data.

The obvious way to get more results is to modify the source code and up the limit, maybe conditioned based on what API key is being used so your users don’t inadvertently bring down your server.

A better solution to your root goal though is to use Post Event Webhooks, located in Admin -> API -> Webhooks. Once configured, discourse will send a post request to whatever URL you specify with a body containing the post id and cooked post text. You can search this text for youtube.com, then do whatever you like including edit the post in discourse using the provided post id. Instructions for setting up webhooks are here. This will not work for links that have already been posted, but at least you would be notified of all incoming links going forward.

3 Likes