Struggling with pagination within search/query.json

支持
1

I’m trying to grab the posts within a date interval from our discourse server using the API. I’m struggling quite a lot with pagination.

Firstly I tried using search.json but that limits us to 50 results:
https://meta.discourse.org/search.json?q=after:2016-01-01%20before:2017-01-01

I tried to use the paginated query instead (but page 2 and page 3 end up the same for some queries):
https://meta.discourse.org/search/query.json?term=after:2016-01-01%20before:2017-01-01
https://meta.discourse.org/search/query.json?term=after:2016-01-01%20before:2017-01-01?page=2
https://meta.discourse.org/search/query.json?term=after:2016-01-01%20before:2017-01-01?page=3

That’s odd because simpler queries appear to paginate correctly:
https://meta.discourse.org/search/query.json?term=API
https://meta.discourse.org/search/query.json?term=API?page=2
https://meta.discourse.org/search/query.json?term=API?page=3
https://meta.discourse.org/search/query.json?term=API?page=51 (more_pages becomes null here, although we can fetch higher pages)

Is is possible to combine advanced search and pagination?

2

Second (3rd, 4th, etc.,) query string should look like below. & symbol instead of ?

https://meta.discourse.org/search/query.json?term=after:2016-01-01%20before:2017-01-01&page=2
https://meta.discourse.org/search/query.json?term=API&page=2

not like this

https://meta.discourse.org/search/query.json?term=after:2016-01-01%20before:2017-01-01?page=2
https://meta.discourse.org/search/query.json?term=API?page=2

3

I don’t know if it’ll help, but I wrote a little ruby program that will download all of a topic or category. (It doesn’t pay attention to dates, though.)

4

Oops. Getting the links wrong is what made me believe discourse support pagination using page=X in the first place (my own bad conversion from curl to a web browser).

Perhaps I should have ask a more general question: If search/query.json has a grouped_search_result with more_posts set true, how do I get to see the additional posts?

6

I know this is quite late, but yes we certainly support pagination now, search for something with lots of results and then use:

How to reverse engineer the Discourse API

To find all the endpoints.

7

我正在尝试了解如何使用 Discourse API。
在下面的 URL 中，我猜测 term 指的是时间段或持续时间。这是基于 term=after:2016-01-01%20before:2017-01-01 等参数得出的。但我不清楚 term=API 指的是什么。有人能说明一下吗？

8

实际上有两个搜索端点可以使用：/search.json?q= 和 /search/query?term=。

term 仅指搜索词，并不表示时间段或持续时间。

若要理解搜索 API，最好参考 https://meta.discourse.org/t/how-to-reverse-engineer-the-discourse-api/20576，并通过 API 执行您打算进行的搜索操作，观察界面如何发出相同的 API 请求。

9

你好 @blake，
感谢你的回复。

实际上，我想获取整个页面的内容（在我的情况下，是某个分类下的主题列表，参见 Obtaining a list of topics from a category Discourse 只是在你滚动时动态加载页面内容。据我所知，没有选项可以一次性显示整个页面。请问有什么方法可以实现吗？

10

没有单个 API 调用可以获取类别中的所有主题。您需要发起多次 API 调用。这是因为一个类别可能包含超过 100 万个主题，这样的查询量巨大，可能会影响您站点的性能。通过逆向工程滚动机制，您可以观察 Discourse UI 如何发起 API 调用来加载更多主题，并在您的 API 中复现该行为。

根据您的具体目标，您还可以使用数据探索查询来获取某个类别中的主题数量，并针对该已保存的查询发起 API 调用。