# Struggling with pagination within search/query.json

**URL:** https://meta.discourse.org/t/struggling-with-pagination-within-search-query-json/59558
**Category:** Support
**Created:** [March 20, 2017, 5:01pm UTC](https://meta.discourse.org/t/struggling-with-pagination-within-search-query-json/59558 "2017-03-20T17:01:03Z")
**Posts on this page:** 10
**Page:** 1

<div class="post-metadata">

### Author: ![danielt](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/danielt/32/121163_2.png) [@danielt](https://meta.discourse.org/u/danielt)
#### Post date: [March 20, 2017, 5:01pm UTC](https://meta.discourse.org/t/struggling-with-pagination-within-search-query-json/59558/1 "2017-03-20T17:01:03Z")

</div>

I’m trying to grab the posts within a date interval from our discourse server using the API. I’m struggling quite a lot with pagination.

Firstly I tried using search.json but that limits us to 50 results:  
[https://meta.discourse.org/search.json?q=after:2016-01-01%20before:2017-01-01](https://meta.discourse.org/search.json?q=after:2016-01-01%20before:2017-01-01)

I tried to use the paginated query instead (but page 2 and page 3 end up the same for some queries):  
[https://meta.discourse.org/search/query.json?term=after:2016-01-01%20before:2017-01-01](https://meta.discourse.org/search/query.json?term=after:2016-01-01%20before:2017-01-01)  
[https://meta.discourse.org/search/query.json?term=after:2016-01-01%20before:2017-01-01?page=2](https://meta.discourse.org/search/query.json?term=after:2016-01-01%20before:2017-01-01?page=2)  
[https://meta.discourse.org/search/query.json?term=after:2016-01-01%20before:2017-01-01?page=3](https://meta.discourse.org/search/query.json?term=after:2016-01-01%20before:2017-01-01?page=3)

That’s odd because simpler queries appear to paginate correctly:  
[https://meta.discourse.org/search/query.json?term=API](https://meta.discourse.org/search/query.json?term=API)  
[https://meta.discourse.org/search/query.json?term=API?page=2](https://meta.discourse.org/search/query.json?term=API?page=2)  
[https://meta.discourse.org/search/query.json?term=API?page=3](https://meta.discourse.org/search/query.json?term=API?page=3)  
[https://meta.discourse.org/search/query.json?term=API?page=51](https://meta.discourse.org/search/query.json?term=API?page=51) (more\_pages becomes null here, although we can fetch higher pages)

Is is possible to combine advanced search and pagination?

---

<div class="post-metadata">

### Author: ![vinothkannans](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/vinothkannans/32/86465_2.png) [@vinothkannans](https://meta.discourse.org/u/vinothkannans)
#### Post date: [March 20, 2017, 5:37pm UTC](https://meta.discourse.org/t/struggling-with-pagination-within-search-query-json/59558/2 "2017-03-20T17:37:08Z")

</div>

Second (3rd, 4th, etc.,) query string should look like below. `&` symbol instead of `?`

[https://meta.discourse.org/search/query.json?term=after:2016-01-01%20before:2017-01-01&page=2](https://meta.discourse.org/search/query.json?term=after:2016-01-01%20before:2017-01-01&page=2)  
[https://meta.discourse.org/search/query.json?term=API&page=2](https://meta.discourse.org/search/query.json?term=API&page=2)

not like this

[https://meta.discourse.org/search/query.json?term=after:2016-01-01%20before:2017-01-01?page=2](https://meta.discourse.org/search/query.json?term=after:2016-01-01%20before:2017-01-01?page=2)  
[https://meta.discourse.org/search/query.json?term=API?page=2](https://meta.discourse.org/search/query.json?term=API?page=2)

---

<div class="post-metadata">

### Author: ![pfaffman](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/pfaffman/32/120154_2.png) [@pfaffman](https://meta.discourse.org/u/pfaffman)
#### Post date: [March 20, 2017, 7:18pm UTC](https://meta.discourse.org/t/struggling-with-pagination-within-search-query-json/59558/3 "2017-03-20T19:18:48Z")

</div>

I don’t know if it’ll help, but I wrote a little [ruby program](https://github.com/pfaffman/discourse-downloader) that will download all of a topic or category. (It doesn’t pay attention to dates, though.)

---

<div class="post-metadata">

### Author: ![danielt](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/danielt/32/121163_2.png) [@danielt](https://meta.discourse.org/u/danielt)
#### Post date: [March 22, 2017, 3:07pm UTC](https://meta.discourse.org/t/struggling-with-pagination-within-search-query-json/59558/4 "2017-03-22T15:07:59Z")

</div>

Oops. Getting the links wrong is what made me believe discourse support pagination using page=X in the first place (my own bad conversion from curl to a web browser).

Perhaps I should have ask a more general question: If search/query.json has a grouped\_search\_result with more\_posts set true, how do I get to see the additional posts?

---

<div class="post-metadata">

### Author: ![system](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/system/32/443519_2.png) [@system](https://meta.discourse.org/u/system)
#### Post date: [May 17, 2019, 10:10pm UTC](https://meta.discourse.org/t/struggling-with-pagination-within-search-query-json/59558/5 "2019-05-17T22:10:29Z")

</div>



---

<div class="post-metadata">

### Author: ![sam](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/sam/32/102149_2.png) [@sam](https://meta.discourse.org/u/sam)
#### Post date: [May 20, 2019, 2:48am UTC](https://meta.discourse.org/t/struggling-with-pagination-within-search-query-json/59558/6 "2019-05-20T02:48:31Z")

</div>

I know this is quite late, but yes we certainly support pagination now, search for something with lots of results and then use:

[Reverse engineer the Discourse API](https://meta.discourse.org/t/how-to-reverse-engineer-the-discourse-api/20576)

To find all the endpoints.

---

<div class="post-metadata">

### Author: ![faheem](https://avatars.discourse-cdn.com/v4/letter/f/c0e974/32.png) [@faheem](https://meta.discourse.org/u/faheem)
#### Post date: [August 1, 2020, 8:21pm UTC](https://meta.discourse.org/t/struggling-with-pagination-within-search-query-json/59558/7 "2020-08-01T20:21:41Z")

</div>

I’m trying to understand how to use the Discourse API.  
In the URLs below, I’m guessing that `term` refers to a period or duration. This is based on `erm=after:2016-01-01%20before:2017-01-01` etc. But I’m unclear what `term=API` refers to. Can anyone clarify?

> [@vinothkannans](#):
>
> Second (3rd, 4th, etc.,) query string should look like below. `&` symbol instead of `?`
> 
> [https://meta.discourse.org/search/query.json?term=after:2016-01-01%20before:2017-01-01&page=2](https://meta.discourse.org/search/query.json?term=after:2016-01-01%20before:2017-01-01&page=2)  
> [https://meta.discourse.org/search/query.json?term=API&page=2](https://meta.discourse.org/search/query.json?term=API&page=2)

---

<div class="post-metadata">

### Author: ![blake](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/blake/32/157322_2.png) [@blake](https://meta.discourse.org/u/blake)
#### Post date: [August 3, 2020, 2:25pm UTC](https://meta.discourse.org/t/struggling-with-pagination-within-search-query-json/59558/8 "2020-08-03T14:25:43Z")

</div>

There are actually two search endpoints you can hit `/search.json?q=` and `/search/query?term=`

`term` just means search term and does not refer to period or duration.

For understanding the search API it would be best to follow [Reverse engineer the Discourse API](https://meta.discourse.org/t/how-to-reverse-engineer-the-discourse-api/20576) and perform the searches you are intending to do via the API and see how the UI makes the same API requests.

---

<div class="post-metadata">

### Author: ![faheem](https://avatars.discourse-cdn.com/v4/letter/f/c0e974/32.png) [@faheem](https://meta.discourse.org/u/faheem)
#### Post date: [August 3, 2020, 4:22pm UTC](https://meta.discourse.org/t/struggling-with-pagination-within-search-query-json/59558/9 "2020-08-03T16:22:19Z")

</div>

hi @blake,  
Thank you for the reply.

> [@blake](#):
>
> For understanding the search API it would be best to follow [How to reverse engineer the Discourse API](https://meta.discourse.org/t/how-to-reverse-engineer-the-discourse-api/20576) and perform the searches you are intending to do via the API and see how the UI makes the same API requests.

Actually, I’d like the grab the entire contents of a page (in my case, a list of topics in a category, see [Obtaining a list of topics from a category](https://meta.discourse.org/t/obtaining-a-list-of-topics-from-a-category/159337)). I don’t want to go page by page if it’s not necessary. And I don’t think reverse engineering this would work, because Discourse just expands the page as you scroll down. There is no option that I am aware of to show the whole page. Is there a way to do this?

---

<div class="post-metadata">

### Author: ![blake](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/blake/32/157322_2.png) [@blake](https://meta.discourse.org/u/blake)
#### Post date: [August 3, 2020, 5:03pm UTC](https://meta.discourse.org/t/struggling-with-pagination-within-search-query-json/59558/10 "2020-08-03T17:03:53Z")

</div>

There is no single API call that will fetch ALL of the topics in the category. You will have to make multiple api calls. This is because a category could have 1 million+ topics and that could be a huge query and affect the performance of your site. When you reverse engineer the scrolling you can see how the discourse UI makes the api calls for more topics and you can replicate that behavior in your api.

Depending on what you are trying to achieve you can also use the data explore query to get the number of topics in a category and make an api call to that saved query.
