How to get more posts from posts.json

Development

AI-generated summary

The discussion revolves around fetching more posts from the posts.json API endpoint. Initially, rmens tries to fetch the latest n posts using parameters like amount, limit, and posts, but none work.

swarnava suggests using the latest.json endpoint, which returns the latest topics, but Dalmiro_Granas notes that it’s different from posts.json which returns the latest posts.

Dalmiro_Granas and rmens discuss the limitations of posts.json and how to filter posts by date. Dalmiro_Granas finds a solution using the before parameter, which returns posts with an ID lower than the specified value.

The solution involves making multiple API calls to fetch more posts, and riking confirms this is the intended usage. rmens finds this solution helpful.

However, John_Lehmann notes that the before parameter is no longer documented, and asks for an alternative to fetch more than the last 200 posts.

1

Hi guys,

I’m trying to integrate Discourse in another front-end and would like to show the latest posts on the frontpage. Im querying the API with for example https://meta.discourse.org/posts.json but don’t get a consistent amount of posts back.

Is there any way to show the latest n posts. I tried https://meta.discourse.org/posts.json?amount=50 https://meta.discourse.org/posts.json?limit=50 and https://meta.discourse.org/posts.json?posts=50 but none of that is working.

Thanks!

2

There is something like that to fetch the latest posts: https://meta.discourse.org/latest.json

3

@swarnava what is the difference between /lastest.json and /posts.json?

Edit after actually testing instead of just asking :slight_smile: :

  • /latest.json returns the latest Topics
  • /posts.json returns the latest Posts

Is there a way to filter any of those by date? I’m also integrating Discourse with another tool, and I need to do “Give me all posts since (Now -7 days)”

4

It’s also still not clear to me how to get the latest n posts, not topics.

5

So the questions we have with @rmens are:

  • How far back in time/amount of posts does /posts.json get you?
  • Is this configurable in any way from the API call?

edit:

This thread looks promising: Latest posts API pagination - #4 by blake

I’ll post my findings tomorrow after testing this a bit more

6

/posts.json?before=X returns the first 50 posts with an ID lower than X. So if you want the get the lastest 75 posts for example, you can do:

  1. GET /posts.json to get the last 50
  2. GET /posts.json?before=x where x is the ID of the oldest post in the set of the previous GET call.
  3. Get the first 25 posts of this last GET set and you’ll have the 75 you wanted.

In my case I needed all the posts from the last 24hs, so my code looks like:

  1. GET /posts.json to get the last 50.
  2. If the oldest post in the set returned by the previous GET was created between the last 24hs, I do GET /posts.json?before=x where x is the ID of the oldest post in the set of the previous GET call.
  3. Repeat until the GET call returns at least 1 post that was created more than 24hs ago.

Hope this helps anyone :slight_smile:

7

You got it, that’s how that endpoint is supposed to be used :slight_smile: (although you can just do +/-50 for pages after the 2nd)

And, if you run the script again, and know what number you stopped/started at…

8

This helped a lot. Thanks guys!

9

I have been relying on the before= parameter, however after noticing my code running in an infinite loop, I see that the API docs no longer include this parameter.

Is there another way to accomplish this now – how can we get more than the last 200 posts?