I’m working on https://mentionme.app and I want to gather the latest content from publicly available Discourse forums. For that I’ve picked the /posts.json API endpoint.
I noticed that this endpoint returns a seemingly random number of elements. The documentation states that it will return the latest 50 posts, but upon closer inspection (e.g. https://meta.discourse.org/posts.json ) we see that rather than 50 latest posts, we see posts with IDs ranging from max-50 to max, and some are missing.
I assume that the fact that there are not 50 posts in that endpoint is an imperfection of the API - am I correct?
I assume that post IDs are assigned from a greater pool of IDs that include other things (user IDs?) and I shouldn’t worry about that.
In particular, for some values, the list is empty, e.g.: https://meta.discourse.org/posts.json?before=1234 - what shall I do if a list with no “before” parameter (“posts.json”) returns an empty list and I’m unable to learn the max post ID to get started with?
Some posts will be inside protected categories you can’t see, and other will be inside personal messages and not listed in the endpoint, which lists the last posts in topics you can read.
For example, right now this post here is the 621820 and the next one is 621818 by @gerhard. The missing one is 621819 which is a PM and that is why you can’t see it.
Thank you for the explanation, now it’s clear. I’m scanning several Discourse forums now, and thus far I’ve hit the issue in production once (posts.json with no “before” parameter returns an empty list). It’s not the end of the world, but something to be aware of.