How are post IDs assigned? (and other /posts.json questions)

AI-generated summary

Michal_Mazurek is developing mentionme.app and using the /posts.json API to fetch recent public posts from Discourse forums. He noticed the endpoint doesn’t always return exactly 50 posts — instead, it returns posts with IDs near the current maximum, but some IDs are missing. He wondered if this was an API bug or if post IDs are shared with other entities like users. Falco clarified that missing IDs are typically due to posts in protected categories or private messages (PMs), which are excluded from the public feed — for example, post ID 621819 was a PM and thus not visible. Michal also noted that sometimes posts.json returns an empty list even without the before parameter, which can make it hard to discover the max post ID to begin pagination — though he considers it a rare edge case. He now understands the behavior and continues scanning multiple forums, remaining aware of this quirk Michal_Mazurek.

I’m working on https://mentionme.app and I want to gather the latest content from publicly available Discourse forums. For that I’ve picked the /posts.json API endpoint.

I noticed that this endpoint returns a seemingly random number of elements. The documentation states that it will return the latest 50 posts, but upon closer inspection (e.g. https://meta.discourse.org/posts.json ) we see that rather than 50 latest posts, we see posts with IDs ranging from max-50 to max, and some are missing.

I assume that the fact that there are not 50 posts in that endpoint is an imperfection of the API - am I correct?
I assume that post IDs are assigned from a greater pool of IDs that include other things (user IDs?) and I shouldn’t worry about that.

Could someone please explain away my worries?

In particular, for some values, the list is empty, e.g.: https://meta.discourse.org/posts.json?before=1234 - what shall I do if a list with no “before” parameter (“posts.json”) returns an empty list and I’m unable to learn the max post ID to get started with?

Some posts will be inside protected categories you can’t see, and other will be inside personal messages and not listed in the endpoint, which lists the last posts in topics you can read.

For example, right now this post here is the 621820 and the next one is 621818 by @gerhard. The missing one is 621819 which is a PM and that is why you can’t see it.

Thank you for the explanation, now it’s clear. I’m scanning several Discourse forums now, and thus far I’ve hit the issue in production once (posts.json with no “before” parameter returns an empty list). It’s not the end of the world, but something to be aware of.