Mark posts in topic as "read'

Development

AI-generated summary

User kittenwater sought assistance in reverse-engineering the Discourse API to mark posts as read via the /topics/timings endpoint. Their initial attempts using a nested JSON timings object returned a 200 status code but failed to update the read history. Canapin suggested altering the payload format, which kittenwater tested without success. Eventually, Canapin provided a working code snippet that successfully updated the individual post read counts and the user’s activity log.

However, kittenwater noted that while post counts updated, the “topics read” metric did not. Canapin hypothesized that this discrepancy was due to a delayed background job (Sidekiq) updating statistics for performance reasons. Moin added that these stats typically update hourly or daily depending on the view. Since kittenwater was a regular user on a production site without access to the admin scheduler, they could not manually trigger the job. Canapin recommended setting up a local or VPS instance to better reverse-engineer the API, as normal users lack the necessary permissions to force these updates immediately.

1

I tried to reverse engineer the site by sending POST requests to “https://{hostUrl}/topics/timings” with content-type, csrf token, and user-agent.

Here is what the body (json) looks like:

payload = {
  "topic_id": topic_id,
  "topic_time": post_count * 60000,
  "timings": timings
}

It returns a status code of 200 but the read history never changes at https://{hostUrl}/u/USERNAME/activity/read

I tried to look into this post but it wasn’t much help:

Here is a good amount of the code:

def get_csrf(session):
    r = session.get(f"https://{hostUrl}/session/csrf.json")

    if r.status_code != 200:
        raise RuntimeError("Failed to get CSRF")

    data = r.json()

    if "csrf" not in data:
        raise RuntimeError("No CSRF in response")

    return data["csrf"]

def load_topics(session, page):
    print(f"[Topics] Page {page}")

    r = session.get(
        f"https://{hostUrl}/latest.json?page={page}"
    )

    if r.status_code != 200:
        return []

    data = r.json()

    return [
        {
            "id": t["id"],
            "posts_count": t["posts_count"]
        }
        for t in data["topic_list"]["topics"]
    ]

def mark_post_as_read(session, topic_id, post_count):
    url = f"https://{hostUrl}/topics/timings"

    timings = {
        str(i): 60000
        for i in range(1, post_count + 1)
    }

    payload = {
        "topic_id": topic_id,
        "topic_time": post_count * 60000,
        "timings": timings
    }

    csrf = get_csrf(session)

    r = session.post(
        url,
        json=payload,
        headers={
            "X-CSRF-Token": csrf,
            "User-Agent": "Mozilla/5.0",
            "Content-Type": "application/json"
        }
    )

    print(f"[Read] {topic_id} → {r.status_code}")

    if r.status_code != 200:
        print(r.text[:300])


def tab_worker(session):
    page = 1

    while True:
        topics = load_topics(session, page)

        if not topics:
            break

        for t in topics:
            mark_post_as_read(
                session,
                t["id"],
                t["posts_count"]
            )

            time.sleep(0.4)

        page += 1
2

I’m bumping this because I still need the answer.

Thanks

3

What if you send this as a flat "timings[post_number]": duration? Does it work?

If I send a request containing that payload:

{
  "timings[1]": 10000,
  "topic_time": 10000,
  "topic_id": 6,
}

It updates the timing tables and the post is marked as read, and /activity/read is updated as well.

Why are you trying to do that? What is the purpose?

4

It doesn’t seem to exactly work.

Here is the code snippet which I modified.

def mark_post_as_read(session, topic_id, post_count):
    url = f"https://{hostUrl}/topics/timings"

    payload = {
        "topic_id": topic_id,
        "topic_time": post_count * 60000
    }

    for i in range(1, post_count):
        payload[f"timings[{i}]"] = 60000

    csrf = get_csrf(session)

    r = session.post(
        url,
        json=payload,
        headers={
            "X-CSRF-Token": csrf,
            "User-Agent": "Mozilla/5.0",
            "Content-Type": "application/json"
        }
    )

    print(f"[Read] {topic_id} → {r.status_code}")

    if r.status_code != 200:
        print(r.text[:300])

It still returns 200 but isn’t updated.

Thanks

5

The code looks fine, I’m not sure where the issue comes from. :confused:
My guts tell me we are just missing something obvious somewhere :sweat_smile:

7

This works:

def load_topics(session, page):
    print(f"[Topics] Loading page {page}")
    r = session.get(f"https://{hostUrl}/latest.json?page={page}")
    if r.status_code != 200:
        return []
    return [{"id": t["id"], "posts_count": t["posts_count"]} for t in r.json()["topic_list"]["topics"]]

    timings = {
        str(i): 60000
        for i in range(1, post_count + 1)
    }
    payload = {
        "topic_id": topic_id,
        "topic_time": post_count * 60000,
        "timings": timings 
    }    

    # Use json=payload to send as application/json
    r = session.post(url, json=payload, headers = {
        "X-CSRF-Token": csrf,
        "User-Agent": "Mozilla/5.0",
        "X-Requested-With": "XMLHttpRequest",
        "Content-Type": "application/json"
      }
    )
8

Thank you so MUCH!!!

This finally works.

9

Actually, one more thing.

This is pretty interesting!

  1. It DOES update at the post read history :white_check_mark:
  2. Posts read count increases :white_check_mark:

BUT:

  1. Topic read count doesn’t increase :cross_mark:

These are the 2 I’m talking about!

Screenshot 2026-02-05 at 11.12.14 PM
Screenshot 2026-02-05 at 11.12.14 PM596×124 6.16 KB

Pretty interesting.

10

I wouldn’t be surprised if those stats were updated by a regular sidekick job, for performance reasons.

What’s your use case to update timings with a script?

11

I can say with 100% certainty that this statement is true!

The posts read has updated several times but the topics read hasn’t. Are they on different intervals? It’s been ~20 hours since and the posts read count keeps increasing but the topics read doesn’t.

I just want to try to reverse engineer the endpoints! It’s cool.

I think I should wait a bit before coming back and seeing if the values have changed

12

You can trigger the sidekiq job manually in /sidekiq/scheduler if you find which one it is :slight_smile:

13

Perhaps it’s Jobs::DirectoryRefreshDaily.

14

Do you see the same in the user directory at /u?period=daily or weekly? There, you can see when the numbers were updated at the top.
image

I think the numbers for “today” are updated once per hour, while the other timespans are updated only once per day.

15

@Canapin, I am not the owner of the website. If I can still trigger it from just being logged-in as a normal user, let me know the method to do so.

@Moin

The site I’m using has that disabled and will always return “A list of community members showing their activity will be shown here. For now the list is empty because your community is still brand new!”

16

In his case you can’t. Being a regular user isn’t ideal to reverse-engineer the API.
If you can, try a local dev install or a production install on a cheap VPS (a 3-4$ server is alright), since Discourse doesn’t require a hostname or SMTP anymore.

17

Hey @Canapin , thanks for continuously helping.

How would would I do that? It currently says 22 topics read and 2.6M posts read