Mark posts in topic as "read'

I tried to reverse engineer the site by sending POST requests to “https://{hostUrl}/topics/timings” with content-type, csrf token, and user-agent.

Here is what the body (json) looks like:

payload = {
  "topic_id": topic_id,
  "topic_time": post_count * 60000,
  "timings": timings
}

It returns a status code of 200 but the read history never changes at https://{hostUrl}/u/USERNAME/activity/read

I tried to look into this post but it wasn’t much help:

Here is a good amount of the code:

def get_csrf(session):
    r = session.get(f"https://{hostUrl}/session/csrf.json")

    if r.status_code != 200:
        raise RuntimeError("Failed to get CSRF")

    data = r.json()

    if "csrf" not in data:
        raise RuntimeError("No CSRF in response")

    return data["csrf"]

def load_topics(session, page):
    print(f"[Topics] Page {page}")

    r = session.get(
        f"https://{hostUrl}/latest.json?page={page}"
    )

    if r.status_code != 200:
        return []

    data = r.json()

    return [
        {
            "id": t["id"],
            "posts_count": t["posts_count"]
        }
        for t in data["topic_list"]["topics"]
    ]

def mark_post_as_read(session, topic_id, post_count):
    url = f"https://{hostUrl}/topics/timings"

    timings = {
        str(i): 60000
        for i in range(1, post_count + 1)
    }

    payload = {
        "topic_id": topic_id,
        "topic_time": post_count * 60000,
        "timings": timings
    }

    csrf = get_csrf(session)

    r = session.post(
        url,
        json=payload,
        headers={
            "X-CSRF-Token": csrf,
            "User-Agent": "Mozilla/5.0",
            "Content-Type": "application/json"
        }
    )

    print(f"[Read] {topic_id} → {r.status_code}")

    if r.status_code != 200:
        print(r.text[:300])


def tab_worker(session):
    page = 1

    while True:
        topics = load_topics(session, page)

        if not topics:
            break

        for t in topics:
            mark_post_as_read(
                session,
                t["id"],
                t["posts_count"]
            )

            time.sleep(0.4)

        page += 1

I’m bumping this because I still need the answer.

Thanks

What if you send this as a flat "timings[post_number]": duration? Does it work?

If I send a request containing that payload:

{
  "timings[1]": 10000,
  "topic_time": 10000,
  "topic_id": 6,
}

It updates the timing tables and the post is marked as read, and /activity/read is updated as well.

Why are you trying to do that? What is the purpose?

2 Likes

It doesn’t seem to exactly work.

Here is the code snippet which I modified.

def mark_post_as_read(session, topic_id, post_count):
    url = f"https://{hostUrl}/topics/timings"

    payload = {
        "topic_id": topic_id,
        "topic_time": post_count * 60000
    }

    for i in range(1, post_count):
        payload[f"timings[{i}]"] = 60000

    csrf = get_csrf(session)

    r = session.post(
        url,
        json=payload,
        headers={
            "X-CSRF-Token": csrf,
            "User-Agent": "Mozilla/5.0",
            "Content-Type": "application/json"
        }
    )

    print(f"[Read] {topic_id} → {r.status_code}")

    if r.status_code != 200:
        print(r.text[:300])

It still returns 200 but isn’t updated.

Thanks

The code looks fine, I’m not sure where the issue comes from. :confused:
My guts tell me we are just missing something obvious somewhere :sweat_smile:

This works:

def load_topics(session, page):
    print(f"[Topics] Loading page {page}")
    r = session.get(f"https://{hostUrl}/latest.json?page={page}")
    if r.status_code != 200:
        return []
    return [{"id": t["id"], "posts_count": t["posts_count"]} for t in r.json()["topic_list"]["topics"]]
    timings = {
        str(i): 60000
        for i in range(1, post_count + 1)
    }
    payload = {
        "topic_id": topic_id,
        "topic_time": post_count * 60000,
        "timings": timings 
    }    

    # Use json=payload to send as application/json
    r = session.post(url, json=payload, headers = {
        "X-CSRF-Token": csrf,
        "User-Agent": "Mozilla/5.0",
        "X-Requested-With": "XMLHttpRequest",
        "Content-Type": "application/json"
      }
    )
1 Like

Thank you so MUCH!!!

This finally works.

1 Like

Actually, one more thing.

This is pretty interesting!

  1. It DOES update at the post read history :white_check_mark:
  2. Posts read count increases :white_check_mark:

BUT:

  1. Topic read count doesn’t increase :cross_mark:

These are the 2 I’m talking about!

Pretty interesting.

I wouldn’t be surprised if those stats were updated by a regular sidekick job, for performance reasons.

What’s your use case to update timings with a script?

I can say with 100% certainty that this statement is true!

The posts read has updated several times but the topics read hasn’t. Are they on different intervals? It’s been ~20 hours since and the posts read count keeps increasing but the topics read doesn’t.

I just want to try to reverse engineer the endpoints! It’s cool.

I think I should wait a bit before coming back and seeing if the values have changed

You can trigger the sidekiq job manually in /sidekiq/scheduler if you find which one it is :slight_smile:

Perhaps it’s Jobs::DirectoryRefreshDaily.

Do you see the same in the user directory at /u?period=daily or weekly? There, you can see when the numbers were updated at the top.
image

I think the numbers for “today” are updated once per hour, while the other timespans are updated only once per day.

1 Like

@Canapin, I am not the owner of the website. If I can still trigger it from just being logged-in as a normal user, let me know the method to do so.

@Moin

The site I’m using has that disabled and will always return “A list of community members showing their activity will be shown here. For now the list is empty because your community is still brand new!”

In his case you can’t. Being a regular user isn’t ideal to reverse-engineer the API.
If you can, try a local dev install or a production install on a cheap VPS (a 3-4$ server is alright), since Discourse doesn’t require a hostname or SMTP anymore.

1 Like

Hey @Canapin , thanks for continuously helping.

How would would I do that? It currently says 22 topics read and 2.6M posts read