I am creating analytics about the usage of our forum, and I download information about the topics of each category. However, the request does not pull all the topics all the time. I only noticed this because it had previously pulled some topics that now disappeared from our latest report. Example: category: https://forum.aragon.org/c/proposals-for-main-dao/50 topic: https://forum.aragon.org/t/the-aragon-association-and-aragon-labs-core-teams-first-response-to-proposal-transfer-the-aragon-project-funds-to-an-aragon-dao-governed-by-delegated-ant/3528 The topic is there when I open the category page in the browser. The code I use: from dotenv import load_dotenv import os import requests load_dotenv() headers = { "Content-Type": "application/json", "Api-Username": os.environ["USERNAME"], "Api-Key": os.environ["API_KEY"], } method = "GET" url = "https://forum.aragon.org/c/proposals-for-main-dao/50.json" response = requests.request(method, url, headers=headers) topics = response.json()["topic_list"]["topics"] found_topics = list() for topic in topics: found_topics.append(topic["id"]) print(sorted(found_topics)) {topic: topic in found_topics for topic in [3519, 3528]} >>> [3075, 3224, 3299, 3312, 3449, 3472, 3474, 3526, 3554, 3555, 3557, 3583, 3586, 3592, 3613, 3624, 3626, 3659, 3674, 3677, 3680, 3694, 3695, 3699, 3702, 3710, 3716, 3720, 3725, 3737] {3519: False, 3528: False} Why is this happening, and how can I fix it?

[image] András Novoszáth: I am creating analytics about the usage of our forum, Then you should use the data explorer plugin. See Discourse Data Explorer and How to run Data Explorer queries with the Discourse API.

This is an official, stable & well-maintained plugin, what is preventing you from installing it? The UI uses a paging system to pull down each set of topics. That’s probably why you can’t see them all. Check the query your browser it’s sending for examples of each call (see network tab) However as @pfaffman rightly points out this is not the most appropriate way of getting data for analytics as it’s optimised for UI.

Category API request downloads all topics

Development

nocibambi (András Novoszáth) September 7, 2022, 10:17am 3

Thanks, but, currently, we cannot install that plugin. Can we do anything to fix the API?

Topic		Replies	Views
Use API to get topics for a period using js Development rest-api	2	907	February 28, 2023
API call for Featured Topics Product Support rest-api	1	95	April 3, 2026
Obtaining a list of topics from a category Product Support	3	2969	July 30, 2020
Is there any pagination in place for fetching category topics data? Product Support	1	142	January 6, 2025
Get all data of topics from categories such as images, links, contents etc General	0	772	December 7, 2022

Category API request downloads all topics

Related topics