I am looking to semi-automate the process of turning a nominations thread into a table of nominations. I am using python (specifically the beautiful soup library) to parse the html. The nominations thread consists of a bunch of posts by users in which they link to a thread or topic that they like.…

You might want to use the API. Also, remember that almost all URLs will respond to a .json suffix. E.g. this thread https://meta.discourse.org/t/scrape-a-discourse-forum-with-python/98520.json

If access the raw markdown pages you won’t have to parse html and will have the URLs without accessing the counters. This topic is https://meta.discourse.org/raw/98520 . Edit: like others say, the API is really what you want. Here’s the scraper I wrote: GitHub - pfaffman/discourse-downloader: Downlo…

Thanks, all, your suggestions are very helpful! I particularly like that examples were offered in two languages. I’m going to go with the Python, but it’s cool to see the Ruby example as well.

Scraping di un forum Discourse con Python

Sviluppatore

mcmcclur (Mark McClure) 2 Ottobre 2018, 2:08pm 4

I also recommend that you use the API, rather than the HTML. That’s how I set up the archival tool described in this topic.

Argomento		Risposte	Visualizzazioni
Search by link Development search	7	95	Dicembre 7, 2024
Converting Wiki Posts to HTML Support	3	573	Febbraio 13, 2022
Grabbing Facebook reactions as posts Feature	8	1363	Dicembre 4, 2025
Converting links from raw markdown to HTML Development	2	1869	Novembre 21, 2018
What method is used by Discourse to identify urls in post body Development	6	1167	Dicembre 12, 2017

Scraping di un forum Discourse con Python

Argomenti correlati