I am looking to semi-automate the process of turning a nominations thread into a table of nominations. I am using python (specifically the beautiful soup library) to parse the html. The nominations thread consists of a bunch of posts by users in which they link to a thread or topic that they like.…

You might want to use the API. Also, remember that almost all URLs will respond to a .json suffix. E.g. this thread https://meta.discourse.org/t/scrape-a-discourse-forum-with-python/98520.json

If access the raw markdown pages you won’t have to parse html and will have the URLs without accessing the counters. This topic is https://meta.discourse.org/raw/98520 . Edit: like others say, the API is really what you want. Here’s the scraper I wrote: GitHub - pfaffman/discourse-downloader: Downlo…

Thanks, all, your suggestions are very helpful! I particularly like that examples were offered in two languages. I’m going to go with the Python, but it’s cool to see the Ruby example as well.

Discourse フォーラムを Python でスクレイピングする

開発

mcmcclur (Mark McClure) 2018 年 10 月 2 日午後 2:08 4

I also recommend that you use the API, rather than the HTML. That’s how I set up the archival tool described in this topic.

トピック		返信	表示
Search by link Development search	7	122	2024 年 12 月 7 日
Converting Wiki Posts to HTML Support	3	602	2022 年 2 月 13 日
Grabbing Facebook reactions as posts Feature	7	1380	2015 年 1 月 26 日
Converting links from raw markdown to HTML Development	2	1879	2018 年 11 月 21 日
What method is used by Discourse to identify urls in post body Development	6	1170	2017 年 12 月 12 日

Discourse フォーラムを Python でスクレイピングする

関連トピック