In a discussion titled ‘Data mining a Discourse site for links,’ user Jeffrey_L_Frusha proposes a plugin to automatically archive and download all links from forum posts for easier categorization and library creation, rather than manually extracting them. codinghorror suggests downloading personal posts and using regex or a custom program to extract links, noting that broader data mining is beyond the scope of existing tools. Mittineague suggests exploring the Data Explorer plugin to query link data. Jeffrey_L_Frusha clarifies that the goal is to extract links from all posts, not just personal ones. riking provides a SQL query via Data Explorer to list all outbound links with their frequencies, noting users can download results as CSV. Jeffrey_L_Frusha assumes this requires admin access and asks kensims to test it. riking adds that filtering out internal links (e.g., to /smp) may be useful. Jeffrey_L_Frusha asks if this could become a built-in feature or plugin. codinghorror responds definitively: ‘No, it cannot.’ The discussion concludes with no official support for such a feature, though manual or admin-assisted methods via Data Explorer are viable alternatives.
Could a plug-in, like the feature where I can archive and download my posts, be adapted to archive and download the links from all posts in a Discourse forum, rather than reading all posts and digging each one out, manually?
This would allow someone to categorize and create a library of those links, for a ‘wiki’…
The advanced search ‘Google’ only brought up some 752 results (which seems low), with:
site:discuss.seasteading.org http
and 732 results with:
site:discuss.seasteading.org www
The question was brought up by another forum member. I’m just another forum user, not a mod, or anything.
If I wanted mine and mine alone, I could do that, however, the goal is to get all, or as many as possible into one download, to be categorized as a library of sorts.
{"query":{"id":10,"sql":"SELECT url, count(*)\nFROM topic_links\nGROUP BY url\nORDER BY count DESC","name":"All outbound links","description":"Use the \"Download Results\" button to get it as a file.","param_info":[]}}
Import that, Run Query, then Click the “Download Results CSV” button.
I assume that’s something only admin can do. Thank you.
@kensims Would you be so kind as to try this? KatOn Tri would certainly like a copy, as would I. As I’m up to it, I’ll try to go through and categorize, so the wiki can be updated.
Could this be done as a feature? Maybe a plug-in capable of archiving and downloading the links from a topic, or category, similar to the way I can archive and download my posts?