An AI can be helpful when sifting through and analysing long topics.
Here is my current procedure model for using the data of a topic in the context of an AI language model:
- Download all discussion posts of a topic.
- Extraction of the relevant data.
- Upload the relevant data to an AI language model.
- Data-related queries to the AI language model.
Download of all discussion contributions to a topic:
- This is possible via the CLI programme ‘discourse-reader’.
- Required:
- One-off application for a user API key.
- Authorisation of the API key in the (forum) user settings.
One-off application for a user API key:
- This is possible via the CLI programme ‘discourse-user-api-key’.
- The exact workflow is described in the programme help.
The download of a topic leads to a complex JSON file:
{
"meta_data": ...,
"post_data": [ ... ]
}
Example:
# download topic from forum
discourse-reader -forum="meta.discourse.org" -topic=210780 -output="210780.json" -userapikey="107dc02b7fad3bc217b6f0c121fdc805"
Extraction of the relevant data:
- This is possible via the CLI programme ‘jq’.
- Changes to the JSON data are also possible.
Example:
# filter out unnecessary data
# meta data
jq '.meta_data | {title, created_by:.details.created_by.username, created_at, id, posts_count, last_posted_at}' 210780.json > 210780.compact.json
# post data
jq '.post_data[] | .post_stream.posts[] | {post_number, username, created_at, reply_to_post_number, post:.cooked, reactions, accepted_answer}' 210780.json >> 210780.compact.json
Upload the relevant data to an AI language model.
Data-related queries to the AI language model.
- A simple CLI programme will be developed shortly for this purpose.
- Initially, the browser interface of an AI language model can be used.
The programs mentioned can be found on GitHub.
And here is a usage example for topic 210780 from this forum:
https://meta.discourse.org/t/matrix-protocol-for-chat/210780/
AI - Google Gemini 2.0 (exp)
Prompt to Gemini:
Create a list with the number of posts per user.
Create a short summary of the discussion.
What is the outcome of the discussion?
What are the next steps?
Response from Gemini:
Posts per User:
- mattdm: 4
- david: 1
- sam: 4
- AquaL1te: 2
- seong: 1
- Stephen: 1
- satonotdead: 1
- user12: 1
- attj: 1
- Falco: 3
- halfshot: 2
- merefield: 2
- anarcat: 1
- dan: 1
- rokejulianlockhart: 3
- volanar: 4
- mcdanlj: 1
- jimkleiber: 2
- sunjam: 6
- dubwoc: 1
- davy: 2
- Lhc_fl: 1
- Thomas_Rother: 12
- thoka: 2
- mcwumbly: 4
- joshsimmons: 2
- hellekin: 5
- kOS: 1
Summary of the Discussion:
The discussion revolves around the idea of integrating the Matrix protocol with Discourse chat. The initial suggestion is to make Discourse chat rooms Matrix rooms, which would be more seamless than a bridge. Participants see the appeal of this, with many voicing strong support for Matrix as an open, federated standard. However, there are concerns raised about users being less likely to engage with the “slow lane” (topics) if using non-Discourse Matrix clients. Others point out that having chat and forums on different platforms is also a barrier. The discussion also touches on the potential for federating Discourse instances via Matrix, as well as the use of ActivityPub.
The development of a Matrix bridge plugin for Discourse chat is noted, and its capabilities are outlined. However, the lack of end-to-end encryption in Discourse chat is also discussed. The complexities of user mapping, proxies, and the overall effort required are highlighted as significant challenges. There’s consensus on the benefits of interoperability between Discourse chat and Matrix, especially given the increasing adoption of Matrix by large organizations.
Outcome of the Discussion:
While there’s a lot of enthusiasm and support for the idea of a Matrix integration with Discourse chat, the discussion highlights significant challenges including:
- User management: Mapping users between the two systems is complex.
- Technical difficulties: Implementing a robust bridge is a substantial undertaking and may require a specific skill set.
- Resource constraints: CDCK has not made this a priority, due to limited resources and has yet to see a compelling business case.
- Funding challenges: The opportunity to seek funding from the German Sovereign Fund has passed.
The existing bridge code is seen as a good starting point, but it’s considered to be in alpha stage, not well documented, and requiring further development. There is no clear commitment from CDCK to prioritize this integration, unless external funding is secured or customers prioritize the request and sponsor development. The discussion concludes with no clear path forward or commitment to implementation.
Next Steps:
- Finding a Skilled Developer: The biggest barrier is the need for a developer with expertise in both Matrix and Discourse.
- Customer Sponsorship: A business case to justify development has not been made; a customer may need to sponsor the effort.
- Engagement with Matrix Foundation: Engage more with the Matrix Foundation for assistance, technical guidance, and potential funding opportunities.
- Assessment of existing code: Review existing code to evaluate the needed development for a beta status and whether this is a maintainable approach.
- Community Initiative: Explore the possibility of the community developing a third-party plugin.
- Crowdfunding: Evaluate the potential for crowdfunding.
- OFFDEM discussion: Discuss more funding opportunities at the next OFFDEM.