Here’s a potentially viable outbound federation support from Discourse idea: Webmentions mediated by Brid.gy.
Flow overview:
Topic is created with a link in the first post - to begin with, detect either a single link in the first post, or URL-pasted-in-title
Optional: To avoid trivial spam, either wait for the first reply or enable a site setting
Probe the page to see if WebMention is supported
A WebMention comment is sent with the text “This page is being discussed on =SITE_DOMAIN=: “=TITLE=” =TOPIC_LINK= =CATEGORY_AND_TAGS=” (using site default_locale).
The site receives the WebMention (potentially delegating to fed.brid.gy, applying anti spam, etc) and eventually displays that as a comment.
The actor (comment “author”) would take either the high-quality site logo or @system avatar, and the short name of the forum.
Just an aside: I didn’t mention it in the post but the protocol supports sending likes and reposts as first class citizens like replies, it would be nice if discourse implemented that too
Pavilion submitted an application to NLnet for 40,000 Euros to add federation support to Discourse (see below). It was rejected and we ended up applying for (and getting accepted) into DAPSI instead.
An update here. Pavilion and CDCK (Discourse.org) are discussing building an ActivityPub plugin for Discourse. After some discussion, we’ve landed on the spec below. Any comments or suggestions welcome before we finalise it. Note that
Support for incoming content (e.g. posts on Mastodon etc being imported into Discourse) is intentionally excluded. It will be possible to add this in a later version.
Support for following users (as opposed to categories) is also intentionally excluded. It would also be possible to add this in a later version.
Pavilion will be building the MVP plugin (I’ll be working on it), and CDCK will own and host it (i.e. it will be a CDCK plugin, not a Pavilion plugin).
Overview
An ActivityPub (AP) Plugin for Discourse.
The goal of the plugin is an MVP implementation of the ActivityPub, ActivityVocab and ActivityStreams specs in Discourse with one demonstrated integration of the MVP functionality for an AP compliant platform, namely Mastodon. As far as possible, the implementation will be built to support further support of the protocols, and any extensions.
This specification concerns enabling AP support on a per-category basis where only the first post in each topic in an AP-enabled category is federated (“first post only”).
Users
The plugin usage will be documented comprehensively on meta in a plugin topic.
Normal User
Subscribes to (aka “follows”) a federation-enabled Discourse category (FDC) on Mastodon (or any other fediverse service) using the category’s handle, e.g. @announcements@meta.discourse.org.
Sees an excerpt of the first post of all FDC topics (posted after they subscribe) in their Mastodon feed, each with a link back to the associated topic, e.g. “Discuss on our forum”.
Any action related to the post in Mastodon does not appear in Discourse.
Any action related to the post in Discourse does not appear in Mastodon.
Federated posts excerpts can be controlled by the post author using markup similar to that used to control topic excerpts, e.g. <div class="note">{text}</div>
Admin
Enables federation on a per-category basis using a category setting. Federation can only be enabled in categories visible to “everyone” on public instances.
Sets the category’s federated username via the category settings UI (cannot be changed).
Sets allow and deny lists of domains via site settings from which activity is automatically accepted or rejected.
Sets a “block list” of federated usernames to be part of a Block object(s) in the server outbox.
Sets the maximum character length of a federated note using a site setting i.e. activitypub_note_excerpt_maxlength.
Sets the text associated with the link included in the first post of a federated Discourse topic in “Customize > Text”.
Sets whether the link back (and text) to the forum is included in federated posts on a per-category basis.
Adds a key pair via site settings to sign federated content with.
Technical
The plugin will include comprehensive testing (unit / integration and js tests).
The category is an automated ActivityPub Actor within Discourse (as a Group ActorType). The federated preferredUsername is set at by the admin when enabling federation and stored in a category custom field. The federated name (aka display name) is the full_name of the category.
Users are Actors in the content federated by the category Actor. The federated preferredUsername is the user’s Discourse username at the time of federation. A user’s preferredUsername is stored in a user custom field in case the user changes their Discourse username. The federated name (aka display name) is the user’s Discourse name.
In general, ActivityPub objects will be associated with their equivalent Discourse objects using custom fields where appropriate (e.g. object or actor ids), and new tables where appropriate (e.g. inbox and outbox).
The primary ActivityPub Activity to implement is Follow and associated activities and models required to implement it, i.e. inbox, outbox and required associated Actions and Collections.
Discourse posts would be federated as ActivityStream Notes, with the content as HTML.
Authentication will be handled using HTTP Signatures, see here and here. All other Security Considerations in the ActivityPub spec will be addressed as appropriate.
Federation endpoints will be protected as appropriate for the specification, i.e. ensure redirect_to_login_if_required is used in controllers, add guardian for category everyone can see permission.
I’m really excited to see this news! Thank you! Also, glad you are starting with a simple initial implementation instead of trying to eat the proverbial elephant (not an oblique reference to Mastodon). I hesitate to suggest adding anything, but since you ask for suggestions, one that I hope is insignificantly more work:
What would you think about optionally also posting excerpts of replies, using inReplyTo to thread them? I think this would highlight the dialog occurring on Discourse and be more inviting; along with the “Discuss on our forum” I would expect that seeing the conversation would be more likely to bring more people in.
The issue with federating replies at this stage is that you may create expectations of engagement which won’t be fulfilled until you have proper support for it. It’s important to keep user expectations clear here. While some users might understand the limitations around those federated replies, others may not. The other issue is the one of trying to reduce the number of technical challenges to tackle at once.
That said, we’ve already contemplated a “whole topic” extension to the “first post only” specification you see above. I’ve shared some of the technical notes for that below to give you a sense of where this could be headed. I would stress that the “whole topic” extension is not part of the initial specification and is just a possibility at this stage.
Whole topic extension
Incoming Notes would be published as new topics or replies (using inReplyTo or Replies).
Discourse security and spam filters will be applied to incoming objects as best as possible using the Accept / Reject objects.
Note posts will be attributed to a staged user associated with the actor id via a custom field. This will require a non-email-based variation of the staged user concept as the user will not have a real email (a placeholder can be generated using the federated actor’s id and preferredUsername).
Associating staged ActivityPub users with standard Discourse users would be possible, but is not currently included in any part of this specification. This is a problem of “federated identity” and it’s associated solutions.
Federated posts would not support federated @mentions, or anything else in this vein outside of the core ActivityPub/Stream specs. Support for such features could be added in later versions, e.g. using Webfinger, following Mastodon.
The key in this first version is to focus on the minimum viable implementation. Once that foundation is laid, full(er) federation support, potentially along the lines of those “whole topic” notes, will be more feasible. I would stress though that the direction of the plugin will be something for CDCK to determine. Anything I mention here are just possible ways the foundation specification could be built on.
PS. I would also like to mention that I was approached by Daniël Klabbers core developer of Flarum in November last year who told me they were working on an ActivityPub extension. Don’t know what the status is, but it might be good if all projects in the Forums domain worked together to get as much interoperability going as possible in the space.
Wrt that interop I would also like to point out the Fediverse Enhancement Proposal process, the FEP on Codeberg:
Here e.g. Lemmy recently finalised a FEP for ActivityPub Groups. Discussions take place on the SocialHub at:
(FEP process is picking up steam with the growth of the Fediverse and currently best place to standardize protocol mechanisms)
Thanks @aschrijver, very useful! I’ll personally be quite focused on the MVP implementation spec’ed above, working closely with CDCK. In @mcdanlj’s pithy phrase it’s critical that that effort doesn’t attempt to “swallow the elephant”. We’re keen to hear any constructive feedback on the specification, particularly on the technical front.
I would add though that two other members of Pavilion@nmeyne (a veteran of the SSI community; particularly verifiable credentials), and @merefield are also working in this space, and they’re interested in some of the broader questions you’ve raised.
By way of illustration of real misunderstandings from an illusion of fidelity, one of the issues we’ve run into on Maker Forums, where we imported a lot of Google+ communities when Google+ went away, is that the import was sufficiently high fidelity that newcomers to the site don’t realize that they are trying to interact with other users who haven’t (yet) arrived at the site from Google+, and so we have a canned reply for letting them know that the person they are trying to talk to isn’t there to respond to them.
Thanks for providing the details on the “whole topic extension” idea.
In case this is added down the line… fwiw, I really like the approach that Pterotype for Wordpress offered, which is that any federated comments are relayed directly to moderation / group approval before being posted back to the original forum topic as a reply.
In the case of a forum this might be a task better suited to TL4 / TL3 types as opposed to staff.
Personally found this proves especially helpful in preventing off-topic / social media-style replies from overpowering the original discussion topic.
Yes, use of the review queue is something we’ve discussed for incoming content. As you note, this is something to be considered if and when we get there (not in the first version).
Just a note that the administrative matters concerning this work have been sorted, so I’m starting work on the specification next week. Any further technical feedback or notes on the specification are welcome.
It’ll take some months for this to come to fruition, so sit tight.
If a post is deleted in Discourse, can a Delete action be sent to followers?
On Maker Forums, we have enough “google points” that we get plenty of SEO spammers. We have relatively relaxed permissions for new users because a common form of new use is a frustrated person seeking site-relevant help, and it helps us help them faster. But this means that spammers tend to get a spam post in and then it’s relatively quickly flagged as spam, delisted, and then deleted.
I’d love for those federated posts to be deleted and the deletion published via Delete so that we clean up the spam from the federated caches. Is that possibly in scope?
Might be helpful in this case if there were a delay for federating, of some number of hours, to allow spam to be dealt with before federation. (Perhaps even apply the delay only to first posts of relatively new users?)
I am anticipating we’ll do this in the MVP for the kind of reasons you mention. I’ll discuss it further with the Discourse folks at the right time.
Useful suggestion. Federation will definitely be asynchronous. How the delay is set and handled is something we’ll need to address in some fashion in the MVP.
Specific suggestions like this for the MVP, particularly based on prior use cases, or technical domain expertise, are most welcome.
If it’s not significantly more work up front, also federating out post edits as Update Activities would be valuable. If it’s one of those things that can be contributed post-MVP, then a design that reasonably accommodates adding that later would be great.
For what it’s worth, I would set the delay low on my instance(s). I would not plan to use it to avoid federating spam posts, because it would reasonably likely be a signal that brings a mod from their fediverse feed into our Discourse to take action. I have chat_integration_delay_seconds set to 20 for quick typo edits, and would expect to do something similar here. At the very least, that setting sets a precedent for such a delay.
A brief update here! We’re a month into development and, broad brush, we now have the following working
People can follow categories with ActivityPub enabled
The first post of topics in ActivityPub-enabled categories are federated to followers
There’s a number of smaller pieces too, but not worth getting into at this stage. There will be at least one more month of development before we start internal testing.
