Common event system for chatrooms - a specification

Feature
1

One of the GSOC ideas on GitHub is creating a “Common Event System for Chatrooms”

I like the sound of this idea, so this is an attempt at a more comprehensive description of how it could be implemented. Hopefully this will give a starting point for some discussion about what people would like to get out of a feature like this.

Edit 17/03/2017: I've added and removed some stuff based on comments below.

What we already have

https://meta.discourse.org/t/the-official-discourse-slack-plugin/51412

The Discourse-Slack plugin is a great way of getting real-time notifications of Discourse posts. It can post to ‘group chats’ in Slack, or post to individuals. Notifications can be setup for specific categories (tags coming soon), either for “new topics”, or “all replies”.

Common event system

The idea

Creating plugins which provide similar functionality to the slack plugin, but for other communication platforms, is totally possible right now. However, this will result in a massive amount of code duplication, as a lot of the logic will be the same between different chat systems. Therefore a ‘shared’ chat system (as a plugin), would be an efficient & maintainable way to expand to multiple platforms.

Scope

As an initial implementation, I wouldn’t expect functionality to go much further than the current functionality of the Slack plugin, however there is scope to add more functionality such as actually making posts on Discourse from your chosen messaging platform, but that’s material for a future discussion.

Structure

The “Common event system” would take care of managing “subscriptions” (which categories/tags to subscribe to for a given “chat”). At its simplest level, it would maintain a table with rows like this. Yes, it would need to be more complicated than 3 text fields, but hopefully this illustrates the functionality.

| CHAT_SYSTEM |     CHAT_ID     |          SUBSCRIPTIONS          |
|-------------|-----------------|---------------------------------|
|  telegram   | @davidtaylorhq  |  c/meta,c/plugins,t/pr-welcome  |
|  facebook   | group_75462363  |             c/lounge            |
|   slack     |     #general    |        c/support, c/bugs        |

Each of the chat_systems would be an independent plugin, which deals with the platform-specific stuff. These plugins would receive information from the “common event system” using similar systems as the current event hooks. So you’d end up with something like:

DiscourseEvent.on(:notify_chat, :telegram) do |information|
    # Send a message to the specified telegram chat using the information provided
end

Permissions

Initially this would be admin-only, so no permission checks would be necessary

Authenticating

Initially this would be admin-only, so no authentication would be necessary

User Interface

The “common event system” should have its own pane in the admin interface, listing all current subscriptions (effectively each of the rows in the pseudo-table above). I imagine the UI to be very similar to the current slack plugin:

Providers could also implement something like “slash commands” for changing subscription settings, like the slack plugin does right now:

/discourse [watch|follow|mute|help|status] [category|all]

Possible providers

Each of these could be implemented in separate plugins, keeping things modular & maintainable:

Slack - the slack plugin already exists, making use of the api
Telegram - Mature REST bot API, which I have used in the past.
Facebook Messenger - Bot API available
Gitter - Can add things to ‘activity stream’ using the services library. @jafeth.diazc has more info here
HipChat (thread) - API available, see discourse-akismet-hipchat
Mattermost (thread) - API available

[others] - Hopefully having a central system for managing the complicated user/permissions logic would enable plugin developers to easily create plugins for any service with an API.

I'd love to hear people's thoughts on this spec, if there's anything that's not clear please let me know and I'll do my best to improve the description.
9 Likes
2

I think subscriptions should have a type.

In the UI, rather than a column just for Category, there would be a column for Type with a dropdown to select Category or Tag. In the future, I think you could also choose Topic or User, for instance.

Depending on the type, the next column over would allow you to choose the elements of that type that you want to subscribe to. If you select Category for Type, then you can choose which categories. If you choose Tag for the type, then you can select which tags.

This avoids ambiguity about whether these choices are unions or intersections and allows for the feature to grow more naturally to meet future needs.

1 Like
4

I have huge mixed feelings here, it feels like starting at step 100 instead of step 0

Why does core need to worry about this stuff?

Isn’t a simpler way of going about this simply renaming “slack plugin” to “chat integration plugin” and adding extra providers for other chat platforms?

5 Likes
5

Also I’m not sure about making this available to every user vs only admins.

And about permissions you should choose one user for each subscription so it’s very clear what he can read. You can create multiple bot accounts and anchor every subscription in only one account.

4 Likes
6

Thanks for the effort @david but I agree with @sam it’s above and beyond what I had in mind. When in doubt I like to keep things simple and build up :slight_smile:

  1. This should definitely be a plugin, as not every discourse wants to integrate with a chat system. Something like discourse-chat for core support, and then discourse-chat-slack, discourse-chat-hipchat for adapters.

  2. I also agree with @falco that it should begin with only supporting admins. I don’t have a problem with an admin entering an API on an admin page to configure it either – fancy configuration by visiting a page would obviously be better but let’s start simple :slight_smile:

  3. The core plugin would attach to existing events in the site and expose others, like: DiscourseEvent.trigger(:notify_chat, msg) where msg would be an object full of things we can expose to chat providers.

  4. Providers would then listen for that and do the work necessary to publish.

6 Likes
7

Yep, I think this is probably plugin material, and that naming system sounds ideal.

That is effectively what I’m thinking of, but rather than just adding it all into one plugin I think a more modular approach would be better. The starting point of any work would most definitely be the current slack plugin.

I guess this entirely depends on the direction you want to take any chat integrations. Is it a tool for the people running the forum to use for admin/moderation, or is it a way for members to keep up-to-date with what’s going on?

Taking meta as an example, if @team wanted to get new bug posts in their slack chat, they could do that nice and easily. However, if I (as a regular user) wanted to get Telegram notifications about new plugins, there’s no way of doing it.

Sure, starting out with it only being for admins is a sensible roadmap, but I really think this would be useful for many users - it would certainly be used on the community I manage.

I don’t fully understand this. If you’re not envisioning any kind of “subscription” model in the common event system, then why is there a need for a separate “notify chat” event, when chat plugins can quite easily attach to a :post_created event?

[quote="eviltrout, post:6, topic:59245"] it's above and beyond what I had in mind. When in doubt I like to keep things simple and build up [/quote] I get this, starting simple is definitely better. I've kind of viewed the discourse-slack plugin as a starting point, which might be why I've gone too far.

Ultimately what I think it all comes down to is who is this feature for, and what’s its use case? I’ve been thinking of it as another way for users to receive updates, whereas I think you’re thinking about it as an admin/mod tool?

2 Likes
8

I assume that if today you started work on a second chat adapter, you’d end up copying and pasting a bunch of code from the slack one. That’s the stuff that should live in the core plugin! In particular, the admin page the filter rules. The core plugin would watch post_created, check those filters, then trigger notify_chat for delivery. Any adapters that receive notify_chat would just send it, and not worry about filters themselves.

Not exactly – an admin has to set it up, but it can be used by public chatrooms. For example, if you are in the emberjs slack, you might want to see when new posts are made on the emberjs discussion board. That has to be set up by an admin, but consumed by regular users.

Once the core product is built we can look into layering user level messages on top, but like we said earlier, that’s a whole bunch of work that depends on other stuff working nicely first.

5 Likes
9

Thanks for the feedback. I’ve trimmed down my “spec” in the first post, so that it’s a more realistic starting point: admin only, with no fancy permission checks or authentication.

Is that more in line with what you’re thinking of?

10

Looks a lot better to me thanks! From there we can build on many more features.

2 Likes
11

Keep in mind, discourse plugins can source gems, so all the central logic can live in a shared gem

It would also be a fairly interesting excersize sorting out workflows, testing and so on

1 Like
12

There seems to be huge overlap between this idea and Matrix.org (also a GSoC org), which provides an existing open standard for interoperable events between chat systems, and much else, including bridges to slack, irc, discord, telegram, etc. Perhaps an alternative angle might be to write a discourse<->matrix bridge?

13

Matrix.org certainly looks like a cool project, but I think I’d view it as just another chat provider that could be hooked into the ‘common event system’. I only had a brief read of their website, so please correct me if I’ve got any of this wrong.

While Matrix does aim to bridge loads of different services, it looks to me like most of those are still in very early development (this list shows most integrations in alpha or early beta). The protocol is currently text-only, and requires a client connecting to some kind of ‘home server’.

Using Matrix as a way to get messages to Slack/Telegram/Gitter etc. will introduce another point of failure - in my opinion, using the native APIs for these services will provide a better user experience, especially when it comes to things like ‘prettifying’ the messages.

4 Likes
14

Yup, there are a few misconceptions here (probably because the matrix.org website is overdue an update, which should be going live next week):

  • Matrix doesn’t aim to be yet another chat provider, but instead decentralised glue between existing chat providers. It’s aiming to fulfil precisely the use case that you’re talking about here.
  • Yes, many bridges are alpha, but the ones in beta are generally very usable - we’re just very conservative about declaring stuff production-grade. Particularly the IRC/Slack/Gitter ones are robust; i believe around 5% of the connections to Freenode for instance are currently from Matrix.
  • The protocol is not remotely text only; the point is that it supports freeform JSON data - folks sync anything from IMs, HTML, Markdown, WebRTC setup, MIDI, IOT data, VR world data etc over it.
  • Yes, the architecture is that you’d go discourse<->Matrix homeserver<->rest-of-matrix (both servers & clients). In future you could implement add matrix server functionality into your server itself - e.g. https://github.com/matrix-org/gomatrixserverlib is a golang matrix server library we’re currently working on. However, in the short term we’ve never had any problems with folks having to run a separate matrix server, especially during dev.
  • Yes, technically the Matrix indirection would introduce another point of failure. However, I think you’re wrong in saying that somehow using the native APIs will provide a better user experience - all you’re doing here is shifting the impedance mismatch to be discourse<->remote_network rather than matrix<->remote_network. Given Matrix is deliberately built to be a superset of the full expressiveness of the remote network and avoid impedance mismatches, I might argue that you would be using the right tool for the job rather than reinventing the wheel…

Meanwhile, Matrix would also give you minor useful things such as better-than-Signal end-to-end encryption, VoIP, conferencing, and support of the whole wider Matrix ecosystem & community (200+ developers).

Whatever, we’d like to get a discourse<->matrix bridge running asap, whether it’s done as a one-of-many transport in your own common event system, or by just having discourse take the easier route and leverage matrix :slight_smile:

edit: my FOSDEM main-track talk might give a bit more context on the current state of the Matrix ecosystem: https://fosdem.org/2017/schedule/event/encrypting_matrix/

3 Likes