Common event system for chatrooms - a specification

[david (David Taylor)]

One of the GSOC ideas on GitHub is creating a “Common Event System for Chatrooms”

Ideally we’d have a set of plugins for various chatrooms, like discourse-hipchat, discourse-slack, etc, that would all expose the same event system.

Then we could do code like DiscourseEvent.trigger(:notify_chat, “some chatroom text”) and have it work regardless of your chat software.

I like the sound of this idea, so this is an attempt at a more comprehensive description of how it could be implemented. Hopefully this will give a starting point for some discussion about what people would like to get out of a feature like this.

---

Edit 17/03/2017: I've added and removed some stuff based on comments below.

---

What we already have

https://meta.discourse.org/t/the-official-discourse-slack-plugin/51412

The Discourse-Slack plugin is a great way of getting real-time notifications of Discourse posts. It can post to ‘group chats’ in Slack, or post to individuals. Notifications can be setup for specific categories (tags coming soon), either for “new topics”, or “all replies”.

Common event system

The idea

Creating plugins which provide similar functionality to the slack plugin, but for other communication platforms, is totally possible right now. However, this will result in a massive amount of code duplication, as a lot of the logic will be the same between different chat systems. Therefore a ‘shared’ chat system (as a plugin), would be an efficient & maintainable way to expand to multiple platforms.

Scope

As an initial implementation, I wouldn’t expect functionality to go much further than the current functionality of the Slack plugin, however there is scope to add more functionality such as actually making posts on Discourse from your chosen messaging platform, but that’s material for a future discussion.

Structure

The “Common event system” would take care of managing “subscriptions” (which categories/tags to subscribe to for a given “chat”). At its simplest level, it would maintain a table with rows like this. Yes, it would need to be more complicated than 3 text fields, but hopefully this illustrates the functionality.

| CHAT_SYSTEM |     CHAT_ID     |          SUBSCRIPTIONS          |
|-------------|-----------------|---------------------------------|
|  telegram   | @davidtaylorhq  |  c/meta,c/plugins,t/pr-welcome  |
|  facebook   | group_75462363  |             c/lounge            |
|   slack     |     #general    |        c/support, c/bugs        |

Each of the chat_systems would be an independent plugin, which deals with the platform-specific stuff. These plugins would receive information from the “common event system” using similar systems as the current event hooks. So you’d end up with something like:

DiscourseEvent.on(:notify_chat, :telegram) do |information|
    # Send a message to the specified telegram chat using the information provided
end

Permissions

Initially this would be admin-only, so no permission checks would be necessary

Authenticating

Initially this would be admin-only, so no authentication would be necessary

User Interface

The “common event system” should have its own pane in the admin interface, listing all current subscriptions (effectively each of the rows in the pseudo-table above). I imagine the UI to be very similar to the current slack plugin:

Providers could also implement something like “slash commands” for changing subscription settings, like the slack plugin does right now:

/discourse [watch|follow|mute|help|status] [category|all]

Possible providers

Each of these could be implemented in separate plugins, keeping things modular & maintainable:

Slack - the slack plugin already exists, making use of the api
Telegram - Mature REST bot API, which I have used in the past.
Facebook Messenger - Bot API available
Gitter - Can add things to ‘activity stream’ using the services library. @jafeth.diazc has more info here
HipChat (thread) - API available, see discourse-akismet-hipchat
Mattermost (thread) - API available

[others] - Hopefully having a central system for managing the complicated user/permissions logic would enable plugin developers to easily create plugins for any service with an API.

---

I'd love to hear people's thoughts on this spec, if there's anything that's not clear please let me know and I'll do my best to improve the description.

[mcwumbly (Dave McClure)]

I think subscriptions should have a type.

In the UI, rather than a column just for Category, there would be a column for Type with a dropdown to select Category or Tag. In the future, I think you could also choose Topic or User, for instance.

Depending on the type, the next column over would allow you to choose the elements of that type that you want to subscribe to. If you select Category for Type, then you can choose which categories. If you choose Tag for the type, then you can select which tags.

This avoids ambiguity about whether these choices are unions or intersections and allows for the feature to grow more naturally to meet future needs.

[sam (Sam Saffron)]

I have huge mixed feelings here, it feels like starting at step 100 instead of step 0

Why does core need to worry about this stuff?

Isn’t a simpler way of going about this simply renaming “slack plugin” to “chat integration plugin” and adding extra providers for other chat platforms?

[Falco]

Also I’m not sure about making this available to every user vs only admins.

And about permissions you should choose one user for each subscription so it’s very clear what he can read. You can create multiple bot accounts and anchor every subscription in only one account.

[eviltrout (Robin Ward)]

Thanks for the effort @david but I agree with @sam it’s above and beyond what I had in mind. When in doubt I like to keep things simple and build up

1. This should definitely be a plugin, as not every discourse wants to integrate with a chat system. Something like discourse-chat for core support, and then discourse-chat-slack, discourse-chat-hipchat for adapters.

2. I also agree with @falco that it should begin with only supporting admins. I don’t have a problem with an admin entering an API on an admin page to configure it either – fancy configuration by visiting a page would obviously be better but let’s start simple

3. The core plugin would attach to existing events in the site and expose others, like: DiscourseEvent.trigger(:notify_chat, msg) where msg would be an object full of things we can expose to chat providers.

4. Providers would then listen for that and do the work necessary to publish.

[david (David Taylor)]

@sam

为什么核心开发需要担心这类问题？

@eviltrout

类似 discourse-chat 这样的核心支持方案，然后配合 discourse-chat-slack、discourse-chat-hipchat 等适配器。

没错，我认为这很可能适合作为插件实现，而那种命名体系听起来也很理想。

@sam

难道更简单的做法不是直接将“Slack 插件”重命名为“聊天集成插件”，并为其他聊天平台添加额外的提供者吗？

这基本上就是我考虑的方向，但与其将所有功能塞进一个插件，我认为采用更模块化的方式会更好。任何工作的起点毫无疑问都应该是当前的 Slack 插件。

@Falco

另外，我不确定该让所有用户都能使用此功能，还是仅限管理员。

我想这完全取决于你希望聊天集成走向何方。它是供论坛运营者用于管理/ Moderation 的工具，还是供成员随时了解最新动态的方式？

以 meta 为例，如果 @team 希望在他们的 Slack 聊天中收到新的 Contribute > Bug 帖子通知，他们可以很轻松地实现。然而，如果我（作为普通用户）希望收到关于新插件的 Telegram 通知，目前却没有任何方法。

当然，一开始仅面向管理员提供该功能是一个合理的路线图，但我真的认为这对许多用户都很有用——在我管理的社区中肯定会被广泛使用。

@eviltrout

核心插件将绑定到站点现有的事件，并暴露其他事件，例如：DiscourseEvent.trigger(:notify_chat, msg)，其中 msg 是一个包含可暴露给聊天提供者的各种信息的对象。

我还没有完全理解这一点。如果你并不打算在通用事件系统中引入某种“订阅”模型，那么为什么还需要一个单独的“通知聊天”事件呢？毕竟聊天插件完全可以轻松地绑定到 :post_created 事件。

---

[quote="eviltrout, post:6, topic:59245"] 这超出了我原本的想法。在拿不准的时候，我倾向于保持简单，逐步构建。 [/quote] 我明白，从简单开始确实更好。我某种程度上一直把 discourse-slack 插件视为起点，这也可能是我考虑得过于复杂的原因。

归根结底，我认为关键在于：这个功能是为谁设计的？它的用例是什么？我一直将其视为用户接收更新的另一种方式，而你认为它更像是一个管理/ Moderation 工具？

[eviltrout (Robin Ward)]

@david

I don’t fully understand this. If you’re not envisioning any kind of “subscription” model in the common event system, then why is there a need for a separate “notify chat” event, when chat plugins can quite easily attach to a :post_created event?

I assume that if today you started work on a second chat adapter, you’d end up copying and pasting a bunch of code from the slack one. That’s the stuff that should live in the core plugin! In particular, the admin page the filter rules. The core plugin would watch post_created, check those filters, then trigger notify_chat for delivery. Any adapters that receive notify_chat would just send it, and not worry about filters themselves.

@david

Ultimately what I think it all comes down to is who is this feature for, and what’s its use case? I’ve been thinking of it as another way for users to receive updates, whereas I think you’re thinking about it as an admin/mod tool?

Not exactly – an admin has to set it up, but it can be used by public chatrooms. For example, if you are in the emberjs slack, you might want to see when new posts are made on the emberjs discussion board. That has to be set up by an admin, but consumed by regular users.

Once the core product is built we can look into layering user level messages on top, but like we said earlier, that’s a whole bunch of work that depends on other stuff working nicely first.

[david (David Taylor)]

Thanks for the feedback. I’ve trimmed down my “spec” in the first post, so that it’s a more realistic starting point: admin only, with no fancy permission checks or authentication.

Is that more in line with what you’re thinking of?

[eviltrout (Robin Ward)]

Looks a lot better to me thanks! From there we can build on many more features.

[sam (Sam Saffron)]

Keep in mind, discourse plugins can source gems, so all the central logic can live in a shared gem

It would also be a fairly interesting excersize sorting out workflows, testing and so on

[ara4n (Matthew Hodgson)]

There seems to be huge overlap between this idea and Matrix.org (also a GSoC org), which provides an existing open standard for interoperable events between chat systems, and much else, including bridges to slack, irc, discord, telegram, etc. Perhaps an alternative angle might be to write a discourse<->matrix bridge?

[david (David Taylor)]

Matrix.org certainly looks like a cool project, but I think I’d view it as just another chat provider that could be hooked into the ‘common event system’. I only had a brief read of their website, so please correct me if I’ve got any of this wrong.

While Matrix does aim to bridge loads of different services, it looks to me like most of those are still in very early development (this list shows most integrations in alpha or early beta). The protocol is currently text-only, and requires a client connecting to some kind of ‘home server’.

Using Matrix as a way to get messages to Slack/Telegram/Gitter etc. will introduce another point of failure - in my opinion, using the native APIs for these services will provide a better user experience, especially when it comes to things like ‘prettifying’ the messages.

[ara4n (Matthew Hodgson)]

Yup, there are a few misconceptions here (probably because the matrix.org website is overdue an update, which should be going live next week):

• Matrix doesn’t aim to be yet another chat provider, but instead decentralised glue between existing chat providers. It’s aiming to fulfil precisely the use case that you’re talking about here.
• Yes, many bridges are alpha, but the ones in beta are generally very usable - we’re just very conservative about declaring stuff production-grade. Particularly the IRC/Slack/Gitter ones are robust; i believe around 5% of the connections to Freenode for instance are currently from Matrix.
• The protocol is not remotely text only; the point is that it supports freeform JSON data - folks sync anything from IMs, HTML, Markdown, WebRTC setup, MIDI, IOT data, VR world data etc over it.
• Yes, the architecture is that you’d go discourse<->Matrix homeserver<->rest-of-matrix (both servers & clients). In future you could implement add matrix server functionality into your server itself - e.g. GitHub - matrix-org/gomatrixserverlib: Go library for matrix federation. · GitHub is a golang matrix server library we’re currently working on. However, in the short term we’ve never had any problems with folks having to run a separate matrix server, especially during dev.
• Yes, technically the Matrix indirection would introduce another point of failure. However, I think you’re wrong in saying that somehow using the native APIs will provide a better user experience - all you’re doing here is shifting the impedance mismatch to be discourse<->remote_network rather than matrix<->remote_network. Given Matrix is deliberately built to be a superset of the full expressiveness of the remote network and avoid impedance mismatches, I might argue that you would be using the right tool for the job rather than reinventing the wheel…

Meanwhile, Matrix would also give you minor useful things such as better-than-Signal end-to-end encryption, VoIP, conferencing, and support of the whole wider Matrix ecosystem & community (200+ developers).

Whatever, we’d like to get a discourse<->matrix bridge running asap, whether it’s done as a one-of-many transport in your own common event system, or by just having discourse take the easier route and leverage matrix

edit: my FOSDEM main-track talk might give a bit more context on the current state of the Matrix ecosystem: FOSDEM 2017 - Encrypting Matrix