An important thing to keep in mind here is that “chat” implementations generally broadcast the actual content to the subscribers.
In Discourse we have quite a complex pipeline which makes the naive implementation complex that leads to large amounts of traffic.
- User posts a reply
- All the users looking at the topic discover via broadcast that there is new content
- All users ask the server for post content (100 viewers = 100 requests)
- We pull images / optimize images
- All the users looking at the topic discover via broadcast that there is new content
- All users ask the server for post content (100 viewers = 100 requests)
(we have various optimisations, rate limits, retries and so on, but that is the gist)
These requests all have to run via our security pipeline to ensure the user has the rights to see the post and so on.
If the content was short-ish and we could somehow figure out how to do security in a more light-weight way for the “fast lane” then we could distribute the chatty messages via broadcast. This would lead to significantly better performance, we could probably handle 10000 users on a single low 2gb digital ocean droplet with that design.
Security is very complex. Caching is also complex cause of cache invalidation issues.
So, yes, we are absolutely thinking about this problem. But as it stands …
Lots of logged in viewers on one topic + lots of new content on one topic = expensive server bills.