Enabling AI Bot in Chat

[Leo_Wong]

@sam

@joffreyjaffeux had a prototype for streaming API in chat that could give us that. I would like us to also support this modality.

We would love to have ai bot in the chat function. our user uses Chat heavily .

[sam (Sam Saffron)]

Can you expand on how you wish it would work?

[Jagster (Jakke Lehtonen)]

My users would use it like

• my vet said dogs can get vitamin D from sunlight
• I don’t think so… everybody says we have to use nutraceuticals
• really? Hi @helpful-bot can dogs get vitamin D from the sun?

Something like that. Means I don’t need different personas. But for me that is not a deal breaker. I have that another chatbot in use at channels

[Leo_Wong]

Apologies for the ambiguity. I meant to say that we want our users to be able to summon the bot in the general chat channel by using @botname. Additionally, users within the eligible group should have the capability to directly message the bot. like the screen shots below;

[sam (Sam Saffron)]

I see, so trying to zero in here there are … with a bit more technical details.

1. @mentioning @gpt3.5-bot would select the first prioritized persona… that would usually be forum helper but you could change it.
2. We would also supply context (lets say 1000 tokens of previous chat messages) so it would be lined up. (is this configurable or not? configurable by persona or not?)
3. We clearly need to restrict the group of people who can @mention in chat, so we would need a site setting for it.
4. We would need to think about auditability, an easy minimum here would be to ship a data explorer query.
5. Given recent chat refactors, we can open “unlimited” new discussions with a bot, so we could design something there for sure.
6. If we allow @mentioning in chat… why not also allow it in forum posts to the same group? different group.

Given all of this… my V0 suggestion would be:

1. Add site setting ai_bot_chat_enabled_groups
2. Hard code 1000 tokens of context for now - choose first prioritized persona
3. Only do @mention support for V0

---

V1

1. Streaming and cancel support

---

V2

1. Add optional support for “Group DMs with ai bots”
2. Ensure we also have a nice UI to find all this stuff and minimize overhead of doing this.

---

V3

1. Optionally allow personas to also stage a dedicated user with a dedicated avatar. Then you can interact with multiple bots in chat. In this mode persona would have a user/model selected as well.

2. Also move some configuration to persona UI … which persona is enabled for chat?

---

V4

1. Think about limits - maximum token usage per day per user, stuff like that

---

V5

1. Think about also adding some of this into standard topics - (needs site setting, etc…) eg: @summary_bot summarize this topic up till now

---

Overall I like a lot of this, just need to figure out how we line this stuff into our roadmap.

[merefield (Robert)]

In the meantime, if you are self-hosted, you can always consider Discourse Chatbot, which has supported Chat from the very beginning (and indeed was the first AI Chatbot for Discourse).

[Falco]

@sam

Given all of this… my V0 suggestion would be:

1. Add site setting ai_bot_chat_enabled_groups
2. Hard code 1000 tokens of context for now - choose first prioritized persona
3. Only do @mention support for V0

This sounds like a very solid plan.

@sam

@mentioning @gpt3.5-bot would select the first prioritized persona… that would usually be forum helper but you could change it.

Maybe we start with a setting for a default chat persona, and eventually migrate to per-channel?

@sam

We clearly need to restrict the group of people who can @mention in chat, so we would need a site setting for it.

100%

@sam

If we allow @mentioning in chat… why not also allow it in forum posts to the same group? different group.

I think chat is a good place to start, then eventually port same behavior to topics.

[Saif (Saif Murtaza)]

@sam

Given all of this… my V0 suggestion would be:

1. Add site setting ai_bot_chat_enabled_groups
2. Hard code 1000 tokens of context for now - choose first prioritized persona
3. Only do @mention support for V0

I pretty much agree with your and @Falco’s thoughts here, the only thing to add would be to make sure it’s working with Chat Threads as well, if not V0 then V1/V2

(Optional) One additional thing that might be worth adding down the line is an option here to initiate a chat with the bot, my thinking is that for certain users might not even know they can do that!

[joffreyjaffeux]

Seems ok for a V0. I think having a better UI to create bots and be able to define allowed groups from this UI would be far better than a site setting in mid/long term.

[sam (Sam Saffron)]

Persona editor is how we will get there, for sure.

We already have all the bones for attaching a new user to a persona

[fbpbdmin (bcqa)]

AI bot in chat will be much better than current private messaging to AI…
hope could add this enhancement soon…

btw, seems gemini API usage will not be free soon… hmm…

[sam (Sam Saffron)]

@fbpbdmin

AI bot in chat will be much better than current private messaging to AI…

I have been experimenting with this, it comes with quite a few limitations given keeping track of history is hard, markdown is not as complete and more… It is not a slam dunk. Yet it is easy to start conversations but it not a complete replacement.

Can you expand expand exactly why you think it is much better?

Gemini 1.0 pro is not that great, I would avoid it anyway, if you are looking for a cheap LLM then Claude 3 Haiku is the way to go.