Advice on a support bot for a technical support forum (Discourse AI vs Discourse Chatbot)

[Falco]

@37Rb

The Discourse AI plugin doesn’t seem to do this (it showed no embeddings activity on the OpenAI dashboard) but it still finds posts on our forum. If it doesn’t generate embeddings to search semantically, how does the Discourse AI plugin search the forum? Does it use regular text search? Does it compute it’s own embeddings?

Our AIBot will use hybrid search, combining both keyword and semantic search. If you didn’t enable embeddings properly, it will only use keyword search. To enable embeddings follow Enable Related Topics. After that it will slowly backfill in the background using the choosen model / provider.

[EricGT]

Any news/updates on how it is working?

[37Rb (Ryan)]

We decided to go with Discourse Chatbot because it has some features we need and Robert is helping us build a few additional features to get us all the way there. I’ll post a link when it’s up and running in case you want to try it but it will probably take at least a month.

[EricGT]

@37Rb

I’ll post a link when it’s up and running in case you want to try it but it will probably take at least a month.

Great to hear about that for your forum!

Thanks for the update.

[EricGT]

Maybe RLHF is on the way out.

Weak-to-strong generalization

[EricGT]

@EricGT

To improve the ability of the bot to help users it should perform a series of questions to collect information to better understand the problem similar to the way a medical doctor will for a diagnosis and prognosis.

FYI

Not suggesting this be added but useful for ideas in the future.

“DiagGPT: An LLM-based Chatbot with Automatic Topic Management for Task-Oriented Dialogue” by Lang Cao (pdf )

For those that recognize the info it was also posted here.

[EricGT]

“RAG vs Fine-tuning: pipelines, tradeoffs, and a case study on agriculture” (pdf) by Aman Gupta, Anup Shirgaonkar, Angels Balaguer, Bruno Silva, Daniel Holstein, Dawei Li, Jennifer Marsman, Leonardo Nunes, Mahsa Rouzbahman, Morris Sharp, Nick Mecklenburg, Rafael Padilha, Ranveer Chandra, Renato Cunha, Roberto Estevão, Ryan Tsang, Sara Malvar, Swati Sharma, Todd Hendry, Vijay Aski, Vijetha Vijayendran, Vinamra Benara

[37Rb (Ryan)]

It took longer than expected but we finally have our support chatbot live, albeit in “beta testing mode”. Thanks to everyone who offered advice along the way! And special thanks to @merefield for writing the Chatbot plugin! If you’d like to try it, here’s how.

Introducing Dexter the Surety Chatbot

[EricGT]

Had a reason to revisit this post.

Care to share any updates about the bot?

Thanks.

[merefield (Robert)]

@37Rb wrote a nice guide here:

Building a technical support chatbot

Adding an AI chatbot to Discourse is easy (thanks to 2 great plugins). Adding a chatbot that performs technical support is a lot harder! This post shares our experience setting up a technical support chatbot for support.suretyhome.com - what we wanted, problems we had, how we solved them, and where we’ll go from here. Our support team is only available during regular business hours but customers want help around the clock. We aren’t trying to replace the support team. Our goal is to augment the…

this is linked from Chatbot Topic

[37Rb (Ryan)]

In addition to the guide above, one of the most apparent things is that the bot’s performance depends on how much the user knows. If they ask good questions they’re more likely to get good answers. If they use the wrong terms or describe things poorly in their question, the bot will happily misunderstand and confidently give them an answer anyway. It’s working better for power users than it is for novices. With less knowledgable users, us humans are better at picking up on nuances in their question and understanding what they really mean to ask. Feel free to give it a try, I’d love your feedback.