I see there are a few interesting topics on integrating bots at various levels into Discourse, but I did not see any topics about integrating the big, large-memory model AI bots like GPT-3, BlenderBot etc. Has anyone thought about this? or even made any progress implementing the idea?
I am thinking of setting up another instance of Discourse with one of the big AI bots as an ordinary user on the system who participates like any other human user - but perhaps only replies when a post is directed at him specifically or where his handle is mentioned in a post.
Before I jump in again with a new instance, I thought I might get some feedback first this time . .
Excellent! My thought ATM is to be able to add lots of different AI apps on to the central entity as time goes on - sort of in the same way that the mammalian / human brain evolved with duplication and specialisation . .
Maybe I should create the Discourse instance to get started with?
Maybe let’s start with some technical design first. What kind of API do we connect to? Do we need a piece of software in between both systems or would a plugin that calls the GPT API suffice?
I did see the Hubot posts before - and I did think about experimenting with that option as a learning exercise before jumping in with something needing wheel invention - do you think that would useful?
Yes, absolutely. I’ll be away for around a week and after that I can spend some time playing around with things. I’m not a big Node fan though, I would prefer to do this in Python, Ruby or PHP. But let’s get some experience with this first.
I’ve been mostly off the internet for the past couple of months and came back to see lots of excitement around GPT-3. That got me thinking about how it could be applied to Discourse.
I’ve been fooling around with the idea of creating a “staff assistant” bot using OpenAI. I’m testing it with the text-davinci-003 model for now. I think the basic design will be a Discourse post event webhook pointing to a server with the Python OpenAI library installed on it.
The first task I’m working on is getting the bot to return Data Explorer queries from plain text descriptions. I haven’t tried fine tuning a model yet, but if information about the Discourse database structure is provided with the request’s prompt, it’s returning surprisingly good results.
It may turn out that using OpenAI for this kind of thing will be too expensive, or that the results aren’t accurate enough to use on a production site.
I’m just working on this for fun. The other obvious thing to look at is training a customer support bot against existing replies on a Discourse forum. This seems like a more difficult problem (both in terms of cost and accuracy) than just setting up a bot for staff users.
I have a different use case (the bot MUST remember previous conversations with our non-profit organisations’ members). I have made a bit of progress in the last week with various versions of BlenderBot - I will post more here where we have setup Discourse specifically for the project:
I was was going to ask the same, a ChatGPT bot so we can ask questions in a chat or a thread would be so useful. Here there is a tutorial running it in a VM
I’d be happy to be wrong about this, but my understanding is that ChatGPT only “remembers” the text of the most recent prompt you have sent it. You can test this in the OpenAI playground by starting a chat, then clicking the “View code” button after submitting your text. You’ll see that each time you submit more text, it’s appended to the prompt:
prompt="The following is a conversation with an AI assistant. The assistant is helpful, creative, clever, and very friendly.\n\nHuman: Just testing to see how the prompt gets set.\nAI: Hi there! It looks like you're trying to find out about how to configure the prompts. Is there anything else I can help you with today?\nHuman: If I use OpenAI's chat functionality on my own application, should I keep updating the prompt each time a user posts a new question? It's definitely a good idea to keep your prompts up-to-date, as it will help ensure your users are getting the most relevant information for each question they post. Depending on the size of your application and the complexity of your conversations, you may want to update your prompts regularly or develop a more sophisticated system to determine when updates are necessary. Let me know if there is anything else I can help you with."
This can continue until you reach the 4000 token (~3k words) limit that’s mentioned in the documentation that you linked to. Note that sending large prompts could get expensive. I think it’s currently $0.02 per 1000 tokens with the Davinci model.
Is my understanding is correct, an application or Discourse plugin that handles ChatGPT for Discourse would need to append each new question and reply to the prompt. It could do that until it reached 4k tokens (~3k words.) If that limit is exceeded, it seems that an error is returned from OpenAI. That error would need to be handled by the application that’s making the requests to OpenAI.
A possible workaround for the token limit would be to find a way of pulling key details out of a conversation and then only updating the prompt with those details. It might be possible to make a request on the user’s behalf to get this information. Something like:
Those details could then be save to the application that’s making the requests to OpenAI and used to restart the conversation at a later date.
You’ll get way more definitive answers about this here: https://community.openai.com/. It’s great to see they are using Discourse for their community!
when you interact with chatGPT and look at the network requests, you can see there is also a conversation_id provided that gets sent to the backend. I think the openai docs also mention this.
