Discourse Chatbot 🤖

[exlabs (Daniel)]

@merefield Thanks!

How on Earth did you debug that so quickly?

I’ll remove Zendesk… simple

[merefield (Robert)]

Bitter experience

[exlabs (Daniel)]

Hello all,

I have made an account with OpenAI now. Should I create an “Assistant” I am just wondering if there are any steps to follow to get the AI integrated with the plugin?

Regards,

[merefield (Robert)]

No, all you need is a token. You may need to credit your account with Open AI to get access to the more advanced models and higher rate limits.

All the API stuff is handled by the plugin.

(btw, fun fact - Chatbot explicitly does not support the Assistants API because it implements it’s own bespoke conversation and “inner thought” system utilising only the Chat Completions API and the Embeddings API. This is great because the Assistants API has a reputation for being profligate with tokens and costing you a lot of money - Chatbot is built to be frugal!)

[exlabs (Daniel)]

Wow it does a lot more than I realised.

Could you explain what is a token in the ChatGPT world? I am paying for ChatGPT preimum at the moment, its 20USD/month. What I was aiming at was allowing our Discourse uses to access the AI have been training, over on chatgpt.com

I hope this is making sense and the use case is clear!

[Jagster (Jakke Lehtonen)]

It is like money. You can think of tokens as words or parts of words. When you ask something, it is worth an amount X of tokens. When OpenAI processes your question, it uses more tokens. And lastly, when your answer is created, that costs an amount of tokens.

Depending on the model used and the conversation, you have to pay for that token use, sometimes more and sometimes less.

[Jagster (Jakke Lehtonen)]

Ability of using o1 has changed quite a lot my AI use (and thank… not God, but Robert’s coding skills I can limit its use only for me). But the o1 series doesn’t use a system prompt, so I have to add such guidance, as how to show math, etc., in my prompt.

It would be easier if we had a standard text to add automatically after the actual prompt, aka. post. Discourse has, kind of, that ability, but either I can’t use it or I’m trying to use it wrong. It should appear in every category and PM when the Chatbot is mentioned or is part of a PM. And only when o1 or o1-mini is used

The o1 series isn’t ready yet, so it may change at some point how it can use tools, etc. So in that way, I’m not asking for a new feature, but perhaps more discussing about options. And the o1 series won’t be a bulk solution ever, it is too specialized (and expensive), so the demand where a user just must remember to add those extra rules may be justified.

Off topic, but GPT models learned math when o1 was published.

[merefield (Robert)]

Yeah, we can’t be getting too much into customisation to make up for Open AI inconsistencies.

I strongly suspect tools and possibly even system prompt features will come as o1 exits preview.

I’m not convinced Chatbot is the place to implement a “snippet” system, though I remember one of the team creating that, let me find it and update the post …

[merefield (Robert)]

Here you go:

Insertable, centrally maintained text/markdown snippets

One possible option is to use a theme component. This goes in header.html <script type="text/discourse-plugin" version="0.8"> var disclaimer = settings.Disclaimer_text, disclaimer_selector = 'div[data-theme="disclaimer"]'; $.fn.disclaimer = function() { if (!this.length) { return; } else { this.each(function() { $(this).html(disclaimer); }); return this; } }; api.decorateCooked($elem => $elem.children(disclaimer_selector).disclaimer()); // create composer butto…

[exlabs (Daniel)]

Thanks @Jagster for the really clear info. on that. I think I was meaning something different, maybe? I am trying to integrate this plugin with OpenAI using this,

. I’m just trying to get some basic information on how to hook things up.

[Jagster (Jakke Lehtonen)]

That is just an API key you will generate to get access. Nothing more, your password we could say.

[merefield (Robert)]

Ah now I understand the confusion.

Yes there is an API token (or key, which you need to get from OpenAI) and there are text tokens that represent parts of words. The latter are processed by the LLM. The former is so you can be authenticated by the API.

Here’s quite a good guide to the terminology from The Verge:

AI is confusing — here’s your cheat sheet

[Amicable]

I’m running Llama locally and I’m so very close to getting this going on a local Discourse instance. Chatbot is chatting but I’m struggling to get embedding done. I have my benchmark user at Trust Level 1. The rake task rake chatbot:refresh_embeddings[1] is failing with an auth problem.

Faraday::UnauthorizedError: the server responded with status 401 (Faraday::UnauthorizedError)

Within the chatbot open ai token field, I pasted the URL that Llama gave me

Might you have any suggestions for how I should be authenticating properly?

[merefield (Robert)]

Great that you had a go!

@Amicable

Chatbot is chatting but I’m struggling to get embedding done.

Glad it’s chatting

I don’t know if it is possible to use Chatbot with embeddings locally.

I would accept a good PR to add support.

I found this:

Embedding models · Ollama Blog

So it might be possible with the right model and relevant additional code and settings in Chatbot. You would probably need a custom embeddings model setting which doesn’t yet exist.

There is this already though:

chatbot open ai embeddings model custom url

which was primarily implemented to help support azure but can be re-used in a similar way to how Chat Completions can point to local ollama service.

Appreciate this currently limits the bot’s usefulness locally.

@Amicable

Might you have any suggestions for how I should be authenticating properly?

This is the token sent to the endpoint. It is simply a text field. I very much doubt a URL here will work. There are custom chat completions endpoint URL settings further down:

e.g. chatbot open ai model custom url high trust

If you fork and make relevant additions that are in line with what already exists, leverages what is already available, but adds the minimum necessary to achieve local embeddings support with ollama, whilst keeping existing behaviour with Open AI and Azure intact, I would definitely review a PR with a view to merge.

[Amicable]

@merefield

I found this:

Embedding models · Ollama Blog

So it might be possible with the right model and relevant additional code and settings in Chatbot.

Coming soon
More features are coming to support workflows that involve embeddings:
…
OpenAI API Compatibility: support for the /v1/embeddings OpenAI-compatible endpoint

Given that soon is usually truly soon in this fast moving domain, maybe this is something worth waiting for in order to solve my use case?

Our instance is a fount of information going back 20 years but leakiness is also huge concern, hence the wish for local testing.

[Amicable]

I think I found an Ollama PR that seems to sort out the API endpoint compatibility . I need to get my head round this better but am I on the right track?

[merefield (Robert)]

well currently there’s no setting for custom embeddings model so that’s a showstopper.

when running locally you could try it by hard coding the model to the one you downloaded for ollama.

thinking about this more, the other issue you will face is that the embeddings table definition has dimensions set to open AI small model standard which is HUGE … perhaps you can think how you might overcome that …

[merefield (Robert)]

Chatbot 1.2. introduces AI powered User Fields collection as an experimental feature.

If a User has some Optional User Fields that are currently blank, enabling this setting, will cause the bot to start asking the user for the information. It will progress through the outstanding User Fields until they are completed and then revert to normal behaviour.

(sorry for slowness of response - this was recorded in dev)

NB This feature only supports:

• Text
• Dropdowns
• Confirmations

Multi-select is not yet supported.

The fields must be optional

The order of the User Fields determines priority.

[merefield (Robert)]

Chatbot 1.3 has been released

• FEATURE: add option to manage quotas by token
  • new setting chatbot_quota_basis allows you to opt in, otherwise it remains by query number
  • the only difference by default is that the quota counts down from the starting max quota (which is still based on same trust level settings).
• FEATURE: include cost of API calls like DALL-E in user consumption
  • token basis only
  • various new settings to tweak to influence token cost of API features
• FEATURE: add function to allow user to check their remaining quota

FEATURE: add option to manage quotas by token

Chatbot has always included the ability to enforce a usage quota.

Today this becomes more sophisticated!

Chatbot can now track and enforce a quota on usage by token.

You can cost calls to API’s to discourage their over-use, so you can feel less worried about giving people access to picture generation, for example.

Token tracking is opt in. If you change to token tracking, make sure you increase each trust level quota, because the values will need to be a few orders of magnitude greater. Experiment.

However, tracking by query has changed slightly. It will now reset to your maximum allowed number of queries and count down to zero.

You might consider running the global quota reset job once as a one-off to make sure everyone starts with a full quota. This is a one-timer!

In sidekiq at /sidekiq/scheduler:

Hit the trigger button.

In either case users can now ask how much quota they have left:

NB: known issue: if this question is asked for the first time before the reset job has run, the answer might be a bit strange! Regardless of the reset job running, the second time this question is asked it should work fine.

[Epoch]

Hey Robert,

I appreciate all the work you put into the bot however I am getting the following error in the logs when trying to invoke it in chat.

Any ideas without uninstall/reinstalling?