Limit the number of AI tokens a user can use in a day?

[Shauny (Shaun Robinson)]

If you give certain members access to AI, what is stopping them from using it all the time for their work etc., is there any way to limit how many tokens they can use per day or week etc?

[Jagster (Jakke Lehtonen)]

There isn’t such limitations when used Discourse AI, so the answer is plain nothing.

The other solution, Chatbot, has weekly limits how many request can be made.

So, you have to choose which one suits better for you. I use both. Chatbot for more general use inside my forum ^[1] and DAI is limited based by a group and (tuned) purposes using personas. So, I kind of exploit the best sides of both, because those two are only partly overlapping.

---

1. general means type of questions, otherwise it is tuned more specialized way ↩︎

[sam (Sam Saffron)]

Moving this to feature, we intend to add a quota system, it keeps popping up

I am thinking of just adding a group selectors with input/output counts and duration on each llm , so you can add optional quotas

[markschmucker (Mark S)]

I kinda need per-user (not per-group) rate limits now, so am trying to roll my own interim solution. Limiting the number of prompts instead of tokens would be fine. I’m thinking of a webhook on post_event that says if it’s a PM and a user is posting to a bot, increment a custom field ‘ai_query_count’ on the user. I think that part would work.

Then what to do if the count gets too high? I tried some javascript in admin > customize > head that reads the user’s count and tries to disable the Reply button if the count is too high, but I can’t come up with a selector term to get the button.

Or maybe there’s a better approach. Any ideas are welcome!

[sam (Sam Saffron)]

@markschmucker

I kinda need per-user (not per-group) rate limits now

There are technically the same, you just create a group of 1.

This should land this week and solve it:

FEATURE: llm quotas

[markschmucker (Mark S)]

I understand per-group quotas are are good for managing budget, but what’s to keep one person from hogging all the group’s quota at the beginning of the time period? EDIT: And potentially hogging it for their own unrelated work, like the OP asked?

@sam

you just create a group of 1

We have 3000 members. So create 3000 groups? Won’t that wreck /g?

[sam (Sam Saffron)]

I am a bit confused about this question, quotas are defined by group now and applied per user

If group is allowed 1000 tokens it means no single user is allowed more than 1000 tokens

Quota is not shared between users, not against adding the concept of absolute quota if we need it later

[markschmucker (Mark S)]

Oh. Then I completely misunderstood. The feature says:

Setting per-group token and usage limits with configurable durations

To me “per-group limits” sounds like the group as a whole has a limit.

So this is exactly what I’m looking for- will wait for the new feature.

[oppman (Craig)]

I’m going to assign this configuration to an intern next Wednesday on a self-hosted Discourse site. Do I need to tell them to pull from a main branch on GitHub? Or, if they add this line in app.yml, is it sufficient?

hooks:
  after_code:
    - exec:
        cd: $home/plugins
        cmd:
          - git clone https://github.com/discourse/docker_manager.git
          ...
          ...
          - git clone https://github.com/discourse/discourse-ai.git 

I want to allow the Discourse AI plugin features to only registered users. Is that the default behavior?

It doesn’t look to me that the AI-enhanced search has per-group control???

For the AI bot, I’m thinking of setting it to trust 0 and setting a limit on tokens per group, increasing as the trust grows. Is this a good strategy?

[sam (Sam Saffron)]

This has been merged. I just need to spend some time documenting it.

[oppman (Craig)]

@sam Thank you for your work. The intern set up quotas for each trust level today.

They have not tested it fully yet.

When the group-limit is hit for the day, does the user get a message? Can this message be customized?

I want the AI to be useful, but protect us from abuse. If a person wants to use more AI search, I want to send them a note that they can contact a human to get the limit increased. Staff can then move them into a pre-configured group manually.

In your example, I notice that you’re using amazon nova pro v1. The intern set up our implementation with OpenAI gpt-o4, likely just because of brand recognition.

The intern is still trying to figure out the effectiveness vs costs of different models. Any advice from anyone on the forum would be wonderful.