Balancing Costs and Functionality in AI-Powered Forums

Thank you for your carefully thought out topic!

I hear you, at the heart one thing that is clearly needed in the medium term is a concept of “quotas”.

Pick LLM → set quota for group (how many tokens are they allowed per day, since everything is billed by tokens)

This let’s you open up AI features, safely, to larger groups without worrying about going bust.

I think we will get there, but it is not quite scheduled.

On the upside though we have done a lot of work recently to normalize our LLM interfaces so this is going to be much more practical and straightforward to build than it was in the past.

5 Likes