That’s some fast action! (Maybe I glossed over a time stamp). I’m so excited to switched back to the official plugin (I was pretty excited that is able to fork and change it as quickly that I did, but that’s different).
2 Likes
Hey @Aizada_M and @swong I have documented the request for us to work on! Could you help me understand why you might want to limit access to certain personas for specific users in the first place?
I’ve linked the Enable AI Bot documentation there so users can follow the guide accordingly
Awesome! In my opinion, having personas like SQL and Settings should really only be accessible to admins of the community. It doesn’t make sense (for my use case anyways) for general community members to have these internal workings exposed to them.
2 Likes
When I looked into a chatbot before there was a cost (fee)
Do these options mean there may be chatbots that don’t add cost?
I think I can confidently answer this one.
As a self-hoster, definitely not.
- You are either paying “pay as you go” for API calls to a service like Open AI OR
- You are having to host your own LLM which will significantly increase the cost of your server infrastructure.
(as a hosted customer, you may or may not get a special “deal” for the time being where use of the bot is included in your package).
1 Like
Hello! I think @swong has already described what I want to implement. I just want to clarify that it would not be bad if it was additionally possible to indicate for which group GPT3.5 is available, and for which group GPT4.
Additionally, I would like to add the following wishes:
- Display the bot icon for all users (currently the icon is only shown for selected groups), as I would like to advertise the presence of the AI on my forum. Scenario: An ordinary user who does not have access to the bot clicks on the robot icon and sends his request, in response the bot writes to the user that the user does not have access to communicate and gives instructions for inclusion in an approved group (possibly a paid group).
- The chatbot plugin has the ability to limit tokens, depending on the user group, which allows the administrator to control the expenditure of the budget. I would like to see a similar solution in the Discourse AI plugin. At this time, there is a risk of abuse by users (both regular users and site staff) when communicating with AI. This, in turn, can lead to uncontrolled spending by users of financial resources of the site administration, while there is no way to see which user has burned how many tokens.
- Allow administrators to independently select an icon (
) as the bot icon, which is usually displayed in the site header . For example, I currently use 2 plugins on my site, the aforementioned chatbot plugin (which also displays a robot icon) and the Discourse AI plugin. Displaying the same icons with different functions in different parts of the site can be confusing to users.
This isn’t answering to your question on the way you would like, but… don’t. That is just as annoying than any other paywall after click.
That information must be given at the same moment an user clicks that icon — and icon should tell too there is a wall.
1 Like
But (If I understood you correctly), I think you agree that it is better to display the icon than not to display it, otherwise how will the user know about the presence of a button that he can only use after joining the group?
If I understand you correctly, you do not recommend displaying access information solely through the bot response? In this case, I might suggest displaying such information on a new page or modal window.
In any case, I think that the bot icon should (if the admins decide so) be displayed for all users, otherwise I will have to hang a banner on the main page (which will take up most of the useful area of the site) stating that there is some kind of icon that is not visible , but will be visible after users fulfill certain conditions.
I have a small forum with few members and likely fewer that would be impressed, older crowd, and I’d bet many mobile users are already utilizing suggestive typing/words etc so this might well be redunant.
But, if you were to guess can you offer an estimate on cost for this example?
Also curious, I seen it enabled here a few days back, but not now, was it shut down for certain users/instances and why?
10 usd a month if OpenAI/3.1 and you will use summarization too. If you play with it and testing if 4.0 is better than 3.5 Turbo then it will be easily 10 bucks a week. Plus images of course but if StabilityAI is in use and ordinary 1024x1024 are enough then those are practically free — if something like 10 USD/4000-5000 images is counted as free.
1 Like
Are there any free AI providers?
Not that I am aware of, and to be honest I don’t think it is likely many services like that will pop up. Running large language models is very costly.
There are some open source models you could run like llama 2, but you will need a rather powerful machine GPU wise
2 Likes
But do I need a cpu to run the program?
On VPS? Yes, and huge amount RAM.
If I’m remembering right you are a brazilian? And there is big risk that your users start to generate images — that was just a wild guess.
I’ve paid now, after three weeks testing, to OpenAI shy 40 bucks and it come from light background jobs, even lighter use from users and from me. It will be close to 20 a month when a new toy stops to be new.
If I would like to use open source models in VPS I would be paying easy 200+ usd a month just because of amount of CPU and RAM.
At my books any other solution than OpenAI etc. is to big players and corporates only. We poor ones are counting pennies.
2 Likes
I wanted to use AI just to create texts on the forum, not for images, is the llama too difficult to configure? The bad thing about being Brazilian is that my monthly salary in dollars is USD 244.95, I spend USD 100 on rent and another USD 100 on bills such as electricity, water, telephone, there is almost nothing left for me =(
So use GPT 3.5 via API? Pay as you go.
To run the 70Bn parameter Open Source Llama 2 model (that performs nearly as well) locally you need something at least as powerful as a MacBook Pro M3 Max which comes in at a jaw dropping $5k-ish (give or take a spec bump or two) or a PC with a $6k GPU (you need at least two 4090’s but apparently even they are not as quick as the M3 Max).
Obviously prices will come down but the demand for better performing models will probably outstrip supply of hardware improvements.
Cheapest way to run on local would probably be a dual 3090, struggling to find benchmarks but I do think it outperforms a a m3 max. You can quantize a 70B quite a lot and run … even on 24GB.
1 Like
Yeah, sure, there are things you can do … but still a bit more than “$9.99” though 
1 Like