In a community discussion titled ‘Trying out the new AI models,’ users explore the performance of AI tools like grok-2-1212 and Gemini Flash 2 for summarizing topics and powering AI bots on Discourse. Don reports that grok-2-1212 works well for summaries but struggles with tool handling in AI bot mode, often failing searches or timing out — despite occasionally returning correct results. MihirR questions whether search failures stem from incomplete indexing or query misunderstanding. Don confirms inconsistent behavior, noting searches sometimes return nothing or hang. sam suggests trying XML tools, which Don finds helpful — though issues persist when starting new conversations, where AI sometimes halts after an English intro sentence. sam proposes allowing “grounding” examples in personas to improve accuracy. MihirR worries about performance slowdowns with large datasets, but sam cautions that examples can cause “contamination” — where models mistakenly assume user input — and recommends crafting clear system messages, possibly in Hungarian, or embedding XML tool examples. Don tests a Hungarian system message with grok-2-1212 without success, but switching to grok-beta resolves issues — even with English system messages. The thread highlights the experimental nature of current AI integrations and the importance of fine-tuning system prompts and tool configurations for reliability.
I make this topic for share experiences with the new AI models using on Discourse.
I set up a few days ago the grok-2-1212 for topic summaries. It works really well. The language detection and quality is well enough. I tried it also with AI bot but it most of time failed I guess it can’t handle the tools well yet.
I also tried the Gemini Flash 2 for AI bot. It works fast and add great answers however sometimes it seems it can’t handle well the tools and broke the answer. Sometimes it is a simple markdowns formatting issue, sometimes it can’t search. On my forum most of time says didn’t find anything on the forum but I know more topic about that subject…
For the search problem, do you think it could be related to the AI not fully indexing the forum content or there might be a mismatch in query understanding?
If have no idea. Most of time it searches for nothing “” or failed with timeout… but sometimes do the search correctly and linked the correct topics. It will be good I think but it’s strongly experimental yet.
Thanks, I tried it now. Yeah looks better, the problem is mostly happening now when I create a new conversation. It starts in English something like: I am searching for…in sitename… and stop replying. Sometimes it continue the answer after the English I am searching… sentence on the correct Hungarian language and add good answer. However if I reply the grok response after that in the conversation it will works great.
Bingo! But do you think it’ll slow things down, especially with a lot of data? Could it affect response times for AI queries or search results, or is it all good?
