I have been trying to setup a chatbot to answer question on some of our private forums, but it only seems to find results from the forum topics if I make the forums public. I have tried setting “Read Private” and “Search Private” in the Persona - and have the persona “allowed groups” set to the same groups as the category security. Can you think of anything else I should try?
The user initiating the conversation with the Persona has access to the private categories?
1 Like
Yes. The topics are in a subcategory - so I will test it again with the parent (both are private) in case it’s a sub-category issue
I just checked, and I get the same problem if I post the topic directly in the parent forum. It works fine if I make these forums public..
Can you recreate this with your setup?
One nice way of debugging this is to enable the ai debugging. When you enable it you will see an (I) button, can you confirm that when you click it and looks through the big blob sent to the LLM that the private content is not found?
Our engine does pass the user to the search backend (the user initiating the convo with bot)
2 Likes
Hi @sam,
Thanks for the debug tip - I had not looked into that before!
I can see several messages being sent to the LLM, but nothing that helps me understand why “search private” isn’t working. There appear to 2 “pairs” of message - 1 pair for the actual search, and another pair for the “titlebot” (which simply derives a title from the search result).
As far as I can tell, the first search message is to Dall-E (not sure why?) so that does not seem relevant (please confirm) but the 2nd message looks like this - with nothing else appearing afterwards. I presume this means it is not finding anything?
1:
-
-
role: user
-
name: NeilEvans
-
content: What is Q03-DZN00-00000?
-
-
stream: true
Thanks in advance for your help!
N
I think I have made some progress - it was not using the persona I intended, so was defaulting to another persona that had “bot” enabled. I have turned off “bot” for the other personas now, so it is using the persona I wanted.
There also seemed to be a problem with having a user assigned to the persona I was using. When I removed this user, I started to get better results - and was getting different behaviour depending on which “Enabled Tools” I had set for the persona..
Will carry on investigating…
OK - I have got it working properly in the AI bot (via discourse-ai/ai-bot/conversations) but the same search finds nothing using the “search assistant” which appears next to the main search results. However, If I click on “continue conversation” - and ask it to try again, it finds what i am looking for in the same way as the AI bot! Is this a bug, or have a set something up wrong?
What do you mean by that?
Are you trying to setup a Persona with Search for the Discovery feature — which shows on search and returns conversational results ?
I’m using the a single persona that has Bot, Read and Search enabled. It uses this persona from the AI bot interface - and it works great. However, it doesn’t find anything if I used the main site search - which pops-up the search assistant on the right hand side. If I click on “continue conversation” i can see it’s using the same Persona - and if I then ask it to “try again” it finds what i am looking for. Does that make sense?
P.S. I get the same results if I just have Bot and Read enabled
Oh I see you mean that proper RAG ins’t happening when using the Persona here:
right?
This feature doesn’t pass the user who initiated the interaction AFAIK, so it doesn’t work with private content.
Let me see how hard it is to wire it up.
1 Like
Exactly - thanks the problem I think. Thanks Falco! Much appreciated!!
1 Like
Just fixed it, please update your site and try again.