Feedback on AI Bot


Item: 1

Not a bug but worth a mention.

Posted a question and the Discourse AI - AI Bot responded.
Then wanted to delete the topic (think testing) using the delete button on the first post,

Received this message


Item: 2

If a earlier reply is edited the bot will not change its reply based on the edit.

Item: 3

If the bot creates a completion such as a long list that needs to be continued, then the user clicks reply for the topic, not reply to the post and uses a word like continue more than once an error happens.


Item: 4


In searching for help it seems that the embeddings did not include the documentation of all of the plugins and/or themes. The example that lead to this idea is from the use of the Discourse AI - AI Bot for this topic and reply

Item: 5


The bot needs thumbs up/thumbs down reply buttons like ChatGPT to give feedback. I have no idea on what OpenAI does with the feedback in the buttons other than currently (08/19/2023) to generate a different reply possibly with the feedback text as part of the prompt. I suspect they do save the feedback and somehow are using it, possibly with other AI agents but for what I know not. I note this because I use other generative bots like ChatGPT, Claude, Bard, Poe and such and at times using the thumbs down forces one to think about why the response is wrong and help guide it to a potentially better reply.

Item: 6

Equal credit for this item goes to @Lilly

This one requires a total redesign of the AI bot. So it might actually be the start of a different type of bot for Discourse.

As we know the context window is limited in size. While bots like ChatGPT allow one to go back to an earlier prompt and edit it, which deletes all of the following prompts and completions, it does however revert back to the earlier context window. Now going forward one can enter a modified prompt which if it gives a better answer sets up the information in the context window for even better results.

A different way to think about this is playing a game with saves or not. If you can save your game at different times and go back to an earlier save then you don’t have to redo all of the work from the start. So if you make a serious mistake, you can just erase it by going back to an earlier save. But without the save you have to start all over again.

The same is true with chat bots like ChatGPT, you can go back to an earlier prompt which restores the context window. But with the Discourse chat bot you can not go back to an earlier prompt so once you fill up the context window with invalid information you have to go back to the start by starting a new topic.

Why is this such an issue worth noting. If you are trying to use the Discourse AI bot to create some code and you know that the bot was generating good code at one point then bad code later because of something in a prompt, you can jump back before that point, fix the prompt and move forward again. Without the ability to go back you have to start all over again.


Item: 7

Allow a reply to be only a count of XYZ.

The Discourse AI bot Forum Helper persona IIRC has a set of commands it can perform, most often starting with a search for topics. In the background the search is often SQL generated based on the Discourse AI prompt.

For those that have used SQL over the years have learned, sometimes it is better to start with just a count result then a selection of items as it is much faster and often all that is needed.

Today used this prompt on the OpenAI forum

How many post have 0 replies that and the post was created before October 2023

which resulted in


Found 50 results for status:noreplies before :2023-10-01.

Apparently there is a cutoff of 50 somewhere; the actual count is more than a few hundred. Also only need the count and not a list of results.

1 Like