Discourse AI — AI-триаж

[sam (Sam Saffron)]

AI-триаж предназначен для улучшения управления и модерации сообщений на форуме за счет автоматизации процесса классификации постов.

Обратите внимание, что для работы этой функции требуются плагины discourse-automation и discourse-ai.

Сценарии использования

1. Автоматическая категоризация постов: AI-триаж может автоматически категоризировать темы на основе их содержания. Это особенно полезно для крупных форумов, где ручная категоризация может занимать много времени. Вы можете применять правила к подмножеству тем (только к первым темам или темам, опубликованным в определённых категориях).

2. Присвоение тегов постам: AI-триаж может присваивать указанные теги постам. Эта функция помогает лучше организовать посты и делает поиск и извлечение тем более эффективным.

3. Автоматические ответы: AI-триаж может генерировать ответы на посты, используя заранее определённые шаблоны. Это полезно для ответов на часто задаваемые вопросы или распространённые запросы, перемещения спама в категорию «мусор» и других задач.

4. Скрытие тем: AI-триаж может скрывать темы на основе определённых критериев. Это можно использовать для управления спамом или неуместным контентом.

5. Обнаружение контента для взрослых (NSFW), токсичности и спама

   • Настройка обнаружения NSFW в вашем сообществе
   • Настройка обнаружения токсичности в вашем сообществе
   • Настройка обнаружения спама в вашем сообществе

Как это работает

AI-триаж использует модели искусственного интеллекта для анализа содержания постов. На основе этого анализа, если модель возвращает определённый текст, выполняются указанные действия. К таким действиям относятся перемещение поста в указанную категорию, добавление тегов, ответ с использованием заранее определённого шаблона и скрытие темы.

Настройка

Поле системного промпта устарело и заменено функцией Агентов. Если у вас была настройка автоматизации на базе ИИ до этого изменения, автоматически будет создан новый Агент с соответствующим системным промптом.

Для настройки AI-триажа необходимо указать следующие параметры:

• Агент: Агент, используемый для триажа. Убедитесь, что он отвечает одним словом, которое можно использовать для запуска указанных ниже действий.
• Поиск текста: Если в ответе LLM появляется указанный текст, применяются указанные ниже действия.
• Максимальное количество токенов поста: Максимальное количество токенов, которое будет просканировано с помощью триажа на базе LLM.
• Стоп-последовательности: Указывают модели прекратить генерацию токенов при достижении одного из этих значений.
• Максимальное количество выходных токенов: Если указано, устанавливает верхний предел максимального количества токенов, которое может сгенерировать модель.
• Включить личные сообщения: Также сканировать и проводить триаж личных сообщений.
• Категория: Это категория, в которую будет перемещён пост, если будет найден указанный текст.
• Теги: Это теги, которые будут добавлены к посту, если будет найден указанный текст.
• Скрыть тему: Если эта опция включена, тема будет скрыта при обнаружении указанного текста.
• Пометить пост: Если эта опция включена, тема будет отправлена в очередь на рассмотрение для модераторов, сотрудников и администраторов для принятия мер.
• Тип флага: Тип флага для применения. Доступные варианты:
  • Добавить пост в очередь на рассмотрение — отправляет только в очередь на рассмотрение
  • Добавить пост в очередь на рассмотрение и скрыть пост — отправляет в очередь на рассмотрение и скрывает пост
  • Добавить пост в очередь на рассмотрение и удалить пост — отправляет в очередь на рассмотрение и мягко удаляет пост
  • Добавить пост в очередь на рассмотрение, удалить пост и заблокировать пользователя — отправляет в очередь на рассмотрение, мягко удаляет пост и блокирует пользователя
  • Пометить как спам и скрыть пост — помечает как спам и скрывает
  • Пометить как спам, скрыть пост и заблокировать пользователя — помечает как спам, скрывает и блокирует пользователя
• Ответ пользователю: Это пользователь, который будет упомянут в шаблонном ответе.
• Ответ: Это заранее определённый ответ, который будет опубликован, если будет найден указанный текст.
• Агент ответа: Агент ИИ, используемый для ответов. Должен иметь настройки LLM по умолчанию LLM, будет иметь приоритет над шаблонным ответом. Будет отвечать условно, в отличие от Discourse AI - AI triage using Agent.
• Ответ как шепот: Должен ли ответ ИИ появляться как шепот.
• Уведомить автора через ЛС: Отправить личное сообщение автору поста, когда его пост поставлен в очередь и удалён.
• Отправитель ЛС: Пользователь, который отправит ЛС (по умолчанию — система).
• Содержимое ЛС: Опциональное пользовательское сообщение для отправки автору.

Важные замечания

• Вызовы LLM могут быть дорогими. При применении классификатора внимательно следите за затратами и всегда рассмотрите возможность запуска этого только на небольших подмножествах данных.
• Конкретный сценарий использования, который мы тестировали (и который работает в продакшене для клиента), — это классификация первых тем пользователя.
• Мы рекомендуем использовать AI bot для помощи в создании системного промпта агента.

[EricGT]

Being one one of the lucky ones to see this in action this is one feature for large sites that you should really understand and consider using.

As it works using an LLM based AI it does not always reach the correct conclusion but it gets so much right in the cases I have seen it used for a few days doing some of the work of a moderator that it definitely was a significant benefit.

Some of the early discussions with this are in the Lounge category on the OpenAI site. While access to that category can be had by anyone, the TL3 requirements must be meet and with OpenAI being a large site it does take quite a bit of effort to get to TL3 on the site.

For those with access here is the link

https://community.openai.com/t/lost-users-first-empirical-data/403082/95

---

Basically what the logic is doing to help the moderators for a specific problem is that we are seeing about 5% of the post from new users that think the forum is where they post questions to ChatGPT, clearly they are lost or perhaps a search result is providing an invalid link. The AI identifies such post, replies with prewritten text and changes the category and tags as needed, e.g.

Education is very important how can children who are disinterested in the...

---

For the case noted, here is what the “Discourse AI Post Classifier - Automation rule” is doing

Automated Post Categorization
The classifier is changing the category to ChatGPT as needed. As most new users will not select a category, the OpenaAI site currently defaults to the category API for new post which is incorrect in this case.

Post Tagging
The classifier is charging the tag(s) to lost-user in this case. The tag name was created by a TL3 user on the forum who was manually changing the tags.

Automated Responses
The classifier is replying with a prewritten post.

Topic Hiding
The topics are being unlisted as they were not of value to developers who use the site.

Reply User
System is being used as the creator of the reply.

Note: I would provide the Configuration for this but lack the access. Maybe @Sam can add the details, AFAIK it can be made public as nothing is confidential in the configuration. As it is for a specific site would not expect to find it in the public repository. If you understand this technology then it is not hard to guess the correct values, or close to them. The System Prompt did take some work and perhaps @sam can share some of the lessons learned, the knowledge of how to craft the prompt was of great value during the development phase.

System Prompt

Note: This is a version posted in the OpenAI Lounge (ref) pretty sure the final version is different but one example saves a lot of guessing.

You are bot that is triaging all first interactions a user has on the OpenAI developer forum.

Please only ever respond with “ok” or “bad”

Posts are on topic and ok if:

They relate to OpenAI APIs / Community / Plugin development / Documentation / Prompting
They are about developing or improving methods for prompting large language models
They relate to AI in a general way
They involve complex discussions or logical problems related to AI
Posts are bad if:

A user appears to be having a random conversation with ChatGPT
A user is off topic discussing an unrelated field
A user is prompting a large language model to generate text without a clear purpose
You have extreme difficulty understanding what they are about
Relates to an OpenAI topic BUT is clearly a conversation with a bot
A user is attempting to train or test the AI model through their post
A user is giving instructions or asking for responses in a non-discussion format
A user is posting content that is not conducive to meaningful discussion or learning
A user is posting hypothetical or speculative content without clear relation to OpenAI’s scope
A user is posting the entire post in a language that is not English
Please classify the following content surrounded by [[[]]]:

[[[
%%POST%%
]]]

---

FYI

If you are a moderator on a site using this and you want/need to see a list of unlisted post, remember that you can not use search. However you can navigate to such a list using the category then selecting the tags. Or you could also hand craft the url, e.g. https://community.openai.com/tags/c/chatgpt/19/lost-user

---

Yes it even works for languages other than English

Rend der Eingewöhnung von ihrem Vater bogustet. Schon Maries Geschwister...

However it does not work with images , it will not convert an image to text then run the check, e.g.

Solving of mathematics equation involving fraction with addition

Here is a recent false positive or at least I think so.

Looking for a developer for a Dungeon Master AI bot on Discord

---

For the details on false/true negative/positive - Classification: True vs. False and Positive vs. Negative

[EricGT]

A possible feature.

At least for moderators reviewing a post, convert the text to the language of the moderator when they view the topic. This is not implying just for a flag but for any viewing of the post.

---

At times when reviewing the actions of the Discourse AI Post Classifier there is a need to convert the text of the post to my language (English) to check if the AI took a correct action. Currently it is easier for me to paste the text into Google Translate.

[sam (Sam Saffron)]

Worth noting and related to this side quest.

@keegan is working on integrating AI helper into our popup menu. So in future in cases where there is a bunch of text in an unknown language you will be able to just highlight the text and hit translate.

We support that in the composer today, but once merged you will be able to highlight text in any Discourse post and get a quick LLM based translation.

[ncaming215 (nate c.)]

This … changes the game for me. Where can I send thank you cards.

[jordan-violet]

@sam

Post Tagging: The post classifier can assign specified tags to posts. This feature aids in the better organization of posts and makes searching and retrieving topics more efficient.

What is an example of how to do this by referencing the existing tags your forum has today? e.g. the post is analyzed by the LLM and the most likely/relevant tags are added to the post.

[sam (Sam Saffron)]

I love this question, can you expand on it please… how would you like to see this work?

• Would you apply “auto tagging” if the topic already has tags?
• Would you apply it on all first posts or just on first posts by particular groups? (eg: tl0 / tl1)
• Is this something you would prefer to run by hand on a subset of topics?

We are going to need to make some adjustments here… at the moment the classifier is binary, but the changes are reasonably easy as long as we know what the goal is?

[jordan-violet]

• Apply to all new topics in a category
• You could certainly have the option to vary by trust level. Like everything (and in this case, every community), it depends. In the case of my community, I’d likely keep it turned on for everyone but tl4 (or just everyone, if that configuration isn’t an option)
• I’d prefer it ran on all new topics in specified categories

The goal here, of course, is to never have to work on tagging topics on our own and to let AI do it entirely based on the initial topic being created and it looking at the existing tags we have now.

It could probably be improved even more if you had the ability to add descriptions to tags in Discourse, and to let it use those descriptions for additional context to add the right tags

[EricGT]

First off let me state that I am fully behind this ability and making it better. The following is just some facts of how it currently works (10/16/2023). Daily as a category moderator on OpenAI I review all of the new post, not all of the replies, and do see all of the AI false positives and negatives.

@jordan-violet

The goal here, of course, is to never have to work on tagging topics

As one who uses this feature, Discourse AI Post Classifier - Automation rule, be aware

• it is not 100% accurate
• currently takes manual action to undo if the AI did a false positive. Depending upon the change may need a category moderator or higher; specifically listing an unlisted topic.
• may miss some topics, false negative
• as many users who will receive the automated reply by the AI do not understand how to flag the AI reply if it is a false positive, they will have to be identified and changed manually. So will need the help of real users to catch these
• understand how to craft the prompt to get it to work as needed
• could potentially need fine tuning and/or agents to get close to a desired satisfaction level which could get expensive
• no way to track false positives and negatives for use with understanding how to change the prompt and/or possible fine tuning and/or agents

---

false positive - The AI should not have made a change and did, which is incorrect.
false negative - The AI should have made a change and did not, which is incorrect.
true positive - The AI made a change, which is correct.
true negative - The AI did not make a change, which is correct.

[sam (Sam Saffron)]

A good opening move here is trying out a prompt in creative mode:

Something like…

You are a Discourse auto tagger, you suggest a list of tags for topic.

The tags are:

tag1: description
tag2: description

Suggest up to 3 tags for the following topic:

TOPIC

Try this on a few of your topics, how well doe it do?

[EricGT]

@EricGT

At times when reviewing the actions of the Discourse AI Post Classifier there is a need to convert the text of the post to my language (English) to check if the AI took a correct action. Currently it is easier for me to paste the text into Google Translate.

@sam

@keegan is working on integrating AI helper into our popup menu. So in future in cases where there is a bunch of text in an unknown language you will be able to just highlight the text and hit translate.

Works great! Thanks

[EricGT]

Recent update (11/01/2023)

The user id for the bot is now gpt-4-triage, at least on the OpenAI forum.

This was done so that users can silence the bot as needed.

Example of the bot responding.

Write a 500 to 1000 words about wolfgang keller

[sam (Sam Saffron)]

Note, this is configurable you can have the automatic reply come from any user you wish

[ncaming215 (nate c.)]

So, following up after giving this a whirl.

It seems like this is in the early stages, and I’m excited to see where it goes, but @jordan-violet’s take is 100% what I’m looking to accomplish. I did try the test prompt that @sam suggested and it appeared smart enough to apply the correct tags in theory. However, it appears that the current script limits applicable tags to 7, and we have considerably more than that.

Short of it? Would be great if it would semantically apply existing tags to a new topic. In terms of forum management (even here on meta, I’ve noticed!), the consistent application of tags to better order a community and its content continues to be an uphill battle, and it’s pure chaos at scale. For those of us who rely on tags, this would be a boon, even if it’s 75% accurate.

I’ll keep my eyes on this one as it develops!

[aas]

This is great! Do you have any suggestions for debugging this? I just configured an automation that wasn’t triggered when expected. I don’t see any relevant logs.

[sam (Sam Saffron)]

Hmmm what ended up happening here, can you open a dedicated topic?

[aas]

IIRC, I didn’t set this option at first. When I did, it started working.

[loginerror (Maciej Kuźmicz)]

Quick question. Are there plans to display an audit log for those automations (maybe similarly to the webhook pattern that you already have)?

It would help ease the “what did you do” scary feeling that I’m getting after an automation is enabled

[sam (Sam Saffron)]

Ai log already stores all llm interactions. You can use that today via a data explorer query

[loginerror (Maciej Kuźmicz)]

@sam

all llm interactions

While this is true, it is rather tricky to link the log entries of llm_triage type to a post/topic that it affects, or to the action that was taken (or not) as a result.

Having an overview of these actions would be great