Discourse AI - AI triage

For that is one answer: AI is bad at those.

You have somewhat romantic idea what AI is, what it can do, or how a human can interact with it.

But there is such semi-automatic function: AI-helper. You can try it. But there is a reason why even here is relatively little titles by AI

What it can do, again: relatively good, is proofreading and translating. Not so big suprise, because those are language models working with propabilities. But even then it struggles when it should guess used language.

Giving all semantic powers to AI would lead to more human job in background. I live in dream world in administration way. I don’t have problematic users and I don’t have spam. And yet I got false positives — quite rarely, but there is. And when trying to fix that, ”administrators or authorized users can edit it and give feedback to AI” is easier to write than do.

My way to write is sometimes hard to AI, that knows the best English. I try to keep structure of English right, but just how and in which order I should use places and times, is something I can’t. When an AI hits in a situation where I write Finnish using English words, it corrects me sometimes totally wrong and quite often misses the context and point.

I mean what are possibilities of an AI when it gets a sentence where is just hieroglyphs, aka. smileys and reactions? Or when a topic is just a title? Classification of such ones in right categories and tags is close to impossible to an AI.

AI works fine, if it succees rate is 85-90 %, and fixing those remaining 10-15% doesn’t cost too much. That is acceptable. But if a moderator do as many errors, admin will change moderation team quite fast. At same time it means that out of 1000 posts 100-150 is in wrong category with wrong tags and titles are after hard hallucination.

Chat GPT is more expensive AI tool that any of us uses in forums. It gave an solution to one question of Varnish Cache. It looked right, but it didn’t work. I asked help from Chat GPT in the first place only to save time, bacause I was too lazy to write needed if-structure by myself.

The reason was Chat GPT couldn’t see the difference between County-Code and Country-Code. Chatting with Chat GPT and then fixing the issue took more time than I would spend doing it from scratch by myself.

Why I use it? Because of that 85% success rate.

But when it can’t tell how many Rs are in word strawberry, we would like to use it working with words and contexts — because it can do it in the most of time. But when it fails, and it does mass job… fixing can and will be quite demanding, because we have other tech too.

AI messed my summaries, and at same time translates. Easy peasy to delete those records. But creating new ones costs me 100 bucks. What if it would, and it would if possible, put all topics in wrong categories with wrong tags? Fixable, but costly error.

AIs aren’t there yet, if ever, that those could do ”everything”. That is more or less scifi. But those can do some some jobs, when there is limits and well structured guidelines, relatively well. But dealing with hazy reality… no.

I like an idea of categoryless forum, where tags are logical based the human needs to find information, and everything is automated, though.

2 Likes

Users who will produce valuable content for the community can see these areas, but new members generally do not bother with titles, categories or tags. In a Q&A community, all that is expected is to ask questions and get answers (for example, like an AI bot). I want this request for users who are more novice and looking for answers to their urgent questions, but at the same time, these questions should be published generally. When AI is used, such topics do not benefit the community and remain in the background. It would be more useful to enrich the site content and to have them in general use for the same questions.

I also create titles and tags with AI and it is very successful in this regard. I am sure that with a good prompt, I think AI will give good results in terms of categories and tags. Of course, this does not mean leaving it like that, the authorities can re-check such topics and make the necessary arrangements, but we can save time by giving control of this to AI in the first place. Content that is not in the right category can be reported or an authority can correct it when they see it. The aim is to make this job easier.

Actually, I can do this with a simple webhook. A new topic is opened, the given page is pinged, the correct category (on the site) and tags are selected with artificial intelligence, and the topic is updated (with API). However, it would be logical to develop a feature that already exists.


I agree with what you wrote, then let’s develop AI and run it like an agent and let the AI ​​make suggestions about opened or edited topics. For example, suitable categories, tags, etc. for an opened topic. If the only change that needs to be made is to approve it with a button and the topic is updated immediately. If not, make the edit and approve it quickly.

This is likely to be an extra cost, but it can be a good tool for administrators who want to make their jobs easier and not give up human control. An AI that monitors the activities on the forum and informs the administrator or the people in the selected group about them. Hmmmm, it doesn’t seem like a bad idea (a bit off topic of course)

YOu know your community better than I do. :person_shrugging:

Then maybe have the AI do that while the user is creating the topic?

I usually do this by running a simple php page that I have prepared when I have a question I am curious about. For example, when I am curious about something like “what is the golden ratio”, I write it to the page I have created and it interacts with AI and adds the site (except for the category). In other words, it creates the title, question, and tags itself. This shows that AI can actually be successful in this regard.

ex: Profil - nuran-kilic - Soru Cevap

Maybe we can hide such fields (we can put a simple button for those who want to edit) and allow the user to create a topic quickly with only the question or topic entry field. We can edit these fields later or we can do it ourselves with a simple websocket request. But Discourse does great things in AI, it would not surprise us if it had this kind of support.

Maybe I can come back with better planning and thinking without getting too off topic. Thank you for being patient with me :slight_smile:

Why not just read the Wikipedia article that makes it possible for the AI to create an answer that’s not as good?

1 Like

Succesful is matter of definition, and isn’t same thing than ability to do something.

To stay on topic, kind of, for me succesful ratio to identify spam is zero or really high, depending of definition. It is zero, because all it found were false positives. It is really high, because everything else it didn’t report spams.

On both cases it has ability to do something.

Same will happend when AI classifies topic. And, this is pure opinion, it writes almost never good enough articles to publish.

We were doing it like that before AI, but the topic here is my idea of ​​using artificial intelligence more effectively in discourse systems. You did a great job, it is not possible to customize it for everyone, that’s why we can create a plugin. I will make a small complaint, I have been trying to set up the discourse development environment for 3 days, it would be great if someone from the community can create a good installation guide on this subject :slight_smile:

Last I looked the scripts in those guides were out of date. Keeping a development environment up to date isn’t easy.

The recommended way seems to be the docker development version, but I’ve not had much luck with it either. I’m not clear how people do it.

1 Like

I will try it out, if I succeed I will share it. I couldn’t do it on the cloud, it works and crashes. I will try it locally (Windows). Eventually I will definitely try to set aside some time for this. If someone does it before me I will love it :slight_smile: