Guida al personaggio AI del ricercatore del forum

Documentazione Gestione del sito
, ,
1

:bookmark: This guide explains the Forum Researcher persona in Discourse AI, how it works, and how to configure it for in-depth forum content analysis.

:person_raising_hand: Required user level: Administrator (to enable and configure), All users (to interact, if granted access)

Understanding and using the Forum Researcher persona

The Discourse AI plugin includes the Forum Researcher persona, a powerful tool designed for conducting in-depth research on the content within your forum. This persona can help you uncover insights, summarize discussions, and analyze trends across your community.

Summary

This document will cover:

  • How the Forum Researcher persona functions.
  • Steps to configure the Forum Researcher.
  • Best practices for interacting with the persona.
  • The distinction between the Forum Researcher and standard forum helper tools.
  • Guidance on selecting an appropriate Large Language Model (LLM).
  • Debugging tips for research tasks.
  • Current limitations of the persona.

How it works

The Forum Researcher persona uses a dedicated Researcher tool. This tool is engineered to:

  1. Access forum content: It can read through various sections of your forum.
  2. Apply advanced filters: A flexible filter system allows the tool to target relevant information precisely. You can specify content by:
    • Specific categories (e.g., category:support)
    • Users or groups (e.g., usernames:sam,jane, group:moderators)
    • Keywords in posts or topic titles (e.g., keywords:regression,bug, topic_keywords:"feature request")
    • Date ranges for posts or topics (e.g., after:2024-01-01 before:2024-06-30)
    • Topic status (e.g., status:open, status:closed)
    • Post type (e.g., post_type:first)
    • Filters can be combined using AND logic (space-separated) or OR logic (using OR between filter groups). For example: category:bugs status:open after:2024-05-01 OR tag:critical usernames:sally.
  3. Analyze content with Large Language Models (LLMs): After retrieving the filtered content, it uses an LLM to analyze the information, extract insights, and answer your specific questions or achieve your research goals.
  4. Follow a structured process: To ensure efficiency and accuracy, especially considering potential costs, the Forum Researcher is designed to:
    • Understand: It will work with you to clarify your research goals at the beginning.
    • Plan: Based on your goals, it designs a comprehensive research approach using the available filters.
    • Test (Dry Run): Before executing the full analysis, the persona typically performs a “dry run.” This involves calculating how many posts match your filter criteria without immediately processing them with the LLM. The persona will then inform you of this count.
    • Refine: Based on the dry run results, if the number of posts is too large (risking high costs or overly broad results) or too small (potentially missing key information), the persona can help you adjust the filters.
    • Execute: Once you confirm the scope is appropriate (after the dry run), the persona runs the final analysis, sending the content to the LLM.
    • Summarize: It presents the findings, typically using Discourse Markdown, with links back to the original forum posts and topics as supporting evidence.

This methodical approach means you can ask the researcher to perform tasks like:

  • “Summarize the most frequently discussed unresolved bugs in the ‘mobile-app’ category from the last quarter, and identify any proposed solutions or workarounds mentioned in the discussions.”
  • “Help me identify the main arguments for and against the ‘New User Onboarding’ proposal topic (link), and list the key proponents of each side.”
  • “Review activity by the ‘documentation-team’ group in the past year and provide a report on their key contributions to how-to articles, highlighting any tutorials that received significant positive feedback.”

Configuring the Forum Researcher

The Forum Researcher is disabled by default because its usage can incur LLM costs.

  1. Enable Persona: Activate it by navigating to Admin → AI → Personas.
  2. Control Access: It is strongly recommended to limit this persona to specific groups to manage LLM costs. You can also use AI quotas for finer control.

Once enabled, the tool has several configuration options:

The image shows options for setting up research using an LLM tool with selections for language model, maximum number of results, private topics, maximum tokens per post, and maximum tokens per batch. (Captioned by AI)
The image shows options for setting up research using an LLM tool with selections for language model, maximum number of results, private topics, maximum tokens per post, and maximum tokens per batch. (Captioned by AI)661×680 46.7 KB

  • LLM: Select a specific LLM for research. This defaults to the bot’s current LLM. This option allows you to balance quality and cost.
  • Maximum number of results: This limits the number of posts processed per query to control costs. The default is 1000.
  • Include private: This allows searching in secure categories and private messages, using the interacting user’s permissions.
  • Maximum tokens per post: This truncates long posts to save token costs. It defaults to 2000 tokens, with a minimum of 50.
  • Maximum tokens per batch: This controls the data chunk size sent to the LLM. It’s useful for LLMs with large context windows or to maintain focus. If set below 8000, it defaults to the LLM’s maximum prompt tokens minus a 2000 token buffer.

Best practices for interaction

To get the most out of the Forum Researcher while managing costs:

  • Be specific with goals: Clearly define what you want to find out before you start. The persona works best when it has precise objectives.
  • Confirm scope after dry run: The persona will typically perform a ‘dry run’ first and inform you how many posts it found based on your request. Pay close attention to this number. If it’s too high (risking high costs or unfocused results) or too low (potentially missing crucial information), discuss refining your filters with the persona before committing to the full analysis.
  • Iterate on filters: If the initial dry run isn’t targeting the right information, work with the persona to adjust filter criteria. Add more specific keywords, narrow date ranges, or specify categories/tags.
  • Consolidate queries: The persona is designed to handle multiple related goals in a single research execution. Try to group related questions into one comprehensive research request to the persona.

Relationship to standard forum helper and related tools

The Forum Researcher persona is distinct from a general Forum Helper that uses standard tools like Search and Read.

  • Standard Search and Read Tools:

    • The Search tool primarily identifies relevant topics. It does this by matching keywords against post content and other criteria (tags, categories, etc.). For each matching topic, it returns a link and a brief snippet from a relevant post, not the full post content.
    • The Read tool is used to access the full content of a specific topic (or selected posts within it) that Search has identified.
    • These tools work in tandem for targeted retrieval: Search finds topics, Read digests their content.

  • Forum Researcher’s researcher Tool:

    • Direct, deep content analysis: The researcher tool doesn’t just identify topics; it directly processes and analyzes the full content of potentially many posts (up to its configured Maximum number of results) that match its comprehensive filter criteria.
    • Advanced filtering and synthesis: It uses a more complex filtering language to build a dataset of posts from across the forum (potentially spanning hundreds of topics), and then synthesizes information from this entire dataset to answer complex questions. This is fundamentally different from reading individual topics one by one.

In essence, while a Forum Helper uses Search to pinpoint topics (presenting snippets) and Read to delve into one, the Forum Researcher conducts broad analysis across the actual text of many posts simultaneously to uncover deeper, synthesized insights.

What LLM should I use?

LLM technology is rapidly evolving, with models continually improving in capability and cost-effectiveness. During the development of the Forum Researcher, models like Gemini 2.5 Flash, Gemini 2.5 Pro, GPT-4.1, and Claude 4 Sonnet provided excellent results for complex research plans.

The best choice depends on your specific needs:

  • High-quality, nuanced analysis: More advanced models might be preferable, though they usually come with higher costs.
  • Broad overviews or cost-sensitive tasks: Faster, more economical models can be very effective.

Here are some point-in-time examples from internal testing at Discourse for a very specific, complex query:

Look at the top 1000 open topics in the feature category - ordered by like (first post only) - all time … make me an executive report of the:

  • Top 20 features CDCK should build
  • Easiest 20 features CDCK could build
  • Obvious duplicates
  • Things that are very poorly defined

ask me no more questions, just run the research

  1. Gemini 2.0 Flash Example
  2. Gemini 2.5 Flash (with thinking) Example
  3. GPT-4.1 Example
  4. Claude 4 Sonnet Example
  5. Gemini 2.5 Pro Example

Hybrid example: Driver is Gemini 2.5 Pro and Researcher LLM is Gemini 2.0 Flash
Hybrid example

Debugging research

In Discourse, you can enable advanced AI debugging by adding groups to the ai_bot_debugging_allowed_groups site setting. With that in place, you are able to see the actual payloads sent to the LLM.

The image shows a user interface for an AI interaction, displaying 'Request tokens: 3596886' and 'Response tokens: 8185'. (Captioned by AI)
The image shows a user interface for an AI interaction, displaying 'Request tokens: 3596886' and 'Response tokens: 8185'. (Captioned by AI)553×233 13.2 KB

The image shows a Documentation section post about a new usercard redesign experiment for meta forums, with updates on user feedback and changes to the user background, header image placement, and button positioning. (Captioned by AI)
The image shows a Documentation section post about a new usercard redesign experiment for meta forums, with updates on user feedback and changes to the user background, header image placement, and button positioning. (Captioned by AI)1040×891 45.5 KB

Limitations

Currently, there is no option to send images to the research LLM. This will be considered in future versions.

FAQs

  • Is the Forum Researcher available on all Discourse plans?
    The Forum Researcher is part of the Discourse AI plugin, which is available for self-hosted sites and on our Enterprise hosting plan.

  • Can the Forum Researcher access content from private categories or messages?
    Yes, if the “Include private” option is enabled in its configuration and the user interacting with the persona has the necessary permissions to access those areas.

  • How can I control the cost of using the Forum Researcher?

    • Limit access to specific, trusted groups.
    • Use the “Maximum number of results” and “Maximum tokens per post” settings to cap processing.
    • Choose cost-effective LLMs.
    • Pay close attention to the “dry run” estimates before executing full research.
    • Utilize AI quotas.

Additional resources

Last edited by @hugh 2025-06-05T05:46:29Z

Last checked by @hugh 2025-06-05T05:46:42Z

Check documentPerform check on document:
17 Mi Piace
2

@sam Ottimo lavoro e molte grazie per i costanti progressi sulle personalitĂ  AI di Discourse: un lavoro davvero impressionante.

Quando sono abilitate diverse personalità, il menu a tendina del compositore può sembrare affollato e confuso per gli utenti. Sto cercando indicazioni sul modo migliore per:

  • L’uso corretto è avere numerose personalitĂ  nel menu a tendina tra cui gli utenti possono scegliere?

  • È possibile che una personalitĂ  predefinita sfrutti personalitĂ  specializzate dietro le quinte?

  • Penso che controllare la visibilitĂ  tramite le autorizzazioni, in modo che le personalitĂ  di supporto rimangano nascoste alla vista degli utenti normali e vengano utilizzate con un’automazione, comporterĂ  la pubblicazione di piĂš post di risposta. Sarebbe fantastico se questi potessero essere usati come strumenti.

Qualsiasi suggerimento di configurazione o esempio di linee guida per l’implementazione sarebbe utile.

2 Mi Piace
3

Ciao ragazzi,

Prima di tutto, ottimo lavoro! È qualcosa che aspettavamo davvero, poter curare tutta la conoscenza nel forum.

Un piccolo problema che abbiamo riscontrato:

  • Dato che il nostro forum è in tedesco, sembra che il LLM abbia tentato di effettuare una ricerca con le virgolette tedesche che appaiono „così“, il che si traduce in risultati di ricerca vuoti. Nota a margine: Abbiamo tradotto il prompt di sistema predefinito di Researcher in tedesco.
1 Mi Piace
4

Quale LLM stai usando? Potrebbe valere la pena copiare la persona e rifare il prompt di sistema in tedesco con suggerimenti

5

L’ho già fatto, ho persino aggiunto un’istruzione aggiuntiva:

- Per parametri di ricerca raffinati nel forum, usa esclusivamente le virgolette ``\"`` e non ``„“``.

Ma il problema persiste. È successo con gpt 4.1 e occasionalmente con gemini 2.5 pro e flash.

A proposito, dove posso trovare maggiori informazioni su come utilizzare i parametri topic_keywords: e keywords:? Non ho trovato nulla né su meta né su ask.discourse.com. Vorrei provare a replicare la ricerca che l’LLM sta cercando di eseguire. Non ottengo risultati di ricerca quando li utilizzo nella ricerca del mio forum (siamo alla versione 3.5.0.beta8-dev).

6

Ho appena riscontrato uno strano comportamento con Gemini 2.5 Researcher:

L’LLM risponde:

Ora compilerò le informazioni da questi e altri contributi per creare la descrizione del ceppo. Ci vorrà un momento. Ti contatterò di nuovo una volta completato.

Ma la risposta è in realtà finita e non continua da qui e deve essere riattivata manualmente per continuare

7

La persona del ricercatore non utilizza l’implementazione di ricerca principale di Discourse, utilizza un’implementazione personalizzata, questa viene analizzata e quindi chiamiamo direttamente la ricerca fulltext.

8

Ah, ho capito. Sarebbe bello avere ancora della documentazione a riguardo per avere un controllo piĂš granulare sul comportamento della ricerca tramite prompt.

1 Mi Piace
9

Questa è una allucinazione al 1000% da parte di un LLM.

Nel corpus dei dati di addestramento questa è una risposta “comune”, quindi se non stiamo attenti può inventarsi questo tipo di cose :frowning:

2 Mi Piace
10

Se abiliti il debug e premi il pulsante (i), verrà mostrata la sezione del prompt in cui specifichiamo la lingua completa del ricercatore all’LLM.

Questa immagine presenta un elenco dettagliato di stringhe di filtro e opzioni logiche per il targeting di contenuti specifici all'interno di un sistema, coprendo criteri come nomi utente, gruppi, tipi di post, parole chiave, argomenti, categorie, tag e intervalli di date. (Sottotitolato dall'IA)
Questa immagine presenta un elenco dettagliato di stringhe di filtro e opzioni logiche per il targeting di contenuti specifici all'interno di un sistema, coprendo criteri come nomi utente, gruppi, tipi di post, parole chiave, argomenti, categorie, tag e intervalli di date. (Sottotitolato dall'IA)635×941 52.9 KB

11

Sebbene pensi che sia fantastico che la persona ricercatore ottenga parametri di ricerca personalizzati e avanzati, questa circostanza rende difficile riprodurre manualmente le query di ricerca e quindi personalizzare o perfezionare il prompt di sistema o effettuare il debug della ricerca quando non restituisce risultati, poichĂŠ non posso utilizzare gli stessi parametri e valori di ricerca nel front-end. Esiste un modo per riprodurre la ricerca personalizzata magari tramite API?

1 Mi Piace
12

Al momento no, ma è un’ottima idea, essenzialmente questo è un tipo di filtro

2 Mi Piace
13

Ottima scrittura, Sam, ed è davvero impressionante che ora possiamo usare Discourse AI per creare facilmente qualcosa come il nostro agente Deep Research!
C’è solo una cosa che mi preoccupa:
image :open_mouth:

3 Mi Piace