sam
3 Abril, 2025 03:56
11
No he publicado en mucho tiempo, a pesar de visitar mi pequeña ventana de chat a diario y que esta fuera útil al menos una o dos veces al día… constantemente .
La razón de mi retraso aquí fue que tuve que trabajar en este cambio bastante grande.
main ← better_upload_support
opened 07:22AM - 25 Mar 25 UTC
**1. What Led to the Change? (Problems with Previous Approach)**
* **Incons… istent Context Handling:** The previous system often passed context information (like `post_id`, `user`, `private_message`, `topic_id`, `custom_instructions`) around using plain Ruby hashes (`context: {}`). This approach lacked structure, was potentially error-prone (typos in keys), and made it harder to track what context was available or required in different parts of the AI Bot system (Tools, Personas, Bot logic). Accessing context often involved `context[:key]`.
* **Inflexible Image/Upload Handling:** Images associated with a user message were previously passed using a separate `upload_ids: [...]` array within the message hash. This made it difficult or impossible to represent prompts where text and images are interleaved naturally (e.g., "Describe this image {image1}, then compare it to this one {image2} and tell me the difference"). The LLM received the text and a list of associated image IDs, but not their precise relationship *within* the user's text flow.
* **Complex/Decentralized Context Building:** Logic for assembling conversation history (e.g., pulling previous posts/messages, handling custom prompts, associating uploads) was somewhat spread out, notably seen in the significant changes and removals within `lib/ai_bot/playground.rb` (specifically the `conversation_context` and `chat_context` logic being refactored).
**2. What New Support Does It Add? (Key Changes & Benefits)**
* **Introduction of `DiscourseAi::AiBot::BotContext`:**
* **What:** A dedicated class (`BotContext`) is introduced to encapsulate all contextual information for an AI Bot interaction. This includes messages, post/topic details, user information, site details (URL, title, description), time, participants, and control flags (like `skip_tool_details`).
* **Why:** Provides a structured, standardized, and object-oriented way to manage and pass context. This improves code readability, maintainability, and reduces the chance of errors compared to using unstructured hashes. Access changes from `context[:key]` to `context.key`.
* **Impact:** This class is now used consistently when initializing Tools (`Tool#initialize`), crafting prompts (`Persona#craft_prompt`), invoking the bot (`Bot#reply`), and within various helper methods, ensuring a uniform context object is available throughout the system.
* **Enhanced Multimodal Input (Inline Images/Uploads):**
* **What:** The format for representing user messages with uploads has fundamentally changed. Instead of a separate `upload_ids` array, uploads are now embedded directly *within* the `content` field, which becomes an array if uploads are present. Example: `content: ["Here is an image:", { upload_id: 123 }, "What do you see?"]`.
* **Why:** This allows for precise interleaving of text and visual elements within a single user turn. It's a much more natural way to represent multimodal prompts for vision-capable LLMs, enabling more complex instructions involving multiple images referenced at specific points in the text.
* **Impact:** Required changes across multiple components:
* **`Prompt` Class:** Logic for handling uploads (`encoded_uploads`, `encode_upload`, `content_with_encoded_uploads`, `text_only`) was refactored to support this new inline structure. Validation was updated.
* **LLM Dialects:** All relevant dialects (`ChatGpt`, `Claude`, `Gemini`, `Mistral`, `Nova`, `Ollama`, `OpenAiCompatible`) were updated to correctly parse the new `content` array format and translate it into the specific structure required by each respective LLM API (e.g., OpenAI's array of text/image_url objects, Gemini's parts array). A helper `to_encoded_content_array` was added to the base `Dialect` class.
* **Modules Using Vision:** Code that passes uploads to LLMs (e.g., `LlmTriage`, `Assistant`, `SpamScanner`, `Playground`) was updated to use the new `content` format.
* **Refactored Context Building:**
* **What:** Logic for building conversation history from posts or chat messages seems to be increasingly centralized in `DiscourseAi::Completions::PromptMessagesBuilder`. New methods like `messages_from_post` and `messages_from_chat` appear to encapsulate this logic.
* **Why:** Simplifies components like the `Playground` by abstracting away the details of fetching and formatting conversation history, including handling the new inline upload format.
* **Impact:** Significant simplification in `lib/ai_bot/playground.rb`, removing large chunks of previous context-building code.
Proporciona una mejora sutil, pero crítica, para Discourse AI.
Notaba regularmente que el bot de moderación hablaba de imágenes completamente irrelevantes, debido a la forma en que construimos el contexto. El cambio nos permite presentar contenido mixto (que contiene imágenes y texto en un orden correcto).
Esto significa que el LLM ya no se confunde.
No tenemos forma de automatizar para que llame a una regla después de que la edición de la publicación se haya “asentado”, las llamadas a LLM pueden ser costosas, solo porque la gente edita errores tipográficos no queremos escanear algo una y otra vez. No estoy seguro de si esto es necesario aquí, pero me gustaría permitir la posibilidad de activar una automatización una vez que una publicación se asiente en su nueva forma.
Ingeniería de prompts: el prompt actual está bien, pero es un poco ruidoso para mi gusto, me molesta demasiado, puede que lo suavice un poco.
Contexto mejorado: una cosa que realmente me molesta es que la automatización ahora no tiene en cuenta la confianza del usuario. Algunos usuarios son mucho más confiables en una comunidad que otros (por ejemplo, moderadores), me gustaría ver si podemos mejorar esta situación.
Capacidad de ejecutar la automatización en lotes de publicaciones para iteraciones rápidas.
Estoy seguro de que surgirán muchas más cosas.
8 Me gusta