# ¿RAG soportará archivos PDF en el futuro?
**URL:** https://meta.discourse.org/t/will-rag-support-pdf-files-in-the-future/335804
**Category:** Feature
**Tags:** completed, ai, ai-bot
**Created:** [30 Septiembre, 2024 17:35 UTC](https://meta.discourse.org/t/will-rag-support-pdf-files-in-the-future/335804 "2024-09-30T17:35:55Z")
**Posts on this page:** 1
**Showing post:** 11
### Author:  [@MachineScholar](https://meta.discourse.org/u/MachineScholar)
#### Post date: [14 Febrero, 2025 10:19 UTC](https://meta.discourse.org/t/will-rag-support-pdf-files-in-the-future/335804/11 "2025-02-14T10:19:05Z")
Debo decir que este commit se ve bastante hermoso 😍
>
>
> This PR introduces several enhancements and refactorings to the AI Persona and R…AG (Retrieval-Augmented Generation) functionalities within the discourse-ai plugin. Here's a breakdown of the changes:
>
> \*\*1. LLM Model Association for RAG and Personas:\*\*
>
> \- \*\*New Database Columns:\*\* Adds \`rag\_llm\_model\_id\` to both \`ai\_personas\` and \`ai\_tools\` tables. This allows specifying a dedicated LLM for RAG indexing, separate from the persona's primary LLM. Adds \`default\_llm\_id\` and \`question\_consolidator\_llm\_id\` to \`ai\_personas\`.
> \- \*\*Migration:\*\* Includes a migration (\`20250210032345\_migrate\_persona\_to\_llm\_model\_id.rb\`) to populate the new \`default\_llm\_id\` and \`question\_consolidator\_llm\_id\` columns in \`ai\_personas\` based on the existing \`default\_llm\` and \`question\_consolidator\_llm\` string columns, and a post migration to remove the latter.
> \- \*\*Model Changes:\*\* The \`AiPersona\` and \`AiTool\` models now \`belong\_to\` an \`LlmModel\` via \`rag\_llm\_model\_id\`. The \`LlmModel.proxy\` method now accepts an \`LlmModel\` instance instead of just an identifier. \`AiPersona\` now has \`default\_llm\_id\` and \`question\_consolidator\_llm\_id\` attributes.
> \- \*\*UI Updates:\*\* The AI Persona and AI Tool editors in the admin panel now allow selecting an LLM for RAG indexing (if PDF/image support is enabled). The RAG options component displays an LLM selector.
> \- \*\*Serialization:\*\* The serializers (\`AiCustomToolSerializer\`, \`AiCustomToolListSerializer\`, \`LocalizedAiPersonaSerializer\`) have been updated to include the new \`rag\_llm\_model\_id\`, \`default\_llm\_id\` and \`question\_consolidator\_llm\_id\` attributes.
>
> \*\*2. PDF and Image Support for RAG:\*\*
>
> \- \*\*Site Setting:\*\* Introduces a new hidden site setting, \`ai\_rag\_pdf\_images\_enabled\`, to control whether PDF and image files can be indexed for RAG. This defaults to \`false\`.
> \- \*\*File Upload Validation:\*\* The \`RagDocumentFragmentsController\` now checks the \`ai\_rag\_pdf\_images\_enabled\` setting and allows PDF, PNG, JPG, and JPEG files if enabled. Error handling is included for cases where PDF/image indexing is attempted with the setting disabled.
> \- \*\*PDF Processing:\*\* Adds a new utility class, \`DiscourseAi::Utils::PdfToImages\`, which uses ImageMagick (\`magick\`) to convert PDF pages into individual PNG images. A maximum PDF size and conversion timeout are enforced.
> \- \*\*Image Processing:\*\* A new utility class, \`DiscourseAi::Utils::ImageToText\`, is included to handle OCR for the images and PDFs.
> \- \*\*RAG Digestion Job:\*\* The \`DigestRagUpload\` job now handles PDF and image uploads. It uses \`PdfToImages\` and \`ImageToText\` to extract text and create document fragments.
> \- \*\*UI Updates:\*\* The RAG uploader component now accepts PDF and image file types if \`ai\_rag\_pdf\_images\_enabled\` is true. The UI text is adjusted to indicate supported file types.
>
> \*\*3. Refactoring and Improvements:\*\*
>
> \- \*\*LLM Enumeration:\*\* The \`DiscourseAi::Configuration::LlmEnumerator\` now provides a \`values\_for\_serialization\` method, which returns a simplified array of LLM data (id, name, vision\_enabled) suitable for use in serializers. This avoids exposing unnecessary details to the frontend.
> \- \*\*AI Helper:\*\* The \`AiHelper::Assistant\` now takes optional \`helper\_llm\` and \`image\_caption\_llm\` parameters in its constructor, allowing for greater flexibility.
> \- \*\*Bot and Persona Updates:\*\* Several updates were made across the codebase, changing the string based association to a LLM to the new model based.
> \- \*\*Audit Logs:\*\* The \`DiscourseAi::Completions::Endpoints::Base\` now formats raw request payloads as pretty JSON for easier auditing.
> \- \*\*Eval Script:\*\* An evaluation script is included.
>
> \*\*4. Testing:\*\*
>
> \- The PR introduces a new eval system for LLMs, this allows us to test how functionality works across various LLM providers. This lives in \`/evals\`
¿Hay alguna posibilidad de que haya un cronograma para el lanzamiento completo de esta función? Veo que por ahora es una función oculta del sitio.
---
_[View the full topic](https://meta.discourse.org/t/will-rag-support-pdf-files-in-the-future/335804)._