is there any tool to split input text into chunks? or which embedding api support more than 8192 tokens? i am using Qwen/Qwen3-Embedding-0.6B Message (15 copies reported) OpenAI Embeddings failed with status: 413 body: {"code":20042,"message":"input must have less than 8192 tokens","data":null} …

If you self-host that same model, it can take up to 32k tokens. It is what we run on our hosting these days. If that’s out of the question, then you need to configure the embeddings model to limit inputs to the maximum allowed on your provider. This way, our AI Bot RAG will split uploaded files int…

how to configure embedding model? in discourse ?or model provider?

i am sure Qwen/Qwen3-Embedding-0.6B support up to 32k tokens and i have configured sequence length 32k [image] [image] but there still are many error of embedding [image] [image] [image]

[圖片] whitewaterdeu: i am sure Qwen/Qwen3-Embedding-0.6B support up to 32k tokens It does if your API providers allows it. [圖片] whitewaterdeu: but there still are many error of embedding If your API provider limits it to 8192, you gotta follow their rules.

嵌入 `input 必须少于 8192 个 token` 的警告与 discourse ai

支持

whitewaterdeu (mocha3901) 2025年十一月2日 12:44 4

我确信 Qwen/Qwen3-Embedding-0.6B 支持高达 32k 的 token
并且我已经配置了 32k 的序列长度

但仍然有很多 embedding 的错误

Error when translate post by ai （Job exception: undefined method `strip' for an instance of DiscourseAi::Completions::StructuredOutput

話題		回覆	觀看
Embedding of post is not being properly truncated in discourse-ai plugin Bug ai	4	481	2023年八月17日
Ai:embeddings:backfill - Handling OpenAI's 400 Error for Excessive Tokens in Embeddings Bug ai	9	1002	2024年三月15日
AI exceeds LLM token thresholds randomly and unpredictably Support ai	3	196	2026年五月6日
Adding Semantic Search feature for our self-hosted discourse site Support ai , ai-search	9	401	2025年三月19日
Gemini API Embedding Configuration Clarification Support ai	3	526	2025年十月16日

嵌入 `input 必须少于 8192 个 token` 的警告与 discourse ai

相關主題