Weekly Summary of AI topics

News and Events Forum summaries
130

Overview

This week’s ai conversations on #meta.discourse.org centered on controlling and understanding AI cost + background usage, tightening up security expectations around summarization inputs, and improving UX and integrations across the Discourse AI feature set.

On the cost/ops side, Discourse shipped estimated dollar-based quotas for Discourse AI usage (in addition to token limits), making it easier to budget across providers (see Cost based quotas for Discourse AI and the companion guide Configuring LLM usage quotas in Discourse AI). Admins also compared notes on periodic AI API calls that can occur even when you’re not actively using AI UI features, due to background jobs like embeddings and summary backfills (see Does discourse call AI APIs in the background and Falco’s confirmation).

On the safety front, there was renewed attention on what exactly gets fed into summarization prompts—specifically whether HTML comments being included could form a “hidden channel” for manipulation (see HTML comments are also summarized by AI, Falco’s reply, and follow-up concerns).

Meanwhile, several threads focused on “making it work in the real world”: troubleshooting 500 errors in AI title suggestions and learning where to find audit logs (see Qwen3.7-plus support thread and guidance on ai_api_audit_logs), configuring internal AI endpoints / LiteLLM sidecars (see How to use internal AI endpoints? and the solution write-up here), and self-hosting translation via Ollama’s OpenAI-compatible API (see Self-Hosting an OpenSource LLM for DiscourseAI, model/tokenizer guidance, and translation constraints discussion here).

UX and product polish also got attention: a fix was proposed for the AI title suggestion icon positioning (see AI title suggest icon is placed above the title input field), “inline” AI suggestions raised latency/behavior questions (see Inline integration of AI Suggestions (in Composer) and latency question), and the docked composer for AI bot conversations was re-enabled after being accidentally toggled off (see question and resolution).

Interesting Topics

  • Cost-based quotas for Discourse AI (estimated dollars, not just tokens) in #Announcements #ai: sam announced a new way to cap AI usage by estimated dollar cost per group, to help budgeting across providers (announcement). The feature ties into the existing quota configuration flow (Configuring LLM usage quotas in Discourse AI).

  • AI summarization includes HTML comments (and potential prompt-injection concerns) in #Feature ai #ai-summarize: Ed_S raised the risk that HTML comments could become a “hidden channel” to influence summaries (concern). Falco acknowledged the possibility—especially with smaller/older models—and pointed to system/user prompt separation as a mitigating concept (reply), which Ed_S pushed back on as not being a true mechanism against jailbreaks (follow-up).

  • Qwen3.7-plus setup: title suggestion tool hitting 500s + how to debug with audit logs in Support #ai: bird reported a 500 error when using AI title generation despite other AI tools working (report). Falco recommended querying ai_api_audit_logs for more detailed diagnostics (tip), then clarified how to inspect the newest entry and that the 500 was coming from Discourse (not Qwen) (follow-up). The thread also pointed to image generation docs when asked whether Discourse AI supports it (answer + link, plus Improved Image Generation Support in Discourse AI).

  • Running Discourse AI against internal endpoints (LiteLLM sidecar, internal host allowlists, MCP cleanup) in Support #ai: evantobin described getting internal connectivity working using DISCOURSE_ALLOWED_INTERNAL_HOSTS for a localhost LiteLLM sidecar, and mentioned follow-on work for Vertex AI auth (details). satonotdead shared a full solution using Docker’s internal gateway IP, port mapping, and cleaning up broken MCP tools (solution).

  • Feature request / PR: add {username} template param to AI persona system prompts in #Feature #ai: 42aross proposed adding {username} as an allowlisted template param so an AI persona can reliably identify the current user server-side (instead of having the LLM infer it from topic text/metadata) (request + rationale). They also noted completing the CLA process (follow-up).

  • Docked composer for AI bot conversations: toggled off on Meta, then re-enabled in #Announcements ai #ai-bot: putty asked why Meta no longer showed the docked composer UI (question). keegan replied it was accidentally toggled off and turned it back on, noting the change moving to beta (status update). nicolsdennis followed up with a product question about limiting reply length for bot conversations (question).

  • Inline AI Suggestions in the composer: consistency + latency/behavior questions in #Announcements ai #ai-helper: chapoi discussed making behavior consistent across similar UI entry points and asked for feedback (post). nicolsdennis asked about round-trip latency and whether suggestions are purely based on topic title (question).

  • UX fix: AI title suggest icon positioning in ux #ai: chapoi posted an implementation fix for the title-suggest icon appearing above the title field, asking for confirmation of side effects (fix + PR link).

  • Gemini API key changes: service accounts, migration concerns in #Integrations how-to #ai: m_terenui noted Google’s reported security changes around Gemini API keys, potentially requiring service accounts and migrating old keys, and asked what Discourse expects admins to configure going forward (question). (Thread reference: Set up Gemini API Keys for Discourse AI)

  • Self-hosting Discourse AI translation with Ollama: provider choice, tokenizer/context window, and why “translation endpoints” aren’t a drop-in in #Self-Hosting #ai: mononym asked how to try Discourse AI translation via Ollama and where the newer admin UI settings live (question). Falco suggested using Ollama’s OpenAI-compatible API by selecting OpenAI as the provider (guidance), then provided practical configuration advice (tokenizer choice, context window) and cautioned against unsuitable/old models (details). The thread also covered why Discourse AI translation isn’t designed as a simple LibreTranslate endpoint replacement and why the older Translator plugin can still be used “as-is” (constraints).

Activity

Thanks for reading, and I’ll see you again next week! :slight_smile: