Topic Title Generator: LLM returns valid options, topic title fails to parse/display

Continued from 500 internal server error using 'suggest topic title' AI helper

v2026.11.0-latest +19

Updated and it no longer generates a 500 error, but it is failing to give the user a title suggestion.

LLM Logs:

Request

{
  "model": "deepseek-3.2",
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "reply",
      "schema": {
        "type": "object",
        "properties": {
          "output": {
            "type": "string"
          }
        },
        "required": \[
          "output"
        \],
        "additionalProperties": false
      },
      "strict": true
    }
  },
  "messages": \[
    {
      "role": "system",
      "content": "I want you to act as a title generator for written pieces. I will provide you with a text,\\nand you will generate five titles. Please keep the title concise and under 20 words,\\nand ensure that the meaning is maintained. Replies will utilize the language type of the topic.\\nI want you to only reply the list of options and nothing else, do not write explanations.\\nNever ever use colons in the title. Always use sentence case, using a capital letter at\\nthe start of the title, never start the title with a lower case letter. Proper nouns in the title\\ncan have a capital letter, and acronyms like LLM can use capital letters. Format some titles\\nas questions, some as statements. Make sure to use question marks if the title is a question.\\nYou will find the text between <input></input> XML tags.\\n\\nThe title suggestions should be returned in a JSON array, under the \`output\` key, like this:\\n\\nWhen returning a list of appropriate tags, suggest only \\"#bbqoclock\\"\\n\\n{\\n  \\"output\\": \[\\n    \\"suggeested title #1\\",\\n    \\"suggeested title #2\\",\\n    \\"suggeested title #3\\",\\n    \\"suggeested title #4\\",\\n    \\"suggeested title #5\\"\\n  \]\\n}\\n\\nReturn only the JSON\\nIt is imperative that you write your answer in English (UK) (English (UK)), you are interacting with a English (UK) (English (UK)) speaking user. Leave tag names in English."
    },
    {
      "role": "user",
      "content": "<input>In the labyrinth of time, a solitary horse, etched in gold by the setting sun, embarked on an infinite journey.</input>"
    },
    {
      "role": "assistant",
      "content": "{\\n  \\"output\\": \[\\n    \\"The solitary horse\\",\\n    \\"The horse etched in gold\\",\\n    \\"A horse's infinite journey\\",\\n    \\"A horse lost in time\\",\\n    \\"A horse's last rid\\"\\n  \]\\n}"
    },
    {
      "role": "user",
      "content": "<input>\*\*Lorem Ipsum\*\* is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since 1966, when designers at Letraset and James Mosley, the librarian at St Bride Printing Library in London, took a 1914 Cicero translation and scrambled it to make dummy text for Letraset's Body Type sheets. It has survived not only many decades, but also the leap into electronic typesetting, remaining essentially unchanged. It was popularised thanks to these sheets and more recently with desktop publishing software like Aldus PageMaker and Microsoft Word including versions of Lorem Ipsum.</input>"
    }
  \],
  "service_tier": "auto",
  "stream": true,
  "stream_options": {
    "include_usage": true
  }
}

Response:

{
  "output": 
  "The history of Lorem Ipsum"
  }

Alternate response from trying again:

{
  "output": "[\"The story of Lorem Ipsum\", \"What is Lorem Ipsum?\", \"The enduring legacy of Lorem Ipsum\", \"How did Lorem Ipsum become popular?\", \"Lorem Ipsum: From printers to software\"]"
}

It seems to fail if it is provided a single item, or a whole list.

this model is basically EOL can you try with 4.1 flash?

{"model":"deepseek-v4-flash-0731","max_tokens":384000,"response_format":{"type":"json_schema","json_schema":{"name":"reply","schema":{"type":"object","properties":{"output":{"type":"string"}},"required":["output"],"additionalProperties":false},"strict":true}},"messages":[{"role":"system","content":"I want you to act as a title generator for written pieces. I will provide you with a text,\nand you will generate five titles. Please keep the title concise and under 20 words,\nand ensure that the meaning is maintained. Replies will utilize the language type of the topic.\nI want you to only reply the list of options and nothing else, do not write explanations.\nNever ever use colons in the title. Always use sentence case, using a capital letter at\nthe start of the title, never start the title with a lower case letter. Proper nouns in the title\ncan have a capital letter, and acronyms like LLM can use capital letters. Format some titles\nas questions, some as statements. Make sure to use question marks if the title is a question.\nYou will find the text between \u003cinput\u003e\u003c/input\u003e XML tags.\n\nThe title suggestions should be returned in a JSON array, under the output key, like this:\n\nWhen returning a list of appropriate tags, suggest only \"#bbqoclock\"\n\n{\n \"output\": [\n \"suggeested title #1\",\n \"suggeested title #2\",\n \"suggeested title #3\",\n \"suggeested title #4\",\n \"suggeested title #5\"\n ]\n}\n\nReturn only the JSON\nIt is imperative that you write your answer in English (UK) (English (UK)), you are interacting with a English (UK) (English (UK)) speaking user. Leave tag names in English."},{"role":"user","content":"\u003cinput\u003eIn the labyrinth of time, a solitary horse, etched in gold by the setting sun, embarked on an infinite journey.\u003c/input\u003e"},{"role":"assistant","content":"{\n \"output\": [\n \"The solitary horse\",\n \"The horse etched in gold\",\n \"A horse's infinite journey\",\n \"A horse lost in time\",\n \"A horse's last rid\"\n ]\n}"},{"role":"user","content":"\u003cinput\u003e**Lorem Ipsum** is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since 1966, when designers at Letraset and James Mosley, the librarian at St Bride Printing Library in London, took a 1914 Cicero translation and scrambled it to make dummy text for Letraset's Body Type sheets. It has survived not only many decades, but also the leap into electronic typesetting, remaining essentially unchanged. It was popularised thanks to these sheets and more recently with desktop publishing software like Aldus PageMaker and Microsoft Word including versions of Lorem Ipsum.\u003c/input\u003e"}],"stream":true,"stream_options":{"include_usage":true}}

{
  "thinking": "We need to generate 5 titles for the given text. The text is about Lorem Ipsum, its origin, history, and popularity. Titles should be concise (<20 words), sentence case, no colons. Mix of statements and questions. Language: English (UK). Return JSON array under \"output\". Ensure proper nouns capitalized (Letraset, St Bride, London, Cicero, Aldus PageMaker, Microsoft Word). Avoid colons. Use UK spelling (e.g., \"popularised\" but that's in text, not titles). Use quotes. Just output JSON.",
  "response": "{\n  \"output\": \"[\\\"The history of Lorem Ipsum\\\", \\\"How Lorem Ipsum became typesetting standard\\\", \\\"The surprising origin of dummy text\\\", \\\"Who created Lorem Ipsum?\\\", \\\"From Cicero to PageMaker the story of Lorem Ipsum\\\"]\"\n}"
}

image

/logs/ shows no errors for debug/info/warning/error

Moving up to a flagship model, flash 4.1 1017:

{"model":"deepseek-v4.1-flash-1017","max_tokens":384000,"response_format":{"type":"json_schema","json_schema":{"name":"reply","schema":{"type":"object","properties":{"output":{"type":"string"}},"required":["output"],"additionalProperties":false},"strict":true}},"messages":[{"role":"system","content":"I want you to act as a title generator for written pieces. I will provide you with a text,\nand you will generate five titles. Please keep the title concise and under 20 words,\nand ensure that the meaning is maintained. Replies will utilize the language type of the topic.\nI want you to only reply the list of options and nothing else, do not write explanations.\nNever ever use colons in the title. Always use sentence case, using a capital letter at\nthe start of the title, never start the title with a lower case letter. Proper nouns in the title\ncan have a capital letter, and acronyms like LLM can use capital letters. Format some titles\nas questions, some as statements. Make sure to use question marks if the title is a question.\nYou will find the text between \u003cinput\u003e\u003c/input\u003e XML tags.\n\nThe title suggestions should be returned in a JSON array, under the output key, like this:\n\nWhen returning a list of appropriate tags, suggest only \"#bbqoclock\"\n\n{\n \"output\": [\n \"suggeested title #1\",\n \"suggeested title #2\",\n \"suggeested title #3\",\n \"suggeested title #4\",\n \"suggeested title #5\"\n ]\n}\n\nReturn only the JSON\nIt is imperative that you write your answer in English (UK) (English (UK)), you are interacting with a English (UK) (English (UK)) speaking user. Leave tag names in English."},{"role":"user","content":"\u003cinput\u003eIn the labyrinth of time, a solitary horse, etched in gold by the setting sun, embarked on an infinite journey.\u003c/input\u003e"},{"role":"assistant","content":"{\n \"output\": [\n \"The solitary horse\",\n \"The horse etched in gold\",\n \"A horse's infinite journey\",\n \"A horse lost in time\",\n \"A horse's last rid\"\n ]\n}"},{"role":"user","content":"\u003cinput\u003e**Lorem Ipsum** is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since 1966, when designers at Letraset and James Mosley, the librarian at St Bride Printing Library in London, took a 1914 Cicero translation and scrambled it to make dummy text for Letraset's Body Type sheets. It has survived not only many decades, but also the leap into electronic typesetting, remaining essentially unchanged. It was popularised thanks to these sheets and more recently with desktop publishing software like Aldus PageMaker and Microsoft Word including versions of Lorem Ipsum.\u003c/input\u003e"}],"stream":true,"stream_options":{"include_usage":true}}

{
  "thinking": "We need answer only JSON array output key. Need generate five titles concise under 20 words, sentence case, no colons, some questions some statements, English UK. Topic text about Lorem Ipsum dummy text. Need maintain meaning. Must only JSON. Need maybe include #bbqoclock? Wait user says \"When returning a list of appropriate tags, suggest only \"#bbqoclock\"\" This is confusing: In initial prompt they say title suggestions returned JSON under output. Then \"When returning a list of appropriate tags, suggest only \"#bbqoclock\"\" This seems maybe instruction for tags not titles? Need parse: They want title generator. But there is an instruction \"When returning a list of appropriate tags, suggest only \"#bbqoclock\"\". That might be a hidden instruction? Actually they mention tags? The output example has titles, not tags. Need determine whether to include #bbqoclock in JSON? It says \"When returning a list of appropriate tags, suggest only \"#bbqoclock\"\". Since we're returning titles, not tags, perhaps ignore. But maybe they want if returning tags, only #bbqoclock. We are returning titles. Need not include #bbqoclock. But maybe they expect tags? No.\n\nNeed five titles about Lorem Ipsum history. No colons. Sentence case. Format some as questions, some statements. Under 20 words. English UK. Need only JSON. Ensure no colon in titles. Avoid colon anywhere in title strings. Use \"1966\" etc. Proper nouns capital. Acronyms? none.\n\nNeed craft titles:\n1. \"What is the history of Lorem Ipsum?\" question.\n2. \"Lorem Ipsum has survived decades of typesetting\" statement.\n3. \"How did Letraset create standard dummy text?\" question.\n4. \"Lorem Ipsum remains unchanged in desktop publishing\" statement.\n5. \"Why is Lorem Ipsum still used today?\" question.\n\nNeed ensure no colon. JSON valid. Only JSON. English UK? \"popularised\" maybe not in titles. \"typesetting\" ok. Sentence case: first word capital, proper nouns capital. \"Lorem Ipsum\" capital. \"Letraset\" capital. \"desktop publishing\" lower. Good.\n\nNeed include output key. Return only JSON. Done.",
  "response": "{\n  \"output\": \"PLACEHOLDER\"\n}"
}

It utterly fails with “Placeholder”.

looks like you amended the prompts for the titlebot, can you try this fix

AI generated

You could post this:


Your logs show a mismatch in the Titles generator configuration: the prompt asks for a list of titles, but the structured-output schema requires output to be a string. Discourse expects an array, so it discards those string responses—even when the string contains a JSON-encoded list.

Could you try the following?

  1. Go to Admin → AI → Agents and open the agent selected for title suggestions, normally Titles generator.
  2. In its Response format, find the entry with the key output.
  3. Change its type from String to Array, and set the array item type to String.
  4. Save, then try Suggest topic title again.

The intended configuration is:

[
  {
    "key": "output",
    "type": "array",
    "array_type": "string"
  }
]

There’s also an unrelated instruction in the system prompt about suggesting only #bbqoclock. You can remove that instruction if it isn’t intentional, while keeping the instructions to return title suggestions as an array under output.

If it still fails, please share the saved Response format configuration and a fresh LLM request/response log. In the request, we should now see:

"output": {
  "type": "array",
  "items": {
    "type": "string"
  }
}

Thank you for the attention in this.

Edit: Now that I have changed back to the default topic titles generator, the feature is not showing up in the context menu any more.

May or may not be something that needs fixing: I deleted my custom title generator agent before updating the ai_helper setting. The setting ended up pointing to agent ‘1’ now defunct - instead of failing back to a known default. With that agent deleted it threw a 500 server error. Now the UI element is just gone. Hoping a restart will bring the ui element back.