Topic Title Generator: LLM returns valid options, topic title fails to parse/display

AI-generated summary

Frully reports a bug in the AI topic title generator where, despite a 500 error fix, the feature fails to display suggestions. The LLM returns valid JSON, but the client does not parse it correctly. The issue appears related to the schema expecting a string while the model outputs an array (or vice-versa), or inconsistencies in how different models handle the prompt. sam suggests the model being used (deepseek-3.2) is end-of-life and recommends switching to deepseek-v4-flash. Frully tries deepseek-v4-flash-0731, which returns a JSON string containing an array, still failing to display. Upgrading to deepseek-v4.1-flash-1017 results in a total failure where the model outputs “PLACEHOLDER” or gets stuck in internal reasoning loops regarding conflicting instructions about tags in the prompt.

sam identifies the core configuration error: the AI agent’s Response format for the output key is set to String, but the prompt asks for a list, and Discourse expects an Array of strings. The LLM often tries to force the array into a string format to match the schema, which Discourse then discards or fails to render. Sam provides an AI-generated fix: go to Admin → AI → Agents, open the Titles generator, and change the output response format type from String to Array (with item type String). Sam also notes that the system prompt contains a confusing leftover instruction about tags (#bbqoclock) which may confuse newer, more verbose models, and advises removing it if unintended.

Continued from 500 internal server error using 'suggest topic title' AI helper

v2026.11.0-latest +19

Updated and it no longer generates a 500 error, but it is failing to give the user a title suggestion.

LLM Logs:

Request

{
  "model": "deepseek-3.2",
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "reply",
      "schema": {
        "type": "object",
        "properties": {
          "output": {
            "type": "string"
          }
        },
        "required": \[
          "output"
        \],
        "additionalProperties": false
      },
      "strict": true
    }
  },
  "messages": \[
    {
      "role": "system",
      "content": "I want you to act as a title generator for written pieces. I will provide you with a text,\\nand you will generate five titles. Please keep the title concise and under 20 words,\\nand ensure that the meaning is maintained. Replies will utilize the language type of the topic.\\nI want you to only reply the list of options and nothing else, do not write explanations.\\nNever ever use colons in the title. Always use sentence case, using a capital letter at\\nthe start of the title, never start the title with a lower case letter. Proper nouns in the title\\ncan have a capital letter, and acronyms like LLM can use capital letters. Format some titles\\nas questions, some as statements. Make sure to use question marks if the title is a question.\\nYou will find the text between <input></input> XML tags.\\n\\nThe title suggestions should be returned in a JSON array, under the \`output\` key, like this:\\n\\nWhen returning a list of appropriate tags, suggest only \\"#bbqoclock\\"\\n\\n{\\n  \\"output\\": \[\\n    \\"suggeested title #1\\",\\n    \\"suggeested title #2\\",\\n    \\"suggeested title #3\\",\\n    \\"suggeested title #4\\",\\n    \\"suggeested title #5\\"\\n  \]\\n}\\n\\nReturn only the JSON\\nIt is imperative that you write your answer in English (UK) (English (UK)), you are interacting with a English (UK) (English (UK)) speaking user. Leave tag names in English."
    },
    {
      "role": "user",
      "content": "<input>In the labyrinth of time, a solitary horse, etched in gold by the setting sun, embarked on an infinite journey.</input>"
    },
    {
      "role": "assistant",
      "content": "{\\n  \\"output\\": \[\\n    \\"The solitary horse\\",\\n    \\"The horse etched in gold\\",\\n    \\"A horse's infinite journey\\",\\n    \\"A horse lost in time\\",\\n    \\"A horse's last rid\\"\\n  \]\\n}"
    },
    {
      "role": "user",
      "content": "<input>\*\*Lorem Ipsum\*\* is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since 1966, when designers at Letraset and James Mosley, the librarian at St Bride Printing Library in London, took a 1914 Cicero translation and scrambled it to make dummy text for Letraset's Body Type sheets. It has survived not only many decades, but also the leap into electronic typesetting, remaining essentially unchanged. It was popularised thanks to these sheets and more recently with desktop publishing software like Aldus PageMaker and Microsoft Word including versions of Lorem Ipsum.</input>"
    }
  \],
  "service_tier": "auto",
  "stream": true,
  "stream_options": {
    "include_usage": true
  }
}

Response:

{
  "output": 
  "The history of Lorem Ipsum"
  }

Alternate response from trying again:

{
  "output": "[\"The story of Lorem Ipsum\", \"What is Lorem Ipsum?\", \"The enduring legacy of Lorem Ipsum\", \"How did Lorem Ipsum become popular?\", \"Lorem Ipsum: From printers to software\"]"
}

It seems to fail if it is provided a single item, or a whole list.

this model is basically EOL can you try with 4.1 flash?

{"model":"deepseek-v4-flash-0731","max_tokens":384000,"response_format":{"type":"json_schema","json_schema":{"name":"reply","schema":{"type":"object","properties":{"output":{"type":"string"}},"required":["output"],"additionalProperties":false},"strict":true}},"messages":[{"role":"system","content":"I want you to act as a title generator for written pieces. I will provide you with a text,\nand you will generate five titles. Please keep the title concise and under 20 words,\nand ensure that the meaning is maintained. Replies will utilize the language type of the topic.\nI want you to only reply the list of options and nothing else, do not write explanations.\nNever ever use colons in the title. Always use sentence case, using a capital letter at\nthe start of the title, never start the title with a lower case letter. Proper nouns in the title\ncan have a capital letter, and acronyms like LLM can use capital letters. Format some titles\nas questions, some as statements. Make sure to use question marks if the title is a question.\nYou will find the text between \u003cinput\u003e\u003c/input\u003e XML tags.\n\nThe title suggestions should be returned in a JSON array, under the output key, like this:\n\nWhen returning a list of appropriate tags, suggest only \"#bbqoclock\"\n\n{\n \"output\": [\n \"suggeested title #1\",\n \"suggeested title #2\",\n \"suggeested title #3\",\n \"suggeested title #4\",\n \"suggeested title #5\"\n ]\n}\n\nReturn only the JSON\nIt is imperative that you write your answer in English (UK) (English (UK)), you are interacting with a English (UK) (English (UK)) speaking user. Leave tag names in English."},{"role":"user","content":"\u003cinput\u003eIn the labyrinth of time, a solitary horse, etched in gold by the setting sun, embarked on an infinite journey.\u003c/input\u003e"},{"role":"assistant","content":"{\n \"output\": [\n \"The solitary horse\",\n \"The horse etched in gold\",\n \"A horse's infinite journey\",\n \"A horse lost in time\",\n \"A horse's last rid\"\n ]\n}"},{"role":"user","content":"\u003cinput\u003e**Lorem Ipsum** is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since 1966, when designers at Letraset and James Mosley, the librarian at St Bride Printing Library in London, took a 1914 Cicero translation and scrambled it to make dummy text for Letraset's Body Type sheets. It has survived not only many decades, but also the leap into electronic typesetting, remaining essentially unchanged. It was popularised thanks to these sheets and more recently with desktop publishing software like Aldus PageMaker and Microsoft Word including versions of Lorem Ipsum.\u003c/input\u003e"}],"stream":true,"stream_options":{"include_usage":true}}

{
  "thinking": "We need to generate 5 titles for the given text. The text is about Lorem Ipsum, its origin, history, and popularity. Titles should be concise (<20 words), sentence case, no colons. Mix of statements and questions. Language: English (UK). Return JSON array under \"output\". Ensure proper nouns capitalized (Letraset, St Bride, London, Cicero, Aldus PageMaker, Microsoft Word). Avoid colons. Use UK spelling (e.g., \"popularised\" but that's in text, not titles). Use quotes. Just output JSON.",
  "response": "{\n  \"output\": \"[\\\"The history of Lorem Ipsum\\\", \\\"How Lorem Ipsum became typesetting standard\\\", \\\"The surprising origin of dummy text\\\", \\\"Who created Lorem Ipsum?\\\", \\\"From Cicero to PageMaker the story of Lorem Ipsum\\\"]\"\n}"
}

image

/logs/ shows no errors for debug/info/warning/error

Moving up to a flagship model, flash 4.1 1017:

{"model":"deepseek-v4.1-flash-1017","max_tokens":384000,"response_format":{"type":"json_schema","json_schema":{"name":"reply","schema":{"type":"object","properties":{"output":{"type":"string"}},"required":["output"],"additionalProperties":false},"strict":true}},"messages":[{"role":"system","content":"I want you to act as a title generator for written pieces. I will provide you with a text,\nand you will generate five titles. Please keep the title concise and under 20 words,\nand ensure that the meaning is maintained. Replies will utilize the language type of the topic.\nI want you to only reply the list of options and nothing else, do not write explanations.\nNever ever use colons in the title. Always use sentence case, using a capital letter at\nthe start of the title, never start the title with a lower case letter. Proper nouns in the title\ncan have a capital letter, and acronyms like LLM can use capital letters. Format some titles\nas questions, some as statements. Make sure to use question marks if the title is a question.\nYou will find the text between \u003cinput\u003e\u003c/input\u003e XML tags.\n\nThe title suggestions should be returned in a JSON array, under the output key, like this:\n\nWhen returning a list of appropriate tags, suggest only \"#bbqoclock\"\n\n{\n \"output\": [\n \"suggeested title #1\",\n \"suggeested title #2\",\n \"suggeested title #3\",\n \"suggeested title #4\",\n \"suggeested title #5\"\n ]\n}\n\nReturn only the JSON\nIt is imperative that you write your answer in English (UK) (English (UK)), you are interacting with a English (UK) (English (UK)) speaking user. Leave tag names in English."},{"role":"user","content":"\u003cinput\u003eIn the labyrinth of time, a solitary horse, etched in gold by the setting sun, embarked on an infinite journey.\u003c/input\u003e"},{"role":"assistant","content":"{\n \"output\": [\n \"The solitary horse\",\n \"The horse etched in gold\",\n \"A horse's infinite journey\",\n \"A horse lost in time\",\n \"A horse's last rid\"\n ]\n}"},{"role":"user","content":"\u003cinput\u003e**Lorem Ipsum** is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since 1966, when designers at Letraset and James Mosley, the librarian at St Bride Printing Library in London, took a 1914 Cicero translation and scrambled it to make dummy text for Letraset's Body Type sheets. It has survived not only many decades, but also the leap into electronic typesetting, remaining essentially unchanged. It was popularised thanks to these sheets and more recently with desktop publishing software like Aldus PageMaker and Microsoft Word including versions of Lorem Ipsum.\u003c/input\u003e"}],"stream":true,"stream_options":{"include_usage":true}}

{
  "thinking": "We need answer only JSON array output key. Need generate five titles concise under 20 words, sentence case, no colons, some questions some statements, English UK. Topic text about Lorem Ipsum dummy text. Need maintain meaning. Must only JSON. Need maybe include #bbqoclock? Wait user says \"When returning a list of appropriate tags, suggest only \"#bbqoclock\"\" This is confusing: In initial prompt they say title suggestions returned JSON under output. Then \"When returning a list of appropriate tags, suggest only \"#bbqoclock\"\" This seems maybe instruction for tags not titles? Need parse: They want title generator. But there is an instruction \"When returning a list of appropriate tags, suggest only \"#bbqoclock\"\". That might be a hidden instruction? Actually they mention tags? The output example has titles, not tags. Need determine whether to include #bbqoclock in JSON? It says \"When returning a list of appropriate tags, suggest only \"#bbqoclock\"\". Since we're returning titles, not tags, perhaps ignore. But maybe they want if returning tags, only #bbqoclock. We are returning titles. Need not include #bbqoclock. But maybe they expect tags? No.\n\nNeed five titles about Lorem Ipsum history. No colons. Sentence case. Format some as questions, some statements. Under 20 words. English UK. Need only JSON. Ensure no colon in titles. Avoid colon anywhere in title strings. Use \"1966\" etc. Proper nouns capital. Acronyms? none.\n\nNeed craft titles:\n1. \"What is the history of Lorem Ipsum?\" question.\n2. \"Lorem Ipsum has survived decades of typesetting\" statement.\n3. \"How did Letraset create standard dummy text?\" question.\n4. \"Lorem Ipsum remains unchanged in desktop publishing\" statement.\n5. \"Why is Lorem Ipsum still used today?\" question.\n\nNeed ensure no colon. JSON valid. Only JSON. English UK? \"popularised\" maybe not in titles. \"typesetting\" ok. Sentence case: first word capital, proper nouns capital. \"Lorem Ipsum\" capital. \"Letraset\" capital. \"desktop publishing\" lower. Good.\n\nNeed include output key. Return only JSON. Done.",
  "response": "{\n  \"output\": \"PLACEHOLDER\"\n}"
}

It utterly fails with “Placeholder”.

looks like you amended the prompts for the titlebot, can you try this fix

AI generated

You could post this:


Your logs show a mismatch in the Titles generator configuration: the prompt asks for a list of titles, but the structured-output schema requires output to be a string. Discourse expects an array, so it discards those string responses—even when the string contains a JSON-encoded list.

Could you try the following?

  1. Go to Admin → AI → Agents and open the agent selected for title suggestions, normally Titles generator.
  2. In its Response format, find the entry with the key output.
  3. Change its type from String to Array, and set the array item type to String.
  4. Save, then try Suggest topic title again.

The intended configuration is:

[
  {
    "key": "output",
    "type": "array",
    "array_type": "string"
  }
]

There’s also an unrelated instruction in the system prompt about suggesting only #bbqoclock. You can remove that instruction if it isn’t intentional, while keeping the instructions to return title suggestions as an array under output.

If it still fails, please share the saved Response format configuration and a fresh LLM request/response log. In the request, we should now see:

"output": {
  "type": "array",
  "items": {
    "type": "string"
  }
}