Giving Discourse AI agent skills

Customization Extras
1

A very common pattern in the industry that is used across a wide variety of agents is Agent Skills

Skills solve a context problem for agents.

Instead of stuffing all of the instructions and ideas you have into one giant system prompt, agent skills allow for progressive disclosure.

Your agent knows what it can do and calls a very specific tool to find how to do it.

This offers significant benefits on Discourse because we can keep the skill catalog in topics and get a nice editor, history and permissions.

Using custom tools in Discourse AI we can approximate this system due to two very powerful features they provide.

  • Custom tools allow modification of system prompt
  • Custom tools can read topics and posts in particular categories.

Our Goal

To approximate skills on Discourse we would like

  1. One topic per skill
  2. Automatic available skill directory in system prompt

So for example, we would like a topic of this shape (raw):

---
name: doc-coauthoring
description: Guide users through a structured workflow for co-authoring documentation. Use when the user wants to write a proposal, technical specification, decision record, or similar document.
---

# Document co-authoring workflow

1. Gather context from the user.
2. Agree on an outline.
3. Draft and refine each section.
4. Test the document from a new reader's perspective.

To be translated into the following in the system prompt:

<available_skills>
  <skill>
    <name>doc-coauthoring</name>
    <description>Guide users through a structured workflow for co-authoring documentation...</description>
    <location>https://example.com/t/skill-doc-coauthoring/123</location>
  </skill>
  ... more skills here ...
</available_skills>

We choose this particular format because many language models are already fine tuned to look for this specific shape, which can improve recall. Notice how the body of the actual skill is absent.

Then if the model detects that a user is trying to co-author a document workflow, it will call the custom tool load_skill to get the actual instructions.

With this in place, we can define a dedicated category on Discourse that contains skills we want our agents to use and allow community members to iterate on them.

How it works?

The entire logic of load_skill is authored in JavaScript, using a custom tool.

discourse.filterTopics can be called to look for topics in a particular category
discourse.getPost can be called to get a post body
customSystemMessage is used to inject a custom system prompt

With these building blocks

First we define a new custom tool:

  • name: load_skill
  • params:
    • “name”: “The name of the skill from available_skills”

Next we teach skill parsing:

var SKILLS_CATEGORY_SLUG = "agent-skills";
var SKILL_NAME_PATTERN = /^[a-z0-9]+(?:-[a-z0-9]+)*$/;

function skillTopics() {
  var result = discourse.filterTopics({
    q: "category:" + SKILLS_CATEGORY_SLUG + " order:created",
    limit: 200,
  });

  return result && Array.isArray(result.topics) ? result.topics : [];
}

function topicOp(topic) {
  if (!topic || !topic.first_post_id) {
    return null;
  }

  return discourse.getPost(topic.first_post_id);
}

function frontmatter(raw) {
  if (typeof raw !== "string") {
    return null;
  }

  var match = raw.match(
    /^---[ \t]*\n([\s\S]*?)\n---(?:[ \t]*\n|$)/
  );

  return match ? match[1] : null;
}

function scalarField(yaml, field) {
  var match = yaml.match(
    new RegExp("^" + field + ":[ \\t]*(.+)[ \\t]*$", "m")
  );

  return match ? match[1].trim() : null;
}

function xmlEscape(value) {
  return String(value)
    .replace(/&/g, "&amp;")
    .replace(/</g, "&lt;")
    .replace(/>/g, "&gt;")
    .replace(/\"/g, "&quot;")
    .replace(/'/g, "&apos;");
}

function skillRecord(topic) {
  var post = topicOp(topic);
  var yaml = post && frontmatter(post.raw);

  if (!yaml) {
    return null;
  }

  var name = scalarField(yaml, "name");
  var description = scalarField(yaml, "description");

  if (
    !name ||
    !description ||
    !SKILL_NAME_PATTERN.test(name) ||
    name.length > 64 ||
    description.length > 1024
  ) {
    return null;
  }

  return {
    name: name,
    description: description,
    location: discourse.baseUrl + topic.url,
    content: post.raw,
  };
}

function availableSkills() {
  var records = [];
  var topics = skillTopics();

  for (var i = 0; i < topics.length; i++) {
    var record = skillRecord(topics[i]);
    if (record) {
      records.push(record);
    }
  }

  records.sort(function (a, b) {
    return a.name.localeCompare(b.name);
  });

  return records;
}

Be sure to set SKILLS_CATEGORY_SLUG to the actual slug of the category containing your skills.

Next we use customSystemMessage to inject the system prompt

function customSystemMessage() {
  var skills = availableSkills();

  if (skills.length === 0) {
    return null;
  }

  var lines = [
    "Skills provide specialized instructions and workflows for specific tasks.",
    "Use the load_skill tool to load a skill when a task matches its description.",
    "<available_skills>",
  ];

  skills.forEach(function (skill) {
    lines.push("  <skill>");
    lines.push("    <name>" + xmlEscape(skill.name) + "</name>");
    lines.push(
      "    <description>" +
        xmlEscape(skill.description) +
        "</description>"
    );
    lines.push(
      "    <location>" + xmlEscape(skill.location) + "</location>"
    );
    lines.push("  </skill>");
  });

  lines.push("</available_skills>");
  return lines.join("\n");
}

Finally, we build the skill loader which goes from slug to content of skill.

function invoke(parameters) {
  var requestedName = parameters && parameters.name;

  if (typeof requestedName !== "string") {
    return "No skill name was provided.";
  }

  requestedName = requestedName.trim();

  if (!SKILL_NAME_PATTERN.test(requestedName)) {
    return "Skill names must use lowercase letters, numbers, and hyphens.";
  }

  var matches = availableSkills().filter(function (skill) {
    return skill.name === requestedName;
  });

  if (matches.length === 0) {
    return "No available skill has the name " + requestedName + ".";
  }

  if (matches.length > 1) {
    return "More than one skill has the name " + requestedName + ".";
  }

  var skill = matches[0];

  return [
    '<skill_content name="' + xmlEscape(skill.name) + '">',
    "# Skill: " + skill.name,
    "",
    skill.content.trim(),
    "",
    "Source topic for this skill: " + skill.location,
    "</skill_content>",
  ].join("\n");
}

A full working agent + custom tool json is available here, you can import into your instance and amend as you see fit:

category-skills-agent.json (5.9 KB)

Seeing this in action

image
image992×1460 182 KB

image
image1446×1564 144 KB

image
image1100×1715 189 KB

Limitations

The current demo still has an N+1 call loading every post with a skill on every user turn. At scale this may not be advisable.

The design could be amended so a skill catalog post is automatically generated using a workflow or perhaps filterTopics could be expanded to allow for a call asking for prefix.

Additionally agent skills support resource and scripts, resources could be approximated by allow the “read” tool and linking to specific topics from your skill. Shell execution is clearly out of scope for Discourse.

Hopefully you find this helpful

3 Likes