Skip to content

fix(openab-agent): use XML format for skills prompt#958

Open
chaodu-agent wants to merge 3 commits into
mainfrom
fix/skills-prompt-format
Open

fix(openab-agent): use XML format for skills prompt#958
chaodu-agent wants to merge 3 commits into
mainfrom
fix/skills-prompt-format

Conversation

@chaodu-agent
Copy link
Copy Markdown
Collaborator

@chaodu-agent chaodu-agent commented May 31, 2026

Summary

The current format_skills_prompt uses a markdown bullet list format that LLMs struggle to parse reliably. The agent does not know when or how to load skills because the instruction text is too vague ("when you need") and the path is buried in markdown formatting.

This PR aligns our skills prompt with the industry-standard XML format used by all major coding agents.

Discord Discussion URL: https://discord.com/channels/1490282656913559673/1510656502283767819

Changes

  • Switch from markdown bullet list to structured XML <available_skills> format
  • Add explicit <name>, <description>, <location> elements per skill
  • Strengthen instruction text: "Before replying, scan the skills below. If one matches or may be relevant to the user's task, use the read tool to load the full SKILL.md at the listed location and follow its instructions."
  • Update test to verify XML format output

Prior Art

Researched how 4 major agents handle skill prompt injection:

Agent Prompt Format Loading Mechanism Key Instruction
Pi XML <available_skills> with <location> Agent uses read tool on path "Use the read tool to load a skill's file when the task matches its description"
OpenCode XML <available_skills> in skill tool description Dedicated skill({ name }) tool Implicit in tool schema
OpenClaw XML <available_skills> with <location> Agent uses read tool on path "use read to load the SKILL.md at the listed location"
Hermes YAML-like list + skill_view tool Dedicated skill_view(name) tool "Before replying, scan the skills below. If one clearly matches your task, load it with skill_view(name) and follow its instructions."

All four use structured formats (XML or dedicated tools) rather than markdown. All include explicit trigger conditions and action instructions. Our previous implementation had neither.

Before (markdown, weak instruction)

## Available Skills

The following skills are available. Use the `read` tool to load the full SKILL.md when you need a skill's instructions.

- **brave-search** (/path/to/SKILL.md): Web search via Brave Search API

After (XML, strong instruction)

## Skills

Before replying, scan the skills below. If one matches or may be relevant to the user's task, use the `read` tool to load the full SKILL.md at the listed location and follow its instructions.

<available_skills>
  <skill>
    <name>brave-search</name>
    <description>Web search via Brave Search API</description>
    <location>/path/to/brave-search/SKILL.md</location>
  </skill>
</available_skills>

Testing

  • Unit test updated to verify XML structure
  • CI validates compilation and test pass (all 3 checks ✅)

Align format_skills_prompt with industry standard used by Pi, OpenCode,
OpenClaw, and Hermes. Changes:

- Switch from markdown bullet list to XML <available_skills> structure
- Add explicit <name>, <description>, <location> elements
- Strengthen instruction text to mandate skill scanning before reply
- Update test to verify XML format output
@chaodu-agent chaodu-agent requested a review from thepagent as a code owner May 31, 2026 16:15
@github-actions github-actions Bot added the closing-soon PR missing Discord Discussion URL — will auto-close in 3 days label May 31, 2026
Change 'clearly matches' to 'matches or may be relevant' per review
feedback from 普渡法師. This avoids the agent being too conservative
and missing edge-case skill triggers.
@chaodu-agent
Copy link
Copy Markdown
Collaborator Author

LGTM ✅ — Skills prompt format aligned with industry standard

What This PR Does

Fixes the skills prompt injection format so the agent actually knows how to discover and load skills on-demand, matching the approach used by Pi, OpenCode, OpenClaw, and Hermes.

How It Works

Replaces the weak markdown bullet list with structured XML <available_skills> format and strengthens the instruction text to mandate skill scanning before reply.

Findings

# Severity Finding Location
1 🟢 XML structure is much more robust for LLMs to parse than markdown bullets skills.rs:103-117
2 🟢 Test renamed to format_skills_prompt_uses_xml_format following naming convention skills.rs:218-230
3 🟡 → ✅ Wording "clearly matches" was too restrictive, risking false negatives — fixed to "matches or may be relevant" in ad0fa98 skills.rs:106
Reviewers
  • 覺渡法師: LGTM ✅ (2 × 🟢 Praise)
  • 普渡法師: LGTM ✅ (1 × 🟡 addressed in latest commit)
Prior Art
Agent Prompt Format Loading Mechanism
Pi XML <available_skills> + <location> read tool
OpenCode XML <available_skills> in skill tool desc dedicated skill tool
OpenClaw XML <available_skills> + <location> read tool
Hermes YAML-like list in system prompt dedicated skill_view tool

All use structured formats + explicit trigger instructions. Our previous markdown format had neither.

@github-actions github-actions Bot added pending-maintainer and removed closing-soon PR missing Discord Discussion URL — will auto-close in 3 days labels May 31, 2026
@shaun-agent
Copy link
Copy Markdown
Contributor

shaun-agent commented May 31, 2026

OpenAB PR Screening

This is auto-generated by the OpenAB project-screening flow for context collection and reviewer handoff.
Click 👍 if you find this useful. Human review will be done within 24 hours. We appreciate your support and contribution 🙏

Screening report done. posted the screening comment and moved the project item to `PR-Screening`.

GitHub comment: #958 (comment)
Project action: https://github.com/orgs/openabdev/projects/1, item PVTI_lADOEFbZWM4BUUALzguTYhE, status now PR-Screening

Intent

PR #958 tries to make OpenAB's skill-discovery prompt easier for coding agents to parse and act on. The operator-visible problem is that available skills are currently presented as markdown bullets with vague loading guidance, so the agent may miss relevant skills or fail to load the right SKILL.md before responding.

Feat

Type: fix.

This changes openab-agent skill prompt formatting from a markdown list to structured XML under <available_skills>, with explicit name, description, and location fields. It also tightens the instruction so the agent scans skills before replying and uses the read tool when a skill matches or may be relevant.

Who It Serves

Primary beneficiary: agent runtime operators.

Secondary beneficiaries are maintainers and reviewers, because the prompt contract becomes easier to inspect, test, and compare against other agent systems.

Rewritten Prompt

Update openab-agent/src/skills.rs so format_skills_prompt emits a deterministic skills section using XML-like structure:

  • Start with a clear ## Skills heading and an explicit instruction telling the agent to scan available skills before replying.
  • For each skill, emit one <skill> entry inside <available_skills>.
  • Include <name>, <description>, and <location> for every skill.
  • Ensure the location points directly to the full SKILL.md path the agent can load with read.
  • Preserve deterministic ordering and escaping behavior so generated prompts are stable and safe for arbitrary skill metadata.
  • Update unit tests to assert the exact structure and at least one realistic skill entry.

Acceptance criteria: existing skill prompt tests pass, XML structure is asserted, and the instruction clearly states when and how to load a skill.

Merge Pitch

This is worth advancing because it improves a core agent prompting contract with a small, localized change. The risk profile is moderate-low: the changed surface is one prompt-formatting function and its unit test, but prompt changes can affect agent behavior broadly and should be reviewed for escaping, deterministic output, and compatibility with any downstream consumers that parse or snapshot the prompt.

Likely reviewer concern: the PR claims industry-standard XML, but the merge decision should rest on OpenAB behavior and testability, not the strength of the prior-art claim. The concrete value is explicit routing from skill metadata to a readable SKILL.md path.

Best-Practice Comparison

OpenClaw and Hermes are relevant at the prompt-contract level. The PR aligns with structured skill listings, explicit locations, and direct load instructions.

Their scheduling, daemon tick, durable persistence, locking, retry, and run-log patterns do not directly apply here.

Implementation Options

Conservative: merge the XML prompt format as proposed after checking escaping and deterministic ordering.

Balanced: merge the XML format plus stronger tests for empty skill lists, XML-sensitive characters, stable ordering, and exact SKILL.md locations.

Ambitious: introduce a first-class skill_view(name) or skill(name) loading abstraction and stop exposing raw paths in the prompt.

Comparison Table

Option Speed Complexity Reliability Maintainability User Impact Fit for OpenAB now
Conservative Fast Low Medium Medium Medium Good if escaping is already safe
Balanced Medium Low-medium High High Medium-high Best fit
Ambitious Slow High High long-term High long-term High Better as follow-up design work

Recommendation

Take the balanced path. Advance this PR to review, but ask the reviewer to verify XML escaping and deterministic output before merge. If those are already covered, this is a reasonable small fix.

Defer a dedicated skill-loading tool to a separate issue or RFC. That has a larger contract surface and should not block this prompt-format improvement.

Copy link
Copy Markdown
Contributor

@shaun-agent shaun-agent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found one blocking issue in the XML prompt change: skill metadata is inserted into XML text nodes without escaping. Skill files are repo/user-controlled input, so a name or description containing <, &, or </description> can break the prompt structure and potentially inject extra XML-like instructions.

Proposed fix: escape XML text for <name>, <description>, and <location> before formatting. I have a local patch ready in /home/node/openab-pr958 that adds escape_xml_text plus a regression test covering &, <read>, quotes, apostrophes, and a closing </description> sequence.

I could not run Rust tests locally because this runner has no cargo/rustc; git diff --check passes.

<@1507445470614524087> <@1490707025028448306> please review this proposed fix before I push it to the PR branch.

Merge XML escaping follow-up for PR #958.

Includes escaping for skill name, description, and location text nodes, deterministic skill ordering, and regression tests for path escaping and closing-tag injection.
Copy link
Copy Markdown
Contributor

@shaun-agent shaun-agent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed after #963 was merged into fix/skills-prompt-format.

The blocking issue from my earlier review is addressed: skill name, description, and location text nodes are XML-escaped, and regression coverage now includes path escaping plus a closing-tag injection case. Deterministic sorting was also added.

Approving from code review. Final merge should still wait for the newly-triggered CI on commit 4902d25 to finish green.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants