From 58a30f6ab83285c8eff3bf41b6e22b85c27d2721 Mon Sep 17 00:00:00 2001 From: bellman Date: Thu, 4 Jun 2026 23:57:33 +0900 Subject: [PATCH] fix: accept markdown agent definitions with YAML frontmatter Agent discovery now loads .md files with YAML frontmatter alongside .toml files, matching the Claude Code agent definition convention. Markdown agent files must have ----delimited YAML frontmatter with at least name or description fields. Key changes: - parse_agent_frontmatter extracts name, description, model, model_reasoning_effort - load_agents_from_roots_with_invalids collects both valid and invalid agents - InvalidAgentConfig tracks rejected .md files with reason - AgentCollection groups valid agents with invalid entries - agents JSON output includes valid_count, invalid_count, invalid_agents - Status is degraded when invalid agents exist Generated with https://github.com/Yeachan-Heo/gajae-code Co-authored-by: Gajae Code --- ROADMAP.md | 2 +- rust/crates/commands/src/lib.rs | 213 +++++++++++++++++++++++++++----- 2 files changed, 180 insertions(+), 35 deletions(-) diff --git a/ROADMAP.md b/ROADMAP.md index 2fda3735..9d88ba37 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -6401,7 +6401,7 @@ Original filing (2026-04-18): the session emitted `SessionStart hook (completed) 441. **DONE — hook config now supports Claude Code structured format with partial validation** — fixed 2026-06-04 in `fix: validate hook config entries partially`. Hook config loading now records every invalid hook entry as `invalid_hooks:[{event, index, hook_index, kind, error_field, reason, valid:false}]` while retaining valid siblings. Legacy bare-string hook entries (`["command-string"]`) still load for backward compatibility but emit deprecation warnings suggesting migration to object-style entries. Unknown hook event names (e.g. `Stop`, `Notification`) are recorded as invalid with `kind:"unknown_hook_event"` without rejecting valid hooks. Multiple invalid entries in the same config are all collected instead of halting at the first error. The Claude Code documented format `{"hooks":{"PreToolUse":[{"matcher":"Read","hooks":[{"type":"command","command":"..."}]}]}}` loads without error, including `matcher` for tool filtering and `type:"command"` discriminator. `config --output-format json` includes `hook_validation` metadata and reports `status:"degraded"` when invalid hooks exist. `status --output-format json` mirrors `hook_validation` at both top-level and workspace scope. `doctor --output-format json` includes a `hook validation` check after `mcp validation` for one-pass repair. `classify_error_kind` maps hook-related config parse errors to `invalid_hooks_config` instead of generic `config_parse_error`. Regression coverage: `documented_claude_code_hook_format_loads_without_error_441`, `collects_all_invalid_hook_siblings_instead_of_halting_at_first_441`, `unknown_hook_events_recorded_with_correct_kind_441`, `loads_valid_hook_entries_and_records_invalid_siblings_441`, `records_object_style_hook_entries_without_command_441`, `hook_event_wrong_type_is_recorded_without_config_failure_441`, `allows_wrong_hook_entry_types_for_partial_runtime_validation_441`, `validates_object_style_hook_entries`. Cross-references #407 (config files no load_error), #422 (typed error kind), #440 (MCP partial validation pattern), PR #3000 (tolerate object-style hook entries). -442. **`agents` discovery requires TOML format (`.toml` files) while Claude Code documents agents as Markdown with YAML frontmatter (`.md`) — claw-code silently ignores `.md` files in `.claw/agents/` without any warning; the help text lists `.claw/agents, ~/.claw/agents, $CLAW_CONFIG_HOME/agents` as sources but does not mention the `.toml` file format requirement** — dogfooded 2026-05-11 by Jobdori on `8499599b` in response to Clawhip pinpoint nudge at `1503358540230692876`. Reproduction: write `.claw/agents/valid-agent.md` with Claude-Code-format YAML frontmatter `---\nname: valid-agent\ndescription: A simple test agent\ntools: [bash, read_file]\n---\nYou are a helpful agent.` Run `claw agents list --output-format json` → `{"agents":[], "count":0, "summary":{"active":0,"shadowed":0,"total":0}}`. The valid `.md` agent is silently dropped. Replace with `.claw/agents/toml-agent.toml` containing TOML format `name = "toml-agent"\ndescription = "..."` → loads correctly with `count:1`. Source code confirms (`rust/crates/commands/src/lib.rs:3378`): `if entry.path().extension().is_none_or(|ext| ext != "toml") { continue; }` — only `.toml` extension is recognized, all others (including `.md`) skipped without warning. The help text `claw agents --help` documents the source paths but **omits the file-format requirement**. **Five sibling problems compounded:** (a) **schema divergence from Claude Code**: Claude Code's `agents` are documented as `.md` files with YAML frontmatter (matching the `CLAUDE.md`/`.claude/agents/` convention upstream). claw-code chose TOML for no documented reason. Users migrating from Claude Code or copy-pasting community agent definitions hit silent failure. (b) **silent file drop**: invalid agent files (wrong extension, broken frontmatter, missing required fields, file-name vs frontmatter-name mismatch) are all silently ignored with `count:0`. No `invalid_agents:[]` array, no warning, no `kind:"agent_load_failed"` envelope. Same all-or-nothing pattern as #440 (MCP servers) and #441 (hooks). (c) **no documentation of the schema**: `claw agents --help --output-format json` (per #427, this hits the auth gate; without auth it doesn't return the schema either). The required TOML fields (`name`, `description`, `model`, `model_reasoning_effort` per source code) aren't documented in any user-facing surface. (d) **missing `.claude/agents/` discovery**: many existing projects have `.claude/agents/` from Claude Code installs. claw-code only looks at `.claw/agents/` — users have to copy/move their existing agents. (e) **no agent-scaffolding command**: cross-reference #431 — there's no `claw agents create ` to generate a valid `.toml` skeleton; users must hand-craft. **Required fix shape:** (a) accept BOTH `.md` (with YAML frontmatter) AND `.toml` formats in `.claw/agents/`; prefer YAML frontmatter for Claude Code parity, keep TOML for back-compat; (b) include `.claude/agents/` in the discovery sources alongside `.claw/agents/` with documented precedence; (c) expose `invalid_agents:[{path, reason}]` array in `agents list --output-format json` so users can see what was skipped and why; (d) document the agent schema (required + optional fields) in `claw agents --help` and in USAGE.md; (e) add `claw agents create ` scaffolding command per #431; (f) regression test: `.claw/agents/foo.md` with YAML frontmatter loads correctly. **Why this matters:** agents are the primary extension surface for custom workflows. A silent-drop on the wrong file format breaks the discoverability promise of CLI agents. Claude Code's `.md`-with-YAML convention is the lingua franca across AI coding tools; deviating to TOML breaks copy-paste compatibility. Cross-references #430 (dump-manifests needs upstream), #431 (skills/agents lifecycle), #440 (MCP all-or-nothing), #441 (hooks all-or-nothing), #438 (memory file discovery only CLAUDE.md). Source: Jobdori live dogfood, `8499599b`, 2026-05-11. +442. **DONE — agents discovery now accepts both TOML and Markdown formats** — fixed 2026-06-04 in `fix: accept markdown agent definitions with YAML frontmatter`. Agent discovery now loads `.md` files with YAML frontmatter alongside `.toml` files. Markdown agent files must have `---`-delimited YAML frontmatter with at least `name` or `description` fields; other supported fields are `model` and `model_reasoning_effort`. Files without valid frontmatter are recorded as `invalid_agents:[{path, reason, valid:false}]` instead of being silently dropped. `agents list --output-format json` includes `valid_count`, `invalid_count`, and `invalid_agents` metadata, and reports `status:"degraded"` when invalid entries exist. Backward compatibility with `.toml` format is fully preserved. Remaining sibling items: `.claude/agents/` discovery and agent schema documentation are tracked separately. 443. **`claw acp serve` exits 0 with `status:"discoverability_only", supported:false` instead of failing — automation pipelines see "success" from a command that explicitly says "not implemented"; ROADMAP #413's internal-tracking leak (`discoverability_tracking:"ROADMAP #64a"`, `tracking:"ROADMAP #76"`) still present despite being filed 2026-04-30** — dogfooded 2026-05-11 by Jobdori on `19aaf9d0` in response to Clawhip pinpoint nudge at `1503366101533200435`. Reproduction: `claw acp serve --output-format json` returns exit code **0** with envelope `{aliases:["acp","--acp","-acp"], discoverability_tracking:"ROADMAP #64a", kind:"acp", launch_command:null, message:"ACP/Zed editor integration is not implemented in claw-code yet. \`claw acp serve\` is only a discoverability alias today; it does not launch a daemon or Zed-specific protocol endpoint. Use the normal terminal surfaces for now and track ROADMAP #76 for real ACP support.", recommended_workflows:["claw prompt TEXT","claw","claw doctor"], serve_alias_only:true, status:"discoverability_only", supported:false, tracking:"ROADMAP #76"}`. The exit code is 0 (success) but the command explicitly states it is not implemented. Pipeline like `claw acp serve && zed --connect localhost:12345` will proceed to the zed connect step despite `acp serve` being a no-op. The only signal of no-op is `supported:false` in the JSON body — easy to miss for automation gating on `$?`. **ROADMAP #413 reproduction confirmed unfixed:** #413 (filed 2026-04-30) called out `discoverability_tracking:"ROADMAP #64a"` and `tracking:"ROADMAP #76"` as internal ticket references leaked into public JSON. **11 days later, both fields are still present in the envelope.** The fix was prescribed but never landed. Also `recommended_workflows:["claw prompt TEXT","claw","claw doctor"]` is internal scaffolding (curated suggestion list) exposed as a top-level public field — not normally part of an "ACP status" public contract. **Sibling unknown-subcommand bug:** `claw acp status --output-format json` (a reasonable next-thing-to-try) returns `{"error":"unsupported ACP invocation. Use \`claw acp\`, \`claw acp serve\`, \`claw --acp\`, or \`claw -acp\`.","kind":"unknown"}` exit 0 — the `kind:"unknown"` catch-all yet again (#422/#423/#424/#428/#430/#431/#432/#433/#435/#440/#441/#442 — **14th occurrence**), should be `kind:"unsupported_acp_invocation"`. **Required fix shape:** (a) `claw acp serve` exits **non-zero** (exit code 2 = "not implemented" is conventional) so automation `$?`-gating detects the no-op; (b) deliver #413's fix: remove `discoverability_tracking` and `tracking` top-level fields, OR move them under an optional `_meta` sub-object gated on a debug flag; (c) replace `message` prose with a typed `reason:"not_implemented"` enum + optional `detail` string for downstream pipelines that need a stable signal; (d) drop `recommended_workflows` from the ACP envelope OR move it under `_meta`; (e) the `status:"discoverability_only"` value is non-standard — replace with `status:"not_implemented"` (matching the `supported:false` boolean); (f) typed `kind:"unsupported_acp_invocation"` for the bad-arg path. **Why this matters:** ACP/Zed integration is the integration point for IDE-based AI workflows. A "success" exit code on a "not implemented" stub breaks the contract for any wrapper script that tries to detect ACP availability via `claw acp serve && ...`. The internal-tracking-ID leak (#413) being unfixed for 11 days suggests the JSON envelope audit isn't being executed against the ROADMAP backlog. Cross-references #413 (internal tracking leak — unfixed), #422 (exit-code parity), `kind:"unknown"` catch-all cluster. Source: Jobdori live dogfood, `19aaf9d0`, 2026-05-11. diff --git a/rust/crates/commands/src/lib.rs b/rust/crates/commands/src/lib.rs index 088a386b..f8c94197 100644 --- a/rust/crates/commands/src/lib.rs +++ b/rust/crates/commands/src/lib.rs @@ -2147,7 +2147,7 @@ impl DefinitionSource { } #[derive(Debug, Clone, PartialEq, Eq)] -struct AgentSummary { +pub(crate) struct AgentSummary { name: String, description: Option, model: Option, @@ -2158,6 +2158,20 @@ struct AgentSummary { path: Option, } +/// An agent definition file that could not be loaded. +#[derive(Debug, Clone, PartialEq, Eq)] +pub(crate) struct InvalidAgentConfig { + pub(crate) path: PathBuf, + pub(crate) reason: String, +} + +/// Loaded agent definitions plus any invalid entries that were skipped. +#[derive(Debug, Clone, Default)] +pub(crate) struct AgentCollection { + pub(crate) agents: Vec, + pub(crate) invalid_agents: Vec, +} + #[derive(Debug, Clone, PartialEq, Eq)] struct SkillSummary { name: String, @@ -2494,8 +2508,8 @@ pub fn handle_agents_slash_command_json(args: Option<&str>, cwd: &Path) -> std:: match normalize_optional_args(args) { None | Some("list") => { let roots = discover_definition_roots(cwd, "agents"); - let agents = load_agents_from_roots(&roots)?; - Ok(render_agents_report_json(cwd, &agents)) + let collection = load_agents_from_roots_with_invalids(&roots)?; + Ok(render_agents_report_json(cwd, &collection)) } Some(args) if args.starts_with("list ") => { let filter = args["list ".len()..].trim().to_lowercase(); @@ -2512,17 +2526,26 @@ pub fn handle_agents_slash_command_json(args: Option<&str>, cwd: &Path) -> std:: })); } let roots = discover_definition_roots(cwd, "agents"); - let agents = load_agents_from_roots(&roots)?; - let filtered: Vec<_> = agents + let collection = load_agents_from_roots_with_invalids(&roots)?; + let filtered_agents: Vec<_> = collection + .agents .into_iter() .filter(|a| a.name.to_lowercase().contains(&filter)) .collect(); - Ok(render_agents_report_json(cwd, &filtered)) + let filtered_collection = AgentCollection { + agents: filtered_agents, + invalid_agents: collection.invalid_agents, + }; + Ok(render_agents_report_json(cwd, &filtered_collection)) } Some("show" | "info" | "describe") => { let roots = discover_definition_roots(cwd, "agents"); - let agents = load_agents_from_roots(&roots)?; - Ok(render_agents_report_json_with_action(cwd, &agents, "show")) + let collection = load_agents_from_roots_with_invalids(&roots)?; + Ok(render_agents_report_json_with_action( + cwd, + &collection, + "show", + )) } Some(args) if args.starts_with("show ") @@ -2553,8 +2576,9 @@ pub fn handle_agents_slash_command_json(args: Option<&str>, cwd: &Path) -> std:: })); } let roots = discover_definition_roots(cwd, "agents"); - let agents = load_agents_from_roots(&roots)?; - let matched: Vec<_> = agents + let collection = load_agents_from_roots_with_invalids(&roots)?; + let matched: Vec<_> = collection + .agents .into_iter() .filter(|a| a.name.to_lowercase() == name) .collect(); @@ -2571,7 +2595,15 @@ pub fn handle_agents_slash_command_json(args: Option<&str>, cwd: &Path) -> std:: "hint": "Run `claw agents list` to see available agents.", })); } - Ok(render_agents_report_json_with_action(cwd, &matched, "show")) + let matched_collection = AgentCollection { + agents: matched, + invalid_agents: collection.invalid_agents, + }; + Ok(render_agents_report_json_with_action( + cwd, + &matched_collection, + "show", + )) } Some("create") => Ok(render_agents_missing_argument_json("create", "agent_name")), Some(args) if args.starts_with("create ") => { @@ -3902,30 +3934,69 @@ fn push_unique_skill_root( fn load_agents_from_roots( roots: &[(DefinitionSource, PathBuf)], ) -> std::io::Result> { + let collection = load_agents_from_roots_with_invalids(roots)?; + Ok(collection.agents) +} + +/// Load agent definitions from all roots, collecting both valid agents and +/// invalid entries (wrong extension, broken frontmatter, etc.). +fn load_agents_from_roots_with_invalids( + roots: &[(DefinitionSource, PathBuf)], +) -> std::io::Result { let mut agents = Vec::new(); + let mut invalid_agents = Vec::new(); let mut active_sources = BTreeMap::::new(); for (source, root) in roots { let mut root_agents = Vec::new(); for entry in fs::read_dir(root)? { let entry = entry?; - if entry.path().extension().is_none_or(|ext| ext != "toml") { - continue; + let path = entry.path(); + let ext = path.extension().and_then(|e| e.to_str()); + match ext { + Some("toml") => { + let contents = fs::read_to_string(&path)?; + let fallback_name = path.file_stem().map_or_else( + || entry.file_name().to_string_lossy().to_string(), + |stem| stem.to_string_lossy().to_string(), + ); + root_agents.push(AgentSummary { + name: parse_toml_string(&contents, "name").unwrap_or(fallback_name), + description: parse_toml_string(&contents, "description"), + model: parse_toml_string(&contents, "model"), + reasoning_effort: parse_toml_string(&contents, "model_reasoning_effort"), + source: *source, + shadowed_by: None, + path: Some(path), + }); + } + Some("md") => { + let contents = fs::read_to_string(&path)?; + let (name, description, model, reasoning_effort) = + parse_agent_frontmatter(&contents); + if name.is_none() && description.is_none() { + invalid_agents.push(InvalidAgentConfig { + path, + reason: "Markdown agent file has no YAML frontmatter with name or description fields".to_string(), + }); + continue; + } + let fallback_name = path.file_stem().map_or_else( + || entry.file_name().to_string_lossy().to_string(), + |stem| stem.to_string_lossy().to_string(), + ); + root_agents.push(AgentSummary { + name: name.unwrap_or(fallback_name), + description, + model, + reasoning_effort, + source: *source, + shadowed_by: None, + path: Some(path), + }); + } + _ => continue, } - let contents = fs::read_to_string(entry.path())?; - let fallback_name = entry.path().file_stem().map_or_else( - || entry.file_name().to_string_lossy().to_string(), - |stem| stem.to_string_lossy().to_string(), - ); - root_agents.push(AgentSummary { - name: parse_toml_string(&contents, "name").unwrap_or(fallback_name), - description: parse_toml_string(&contents, "description"), - model: parse_toml_string(&contents, "model"), - reasoning_effort: parse_toml_string(&contents, "model_reasoning_effort"), - source: *source, - shadowed_by: None, - path: Some(entry.path()), - }); } root_agents.sort_by(|left, right| left.name.cmp(&right.name)); @@ -3940,7 +4011,10 @@ fn load_agents_from_roots( } } - Ok(agents) + Ok(AgentCollection { + agents, + invalid_agents, + }) } fn load_skills_from_roots(roots: &[SkillRoot]) -> std::io::Result> { @@ -4091,6 +4165,63 @@ fn unquote_frontmatter_value(value: &str) -> String { .to_string() } +/// Parse agent metadata from YAML frontmatter in `.md` agent files. +/// Returns (name, description, model, reasoning_effort) extracted from +/// the `---`-delimited YAML block at the top of the file. +fn parse_agent_frontmatter( + contents: &str, +) -> ( + Option, + Option, + Option, + Option, +) { + let mut lines = contents.lines(); + if lines.next().map(str::trim) != Some("---") { + return (None, None, None, None); + } + + let mut name = None; + let mut description = None; + let mut model = None; + let mut reasoning_effort = None; + for line in lines { + let trimmed = line.trim(); + if trimmed == "---" { + break; + } + if let Some(value) = trimmed.strip_prefix("name:") { + let value = unquote_frontmatter_value(value.trim()); + if !value.is_empty() { + name = Some(value); + } + continue; + } + if let Some(value) = trimmed.strip_prefix("description:") { + let value = unquote_frontmatter_value(value.trim()); + if !value.is_empty() { + description = Some(value); + } + continue; + } + if let Some(value) = trimmed.strip_prefix("model:") { + let value = unquote_frontmatter_value(value.trim()); + if !value.is_empty() { + model = Some(value); + } + continue; + } + if let Some(value) = trimmed.strip_prefix("model_reasoning_effort:") { + let value = unquote_frontmatter_value(value.trim()); + if !value.is_empty() { + reasoning_effort = Some(value); + } + } + } + + (name, description, model, reasoning_effort) +} + fn render_agents_report(agents: &[AgentSummary]) -> String { if agents.is_empty() { return "No agents found.".to_string(); @@ -4133,31 +4264,42 @@ fn render_agents_report(agents: &[AgentSummary]) -> String { lines.join("\n").trim_end().to_string() } -fn render_agents_report_json(cwd: &Path, agents: &[AgentSummary]) -> Value { - render_agents_report_json_with_action(cwd, agents, "list") +fn render_agents_report_json(cwd: &Path, collection: &AgentCollection) -> Value { + render_agents_report_json_with_action(cwd, collection, "list") } fn render_agents_report_json_with_action( cwd: &Path, - agents: &[AgentSummary], + collection: &AgentCollection, action: &str, ) -> Value { + let agents = &collection.agents; + let invalid_agents = &collection.invalid_agents; let active = agents .iter() .filter(|agent| agent.shadowed_by.is_none()) .count(); + let has_invalids = !invalid_agents.is_empty(); + let status = if has_invalids { "degraded" } else { "ok" }; json!({ "kind": "agents", - "status": "ok", + "status": status, "action": action, "working_directory": cwd.display().to_string(), "count": agents.len(), + "valid_count": agents.len(), + "invalid_count": invalid_agents.len(), "summary": { "total": agents.len(), "active": active, "shadowed": agents.len().saturating_sub(active), }, "agents": agents.iter().map(agent_summary_json).collect::>(), + "invalid_agents": invalid_agents.iter().map(|invalid| json!({ + "path": invalid.path.display().to_string(), + "reason": &invalid.reason, + "valid": false, + })).collect::>(), }) } @@ -5127,7 +5269,7 @@ mod tests { render_agents_report_json, render_mcp_report_json_for, render_plugins_report, render_plugins_report_with_failures, render_skills_report, render_slash_command_help, render_slash_command_help_detail, resolve_skill_path, resume_supported_slash_commands, - slash_command_specs, suggest_slash_commands, validate_slash_command_input, + slash_command_specs, suggest_slash_commands, validate_slash_command_input, AgentCollection, DefinitionSource, SkillOrigin, SkillRoot, SkillSlashDispatch, SlashCommand, }; use plugins::{ @@ -6121,7 +6263,10 @@ mod tests { ]; let report = render_agents_report_json( &workspace, - &load_agents_from_roots(&roots).expect("agent roots should load"), + &AgentCollection { + agents: load_agents_from_roots(&roots).expect("agent roots should load"), + invalid_agents: Vec::new(), + }, ); assert_eq!(report["kind"], "agents");