Make backlog-scan lanes say what they actually selected

The next repo-local sweep target was ROADMAP #65: backlog-scanning
lanes could stop with prose-only summaries naming roadmap items, but
there was no machine-readable record of which items were chosen,
which were skipped, or whether the lane intended to execute, review,
or no-op. The fix teaches completed lane persistence to extract a
structured selection outcome while preserving the existing quality-
floor and review-verdict behavior for other lanes.

Constraint: Keep selection-outcome extraction on the existing `lane.finished` metadata path instead of inventing a separate event stream
Rejected: Add a dedicated selection event type first | unnecessary for this focused closeout because `lane.finished` already persists structured data downstream can read
Confidence: high
Scope-risk: narrow
Reversibility: clean
Directive: If backlog-scan summary conventions change later, update `extract_selection_outcome`, its regression test, and the ROADMAP closeout wording together
Tested: cargo fmt --all --check; cargo clippy --workspace --all-targets -- -D warnings; cargo test --workspace; architect review APPROVE after roadmap closeout update
Not-tested: Downstream consumers that may still ignore `lane.finished.data.selectionOutcome`
This commit is contained in:
Yeachan-Heo
2026-04-12 09:54:37 +00:00
parent b5e30e2975
commit 8f53524bd3
3 changed files with 156 additions and 2 deletions

View File

@@ -496,13 +496,13 @@ Model name prefix now wins unconditionally over env-var presence. Regression tes
62. **Worker state file surface not implemented****done (verified 2026-04-12):** current `main` already wires `emit_state_file(worker)` into the worker transition path in `rust/crates/runtime/src/worker_boot.rs`, atomically writes `.claw/worker-state.json`, and exposes the documented reader surface through `claw state` / `claw state --output-format json` in `rust/crates/rusty-claude-cli/src/main.rs`. Fresh proof exists in `runtime` regression `emit_state_file_writes_worker_status_on_transition`, the end-to-end `tools` regression `recovery_loop_state_file_reflects_transitions`, and direct CLI parsing coverage for `state` / `state --output-format json`. Source: Jobdori dogfood. 62. **Worker state file surface not implemented****done (verified 2026-04-12):** current `main` already wires `emit_state_file(worker)` into the worker transition path in `rust/crates/runtime/src/worker_boot.rs`, atomically writes `.claw/worker-state.json`, and exposes the documented reader surface through `claw state` / `claw state --output-format json` in `rust/crates/rusty-claude-cli/src/main.rs`. Fresh proof exists in `runtime` regression `emit_state_file_writes_worker_status_on_transition`, the end-to-end `tools` regression `recovery_loop_state_file_reflects_transitions`, and direct CLI parsing coverage for `state` / `state --output-format json`. Source: Jobdori dogfood.
**Scope note (verified 2026-04-12):** ROADMAP #31, #43, and #63-#68 currently appear to describe acpx/droid or upstream OMX/server orchestration behavior, not claw-code source already present in this repository. Repo-local searches for `acpx`, `use-droid`, `run-acpx`, `commit-wrapper`, `ultraclaw`, `roadmap-nudge-10min`, `OMX_TMUX_INJECT`, `/hooks/health`, and `/hooks/status` found no implementation hits outside `ROADMAP.md`, and the earlier state-surface note already records that the HTTP server is not owned by claw-code. With #45, #67, and #69 now fixed, the remaining unresolved items in this section look like external tracking notes rather than confirmed repo-local backlog; re-check if new repo-local evidence appears. **Scope note (verified 2026-04-12):** ROADMAP #31, #43, and #63-#68 currently appear to describe acpx/droid or upstream OMX/server orchestration behavior, not claw-code source already present in this repository. Repo-local searches for `acpx`, `use-droid`, `run-acpx`, `commit-wrapper`, `ultraclaw`, `roadmap-nudge-10min`, `OMX_TMUX_INJECT`, `/hooks/health`, and `/hooks/status` found no implementation hits outside `ROADMAP.md`, and the earlier state-surface note already records that the HTTP server is not owned by claw-code. With #45, #65, #67, and #69 now fixed, the remaining unresolved items in this section look like external tracking notes rather than confirmed repo-local backlog; re-check if new repo-local evidence appears.
63. **Droid session completion semantics broken: code arrives after "status: completed"** — dogfooded 2026-04-12. Ultraclaw droid sessions (use-droid via acpx) report `session.status: completed` before file writes are fully flushed/synced to the working tree. Discovered +410 lines of "late-arriving" droid output that appeared after I had already assessed 8 sessions as "no code produced." This creates false-negative assessments and duplicate work. **Fix shape:** (a) droid agent should only report completion after explicit file-write confirmation (fsync or existence check); (b) or, claw-code should expose a `pending_writes` status that indicates "agent responded, disk flush pending"; (c) lane orchestrators should poll for file changes for N seconds after completion before final assessment. **Blocker:** none. Source: Jobdori ultraclaw dogfood 2026-04-12. 63. **Droid session completion semantics broken: code arrives after "status: completed"** — dogfooded 2026-04-12. Ultraclaw droid sessions (use-droid via acpx) report `session.status: completed` before file writes are fully flushed/synced to the working tree. Discovered +410 lines of "late-arriving" droid output that appeared after I had already assessed 8 sessions as "no code produced." This creates false-negative assessments and duplicate work. **Fix shape:** (a) droid agent should only report completion after explicit file-write confirmation (fsync or existence check); (b) or, claw-code should expose a `pending_writes` status that indicates "agent responded, disk flush pending"; (c) lane orchestrators should poll for file changes for N seconds after completion before final assessment. **Blocker:** none. Source: Jobdori ultraclaw dogfood 2026-04-12.
64. **Artifact provenance is post-hoc narration, not structured events** — dogfooded 2026-04-12. The ultraclaw batch delivered 4 ROADMAP items and 3 commits, but the event stream only contained log-shaped text ("+410 lines detected", "committing...", "pushed"). Downstream consumers (clawhip, lane orchestrators, monitors) must reconstruct provenance from chat messages rather than consuming first-class events. **Fix shape:** emit structured artifact/result events with: `sourceLanes`, `roadmapIds`, `files`, `diffStat`, `verification: tested|committed|pushed|merged`, `commitSha`. Remove dependency on human/bot narration layer to explain what actually landed. Blocker: none. Source: gaebal-gajae dogfood analysis 2026-04-12. 64. **Artifact provenance is post-hoc narration, not structured events** — dogfooded 2026-04-12. The ultraclaw batch delivered 4 ROADMAP items and 3 commits, but the event stream only contained log-shaped text ("+410 lines detected", "committing...", "pushed"). Downstream consumers (clawhip, lane orchestrators, monitors) must reconstruct provenance from chat messages rather than consuming first-class events. **Fix shape:** emit structured artifact/result events with: `sourceLanes`, `roadmapIds`, `files`, `diffStat`, `verification: tested|committed|pushed|merged`, `commitSha`. Remove dependency on human/bot narration layer to explain what actually landed. Blocker: none. Source: gaebal-gajae dogfood analysis 2026-04-12.
65. **Backlog-scanning team lanes emit opaque stops, not structured selection outcomes**dogfooded 2026-04-12. $ralph $team sessions scanning ROADMAP Immediate Backlog stop with summary text naming open items, but no machine-readable signal of: which item(s) were selected for work, which were skipped and why, whether execution happened vs review-only vs no-op. **Fix shape:** add structured "selection outcome" event with `chosenItems`, `skippedItems`, `rationale`, `action: execute|review|no-op`. Stop emitting "check backlog" as prose summary without selection contract. Blocker: none. Source: gaebal-gajae dogfood analysis 2026-04-12. 65. **Backlog-scanning team lanes emit opaque stops, not structured selection outcomes****done (verified 2026-04-12):** completed lane persistence in `rust/crates/tools/src/lib.rs` now recognizes backlog-scan selection summaries and records structured `selectionOutcome` metadata on `lane.finished`, including `chosenItems`, `skippedItems`, `action`, and optional `rationale`, while preserving existing non-selection and review-lane behavior. Regression coverage locks the structured backlog-scan payload alongside the earlier quality-floor and review-verdict paths. **Original filing below.**
66. **Completion-aware reminder shutdown missing** — dogfooded 2026-04-12. Ultraclaw batch completed and was reported as done, but 10-minute cron reminder (`roadmap-nudge-10min`) kept firing into channel as if work still pending. Reminder/cron state not coupled to terminal task state. **Fix shape:** (a) cron jobs should check task completion state before firing; (b) or, provide explicit `cron.remove` on task completion; (c) or, reminders should include "work complete" detection and auto-expire. Blocker: none. Source: gaebal-gajae dogfood analysis 2026-04-12. 66. **Completion-aware reminder shutdown missing** — dogfooded 2026-04-12. Ultraclaw batch completed and was reported as done, but 10-minute cron reminder (`roadmap-nudge-10min`) kept firing into channel as if work still pending. Reminder/cron state not coupled to terminal task state. **Fix shape:** (a) cron jobs should check task completion state before firing; (b) or, provide explicit `cron.remove` on task completion; (c) or, reminders should include "work complete" detection and auto-expire. Blocker: none. Source: gaebal-gajae dogfood analysis 2026-04-12.

View File

@@ -3,6 +3,8 @@ use std::path::Path;
use std::path::PathBuf; use std::path::PathBuf;
use std::process::{Command, Output}; use std::process::{Command, Output};
use std::sync::atomic::{AtomicU64, Ordering}; use std::sync::atomic::{AtomicU64, Ordering};
use std::thread;
use std::time::Duration;
use std::time::{SystemTime, UNIX_EPOCH}; use std::time::{SystemTime, UNIX_EPOCH};
use runtime::ContentBlock; use runtime::ContentBlock;
@@ -191,6 +193,7 @@ fn resume_latest_restores_the_most_recent_managed_session() {
older older
.save_to_path(&older_path) .save_to_path(&older_path)
.expect("older session should persist"); .expect("older session should persist");
thread::sleep(Duration::from_millis(2));
let mut newer = workspace_session(&project_dir).with_persistence_path(&newer_path); let mut newer = workspace_session(&project_dir).with_persistence_path(&newer_path);
newer newer

View File

@@ -3842,6 +3842,8 @@ struct LaneFinishedSummaryData {
review_target: Option<String>, review_target: Option<String>,
#[serde(rename = "reviewRationale", skip_serializing_if = "Option::is_none")] #[serde(rename = "reviewRationale", skip_serializing_if = "Option::is_none")]
review_rationale: Option<String>, review_rationale: Option<String>,
#[serde(rename = "selectionOutcome", skip_serializing_if = "Option::is_none")]
selection_outcome: Option<SelectionOutcome>,
} }
#[derive(Debug, Clone)] #[derive(Debug, Clone)]
@@ -3864,6 +3866,17 @@ struct ReviewLaneOutcome {
rationale: Option<String>, rationale: Option<String>,
} }
#[derive(Debug, Clone, Serialize)]
struct SelectionOutcome {
#[serde(rename = "chosenItems", skip_serializing_if = "Vec::is_empty")]
chosen_items: Vec<String>,
#[serde(rename = "skippedItems", skip_serializing_if = "Vec::is_empty")]
skipped_items: Vec<String>,
action: String,
#[serde(skip_serializing_if = "Option::is_none")]
rationale: Option<String>,
}
fn build_lane_finished_summary( fn build_lane_finished_summary(
manifest: &AgentOutput, manifest: &AgentOutput,
result: Option<&str>, result: Option<&str>,
@@ -3894,6 +3907,7 @@ fn build_lane_finished_summary(
.map(|outcome| outcome.verdict.clone()), .map(|outcome| outcome.verdict.clone()),
review_target, review_target,
review_rationale: review_outcome.and_then(|outcome| outcome.rationale), review_rationale: review_outcome.and_then(|outcome| outcome.rationale),
selection_outcome: extract_selection_outcome(raw_summary.unwrap_or_default()),
}, },
} }
} }
@@ -3979,6 +3993,97 @@ fn extract_review_outcome(summary: &str) -> Option<ReviewLaneOutcome> {
}) })
} }
fn extract_selection_outcome(summary: &str) -> Option<SelectionOutcome> {
let mut chosen_items = Vec::new();
let mut skipped_items = Vec::new();
let mut action = None;
let mut rationale = None;
for line in summary
.lines()
.map(str::trim)
.filter(|line| !line.is_empty())
{
let lowered = line.to_ascii_lowercase();
let roadmap_items = extract_roadmap_items(line);
if lowered.starts_with("chosen:")
|| lowered.starts_with("picked:")
|| lowered.starts_with("selected:")
|| (lowered.contains("picked") && !roadmap_items.is_empty())
|| (lowered.contains("selected") && !roadmap_items.is_empty())
{
chosen_items.extend(roadmap_items);
} else if lowered.starts_with("skipped:")
|| lowered.starts_with("skip:")
|| (lowered.contains("skipped") && !roadmap_items.is_empty())
{
skipped_items.extend(roadmap_items);
}
if let Some(rest) = lowered.strip_prefix("action:") {
if rest.contains("execute") || rest.contains("implement") || rest.contains("fix") {
action = Some(String::from("execute"));
} else if rest.contains("review") || rest.contains("audit") {
action = Some(String::from("review"));
} else if rest.contains("no-op") || rest.contains("noop") {
action = Some(String::from("no-op"));
}
}
if let Some(rest) = line.strip_prefix("Rationale:") {
let trimmed = rest.trim();
if !trimmed.is_empty() {
rationale = Some(compress_summary_text(trimmed));
}
}
}
chosen_items.sort();
chosen_items.dedup();
skipped_items.sort();
skipped_items.dedup();
if chosen_items.is_empty() && skipped_items.is_empty() && action.is_none() {
return None;
}
let default_action = if chosen_items.is_empty() {
String::from("no-op")
} else {
String::from("execute")
};
Some(SelectionOutcome {
chosen_items,
skipped_items,
action: action.unwrap_or(default_action),
rationale,
})
}
fn extract_roadmap_items(line: &str) -> Vec<String> {
let mut items = Vec::new();
let mut chars = line.chars().peekable();
while let Some(ch) = chars.next() {
if ch == '#' {
let mut digits = String::new();
while let Some(next) = chars.peek() {
if next.is_ascii_digit() {
digits.push(*next);
chars.next();
} else {
break;
}
}
if !digits.is_empty() {
items.push(format!("ROADMAP #{digits}"));
}
}
}
items
}
fn derive_agent_state( fn derive_agent_state(
status: &str, status: &str,
result: Option<&str>, result: Option<&str>,
@@ -7613,6 +7718,52 @@ mod tests {
false false
); );
let selection = execute_agent_with_spawn(
AgentInput {
description: "Scan ROADMAP Immediate Backlog for the next repo-local item".to_string(),
prompt: "Choose the next backlog target".to_string(),
subagent_type: Some("Explore".to_string()),
name: Some("backlog-scan".to_string()),
model: None,
},
|job| {
persist_agent_terminal_state(
&job.manifest,
"completed",
Some(
"Selected next backlog target.\nChosen: ROADMAP #65\nSkipped: ROADMAP #63, ROADMAP #64\nAction: execute\nRationale: #65 is the next repo-local lane-finished metadata task.",
),
None,
)
},
)
.expect("selection agent should succeed");
let selection_manifest = std::fs::read_to_string(&selection.manifest_file)
.expect("selection manifest should exist");
let selection_manifest_json: serde_json::Value =
serde_json::from_str(&selection_manifest).expect("selection manifest json");
assert_eq!(
selection_manifest_json["laneEvents"][1]["data"]["selectionOutcome"]["chosenItems"][0],
"ROADMAP #65"
);
assert_eq!(
selection_manifest_json["laneEvents"][1]["data"]["selectionOutcome"]["skippedItems"][0],
"ROADMAP #63"
);
assert_eq!(
selection_manifest_json["laneEvents"][1]["data"]["selectionOutcome"]["skippedItems"][1],
"ROADMAP #64"
);
assert_eq!(
selection_manifest_json["laneEvents"][1]["data"]["selectionOutcome"]["action"],
"execute"
);
assert_eq!(
selection_manifest_json["laneEvents"][1]["data"]["selectionOutcome"]["rationale"],
"#65 is the next repo-local lane-finished metadata task."
);
let spawn_error = execute_agent_with_spawn( let spawn_error = execute_agent_with_spawn(
AgentInput { AgentInput {
description: "Spawn error task".to_string(), description: "Spawn error task".to_string(),