Cycle #26 dogfood found a real red-state bug in the JSON envelope contract.
## The Bug
exec-command and exec-tool not-found cases return exit code 1 from the
process, but the envelope reports exit_code: 0 (the default from
wrap_json_envelope). This is a protocol violation.
Repro (before fix):
$ claw exec-command unknown-cmd test --output-format json > out.json
$ echo $?
1
$ jq '.exit_code' out.json
0 # WRONG — envelope lies about exit code
Claws reading the envelope's exit_code field get misinformation. A claw
implementing the canonical ERROR_HANDLING.md pattern (check exit_code,
then classify by error.kind) would incorrectly treat failures as
successes when dispatching on the envelope alone.
## Root Cause
main.py lines 687–739 (exec-command + exec-tool handlers):
- Return statement: 'return 0 if result.handled else 1' (correct)
- Envelope wrap: 'wrap_json_envelope(envelope, args.command)'
(uses default exit_code=0, IGNORES the return value)
The envelope wrap was called BEFORE the return value was computed, so
the exit_code field was never synchronized with the actual exit code.
## The Fix
Compute exit_code ONCE at the top:
exit_code = 0 if result.handled else 1
Pass it explicitly to wrap_json_envelope:
wrap_json_envelope(envelope, args.command, exit_code=exit_code)
Return the same value:
return exit_code
This ensures the envelope's exit_code field is always truth — the SAME
value the process returns.
## Tests Added (3)
TestEnvelopeExitCodeMatchesProcessExit in test_exec_route_bootstrap_output_format.py:
1. test_exec_command_not_found_envelope_exit_matches:
Verifies exec-command unknown-cmd returns exit 1 in both envelope
and process.
2. test_exec_tool_not_found_envelope_exit_matches:
Same for exec-tool.
3. test_all_commands_exit_code_invariant:
Audit across 4 known non-zero cases (show-command, show-tool,
exec-command, exec-tool not-found). Guards against the same bug
in other surfaces.
## Impact
- 206 → 209 passing tests (+3)
- Zero regressions
- Protocol contract now truthful: envelope.exit_code == process exit
- Claws using the one-handler pattern from ERROR_HANDLING.md now get
correct information
## Related
- ERROR_HANDLING.md (cycle #22): Documented exit_code as machine-readable
contract field
- #178/#179 (cycles #19/#20): Closed parser-front-door contract
- This closes a gap in the WORK PROTOCOL contract — envelope values must
match reality, not just be structurally present.
Classification (per cycle #24 calibration):
- Red-state bug: ✓ (contract violation, claws get misinformation)
- Real friction: ✓ (discovered via dogfood, not speculative)
- Fix ships same-cycle: ✓ (discipline per maintainership mode)
Source: Jobdori cycle #26 dogfood — ran multiple edge-case probes, noticed
exec-command envelope showed exit_code: 0 while process exited 1.
Investigated wrap_json_envelope default behavior, confirmed bug, fixed
and tested in same cycle.
Dogfood discovered #178 had two residual gaps:
1. Stderr pollution: argparse usage + error text still leaked to stderr even in
JSON mode (envelope was correct on stdout, but stderr noise broke the
'machine-first protocol' contract — claws capturing both streams got dual output)
2. Generic error message: envelope carried 'invalid command or argument (argparse
rejection)' instead of argparse's actual text like 'the following arguments
are required: session_id' or 'invalid choice: typo (choose from ...)'
Before #179:
$ claw load-session --output-format json
[stdout] {"error": {"message": "invalid command or argument (argparse rejection)"}}
[stderr] usage: main.py load-session [-h] ...
main.py load-session: error: the following arguments are required: session_id
[exit 1]
After #179:
$ claw load-session --output-format json
[stdout] {"error": {"message": "the following arguments are required: session_id"}}
[stderr] (empty)
[exit 1]
Implementation:
- New _ArgparseError exception class captures argparse's real message
- main() monkey-patches parser.error (+ all subparser.error) in JSON mode to raise
_ArgparseError instead of print-to-stderr + sys.exit(2)
- _emit_parse_error_envelope() now receives the real message verbatim
- Text mode path unchanged: still uses original argparse print+exit behavior
Contract:
- JSON mode: stdout carries envelope with argparse's actual error; stderr silent
- Text mode: unchanged — argparse usage to stderr, exit 2
- Parse errors still error.kind='parse', retryable=false
Test additions (5 new, 14 total in test_parse_error_envelope.py):
- TestParseErrorStderrHygiene (5):
- test_json_mode_stderr_is_silent_on_unknown_command
- test_json_mode_stderr_is_silent_on_missing_arg
- test_json_mode_envelope_carries_real_argparse_message
- test_json_mode_envelope_carries_invalid_choice_details (verifies valid-choices list)
- test_text_mode_stderr_preserved_on_unknown_command (backward compat)
Operational impact:
Claws capturing both stdout and stderr no longer get garbled output. The envelope
message now carries discoverability info (valid command list, missing-arg name)
that claws can use for retry/recovery without probing the CLI a second time.
Test results: 201 → 206 passing, 3 skipped unchanged, zero regression.
Pinpoint discovered via dogfood at 2026-04-22 20:30 KST (cycle #20).
#164 Stage B requires exposing whether cancellation was observed at the
turn-result level. This commit adds the infrastructure field:
Changes:
- TurnResult.cancel_observed: bool = False (query_engine.py)
- _build_timeout_result() accepts cancel_observed parameter (runtime.py)
- Two timeout paths now pass cancel_event.is_set() to signal observation (runtime.py)
- bootstrap command includes cancel_observed in turn JSON (main.py)
- SCHEMAS.md documents Turn Result Fields with cancel_observed contract
Usage:
When a turn timeout occurs, cancel_observed=true indicates that the
engine observed the cancellation event being set. This allows callers
to distinguish:
- timeout with no cancel → infrastructure/network stall
- timeout with cancel observed → cooperative cancellation was triggered
Backward compat:
- Existing TurnResult construction without cancel_observed defaults to False
- bootstrap JSON output still validates per SCHEMAS.md (new field is always present)
Test results: 182 passing, 3 skipped, zero regression.
Related: #161 (wall-clock timeout), #164 (cancellation observability protocol)
ROADMAP continues #164 with Stage C (test coverage for cancellation + turn envelope).
Final diagnostic surface in the JSON parity sweep: bootstrap-graph
(the runtime bootstrap/prefetch visualization) now supports --output-format.
Concrete addition:
- bootstrap-graph: --output-format {text,json}
JSON envelope:
{stages: [str], note: 'bootstrap-graph is markdown-only in this version'}
Envelope explanation: bootstrap-graph's Markdown output is rich and
textual; raw JSON embedding maintains the markdown format (split into
lines array) rather than attempting lossy structural extraction that
would lose information. This is an honest limitation in this cycle;
full JSON schema can be added in a future audit if claws require
structured bootstrap data (dependency graphs, prefetch timing, etc.).
Backward compatibility:
- Default is 'text' (Markdown unchanged)
Closes ROADMAP #170.
Related: #167, #168, #169. Diagnostic/inventory surface family is now
uniformly JSON-capable. Summary, manifest, parity-audit, setup-report,
command-graph, tool-pool, bootstrap-graph all accept --output-format.
Extends the diagnostic surface audit with the two inventory-structure
commands: command-graph (command family segmentation) and tool-pool
(assembled tool inventory). Both now expose their underlying rich
datastructures via JSON envelope.
Concrete additions:
- command-graph: --output-format {text,json}
- tool-pool: --output-format {text,json}
JSON envelope shapes:
command-graph:
{builtins_count, plugin_like_count, skill_like_count, total_count,
builtins: [{name, source_hint}],
plugin_like: [{name, source_hint}],
skill_like: [{name, source_hint}]}
tool-pool:
{simple_mode, include_mcp, tool_count,
tools: [{name, source_hint}]}
Backward compatibility:
- Default is 'text' (Markdown unchanged)
- Text output byte-identical to pre-#169
Tests (4 new, test_command_graph_tool_pool_output_format.py):
- TestCommandGraphOutputFormat (2): JSON structure + text compat
- TestToolPoolOutputFormat (2): JSON structure + text compat
Full suite: 137 → 141 passing, zero regression.
Closes ROADMAP #169.
Why this matters:
Claws auditing the codebase can now ask 'what commands exist' and
'what tools exist' and get structured, parseable answers instead of
regex-parsing Markdown headers and counting list items.
Related clusters:
- Diagnostic surfaces (#169 adds to #167/#168 work-verb parity)
- Inventory introspection (command-graph + tool-pool are the two
foundational 'what do we have?' queries)
Closes the inspect-capability parity gap: show-command and show-tool were
the only discovery/inspection CLI commands lacking --output-format support,
making them outliers in the ecosystem that already had unified JSON
contracts across list-sessions, load-session, delete-session, and
flush-transcript (#160/#165/#166).
Concrete additions:
- show-command: --output-format {text,json}
- show-tool: --output-format {text,json}
JSON envelope shape (found case):
{name, found: true, source_hint, responsibility}
JSON envelope shape (not-found case):
{name, found: false, error: {kind:'command_not_found'|'tool_not_found',
message, retryable: false}}
Exit codes:
0 = success
1 = not found
Backward compatibility:
- Default (no --output-format) is 'text' (unchanged)
- Text output byte-identical to pre-#167 (three newline-separated lines)
Tests (10 new, test_show_command_tool_output_format.py):
- TestShowCommandOutputFormat (5): found + not-found in JSON; text mode
backward compat; text is default
- TestShowToolOutputFormat (3): found + not-found in JSON; text mode
backward compat
- TestShowCommandToolFormatParity (2): both accept same flag choices;
consistent JSON envelope shape
Full suite: 114 → 124 passing, zero regression.
Closes ROADMAP #167.
Why this matters:
Before: Claws calling show-command/show-tool had to parse human-readable
prose output via regex, with no structured error signal.
After: Same envelope contract as load-session and friends: JSON-first,
typed errors, machine-parseable.
Related clusters:
- Session-lifecycle CLI parity family (#160, #165, #166, #167)
- Machine-readable error contracts (same vein as #162 atomicity + #164
cancellation state-safety: structured boundaries for orchestration)
The #160 session-lifecycle CLI triplet was asymmetric: list-sessions and
delete-session accepted --directory + --output-format and emitted typed
JSON error envelopes, but load-session had neither flag and dumped a raw
Python traceback (including the SessionNotFoundError class name) on a
missing session.
Three concrete impacts this fix closes:
1. Alternate session-store locations (e.g. /tmp/claw-run-XXX/.port_sessions)
were unreachable via load-session; claws had to chdir or monkeypatch
DEFAULT_SESSION_DIR to work around it.
2. Not-found emitted a multi-line Python stack, not a parseable envelope.
Claws deciding retry/escalate/give-up had only exit code 1 to work with.
3. The traceback leaked 'src.session_store.SessionNotFoundError' verbatim,
coupling version-pinned claws to our internal exception class name.
Now all three triplet commands accept the same flag pair and emit the
same JSON error shape:
Success (json mode):
{"session_id": "alpha", "loaded": true, "messages_count": 3,
"input_tokens": 42, "output_tokens": 99}
Not-found:
{"session_id": "missing", "loaded": false,
"error": {"kind": "session_not_found",
"message": "session 'missing' not found in /path",
"directory": "/path", "retryable": false}}
Corrupted file:
{"session_id": "broken", "loaded": false,
"error": {"kind": "session_load_failed",
"message": "...", "directory": "/path",
"retryable": true}}
Exit code contract:
- 0 on successful load
- 1 on not-found (preserves existing $?)
- 1 on OSError/JSONDecodeError (distinct 'kind' in JSON)
Backward compat: legacy 'claw load-session ID' text output unchanged
byte-for-byte. Only new behaviour is the flags and structured error path.
Tests (tests/test_load_session_cli.py, 13 tests):
- TestDirectoryFlagParity (2): --directory works + fallback to CWD/.port_sessions
- TestOutputFormatFlagParity (2): json schema + text-mode backward compat
- TestNotFoundTypedError (2): JSON envelope on not-found; no traceback in
either mode; no internal class name leak
- TestLoadFailedDistinctFromNotFound (1): corrupted file = session_load_failed
with retryable=true, distinct from session_not_found
- TestTripletParityConsistency (6): parametrised over [list, delete, load] *
[--directory, --output-format] — explicit parity guard for future regressions
Full suite: 80/80 passing, zero regression.
Discovered via Jobdori dogfood sweep 2026-04-22 17:44 KST — ran
'claw load-session nonexistent' expecting a clean error, got a Python
traceback. Filed #165 + fixed in same commit.
Closes ROADMAP #165.
#163: run_turn_loop no longer injects f'{prompt} [turn N]' into follow-up
prompts. The suffix was never defined or interpreted anywhere — not by the
engine, not by the system prompt, not by any LLM. It looked like a real
user-typed annotation in the transcript and made replay/analysis fragile.
New behaviour:
- turn 0 submits the original prompt (unchanged)
- turn > 0 submits caller-supplied continuation_prompt if provided, else
the loop stops cleanly — no fabricated user turn
- added continuation_prompt: str | None = None parameter to run_turn_loop
- added --continuation-prompt CLI flag for claws scripting multi-turn loops
- zero '[turn' strings ever appear in mutable_messages or stdout now
Behaviour change for existing callers:
- Before: run_turn_loop(prompt, max_turns=3) submitted 3 turns
('prompt', 'prompt [turn 2]', 'prompt [turn 3]')
- After: run_turn_loop(prompt, max_turns=3) submits 1 turn ('prompt')
- To preserve old multi-turn behaviour, pass continuation_prompt='Continue.'
or any structured follow-up text
One existing timeout test (test_budget_is_cumulative_across_turns) updated
to pass continuation_prompt so the cumulative-budget contract is actually
exercised across turns instead of trivially satisfied by a one-turn loop.
#164 filed: addresses reviewer feedback on #161. The wall-clock timeout
bounds the caller-facing wait, but the underlying submit_message worker
thread keeps running and can mutate engine state after the timeout
TurnResult is returned. A cooperative cancel_event pattern is sketched in
the pinpoint; real asyncio.Task.cancel() support will come once provider
IO is async-native (larger refactor).
Tests (tests/test_run_turn_loop_continuation.py, 8 tests):
- TestNoTurnSuffixInjection (2): zero '[turn' strings in any submitted
prompt, both default and explicit-continuation paths
- TestContinuationDefaultStopsAfterTurnZero (2): default loops run exactly
one turn; engine.submit_message called exactly once despite max_turns=10
- TestExplicitContinuationBehaviour (2): turn 0 = original, turn N = continuation
verbatim; max_turns still respected
- TestCLIContinuationFlag (2): CLI default emits only '## Turn 1';
--continuation-prompt wires through to multi-turn behaviour
Full suite: 67/67 passing.
Closes ROADMAP #163. Files #164.
Previously, run_turn_loop was bounded only by max_turns (turn count). If
engine.submit_message stalled — slow provider, hung network, infinite
stream — the loop blocked indefinitely with no cancellation path. Claws
calling run_turn_loop in CI or orchestration had no reliable way to
enforce a deadline; the loop would hang until OS kill or human intervention.
Fix:
- Add timeout_seconds parameter to run_turn_loop (default None = legacy unbounded).
- When set, each submit_message call runs inside a ThreadPoolExecutor and is
bounded by the remaining wall-clock budget (total across all turns, not per-turn).
- On timeout, synthesize a TurnResult with stop_reason='timeout' carrying the
turn's prompt and routed matches so transcripts preserve orchestration context.
- Exhausted/negative budget short-circuits before calling submit_message.
- Legacy path (timeout_seconds=None) bypasses the executor entirely — zero
overhead for callers that don't opt in.
CLI:
- Added --timeout-seconds flag to 'turn-loop' command.
- Exit code 2 when the loop terminated on timeout (vs 0 for completed),
so shell scripts can distinguish 'done' from 'budget exhausted'.
Tests (tests/test_run_turn_loop_timeout.py, 6 tests):
- Legacy unbounded path unchanged (timeout_seconds=None never emits 'timeout')
- Hung submit_message aborted within budget (0.3s budget, 5s mock hang → exit <1.5s)
- Budget is cumulative across turns (0.6s budget, 0.4s per turn, not per-turn)
- timeout_seconds=0 short-circuits first turn without calling submit_message
- Negative timeout treated as exhausted (guard against caller bugs)
- Timeout TurnResult carries correct prompt, matches, UsageSummary shape
Full suite: 49/49 passing, zero regression.
Blocker: none. Closes ROADMAP #161.
The old tracked TypeScript snapshot has been removed from the repository history and the root directory is now a Python porting workspace. README and tests now describe and verify the Python-first layout instead of treating the exposed snapshot as the active source tree.
A local archive can still exist outside Git, but the tracked repository now presents only the Python porting surface, related essay context, and OmX workflow artifacts.
Constraint: Tracked history should collapse to a single commit while excluding the archived snapshot from Git
Rejected: Keep the exposed TypeScript tree in tracked history under an archive path | user explicitly wanted only the Python porting repo state in Git
Confidence: medium
Scope-risk: broad
Reversibility: messy
Directive: Keep future tracked additions focused on the Python port itself; do not reintroduce the exposed snapshot into Git history
Tested: python3 -m unittest discover -s tests -v; python3 -m src.main summary; git diff --check
Not-tested: Behavioral parity with the original TypeScript system beyond the current Python workspace surface
