mirror of
https://github.com/instructkr/claw-code.git
synced 2026-04-26 19:14:59 +08:00
docs+test: cycle #29 — document + lock text-mode vs JSON-mode exit divergence
Cycle #29 dogfood found a real pinpoint: cross-mode exit code divergence. ## The Pinpoint Dogfooding the CLI revealed that unknown subcommand errors return different exit codes depending on output mode: $ python3 -m src.main nonexistent-cmd # exit 2 $ python3 -m src.main nonexistent-cmd --output-format json # exit 1 ERROR_HANDLING.md documented the exit-code contract (1=parse, 2=timeout) but did NOT explicitly state the contract applies only to JSON mode. Text mode follows argparse defaults (exit 2 for any parse error), which violates the documented contract when interpreted generally. A claw using text mode with 'claw nonexistent' would see exit 2 and misclassify as timeout per the docs. Real protocol contract gap, not implementation bug. ## Classification This is a DOCUMENTATION gap, not a behavior bug: - Text mode follows argparse convention (reasonable for humans) - JSON mode normalizes to documented contract (reasonable for claws) - The divergence is intentional; only the docs were silent about it Fix = document the divergence explicitly + lock it with tests. NOT fix = change text mode exit code to 1 (would break argparse conventions and confuse human users). ## Documentation Changes ERROR_HANDLING.md: 1. Added IMPORTANT callout in Quick Reference section: 'The exit code contract applies ONLY when --output-format json is explicitly set. Text mode follows argparse conventions.' 2. New 'Text mode vs JSON mode exit codes' table showing exact divergence: - Unknown subcommand: text=2, json=1 - Missing required arg: text=2, json=1 - Session not found: text=1, json=1 (app-level, identical) - Success: text=0, json=0 (identical) - Timeout: text=2, json=2 (identical, #161) 3. Practical rule: 'always pass --output-format json' ## Tests Added (5) TestTextVsJsonModeDivergence in test_cross_channel_consistency.py: 1. test_unknown_command_text_mode_exits_2 — text mode argparse default 2. test_unknown_command_json_mode_exits_1 — JSON mode contract normalized 3. test_missing_required_arg_text_mode_exits_2 — same for missing args 4. test_missing_required_arg_json_mode_exits_1 — same normalization 5. test_success_path_identical_in_both_modes — success exit identical These tests LOCK the expected divergence so: - Documentation stays aligned with implementation - Future changes (either direction) are caught as intentional - Claws trust the docs ## Test Status - 217 → 222 tests passing (+5) - Zero regressions ## Discipline This cycle follows the cycle #28 template exactly: - Dogfood probe revealed real friction (test said exit=2, docs said exit=1) - Minimal fix shape (documentation clarification, not code change) - Regression guard via tests - Evidence-backed, not speculative Relationship to #181: - #181 fixed env.exit_code != process exit (WITHIN JSON mode) - #29 clarifies exit code contract scope (ONLY JSON mode) - Both establish: exit codes are deterministic, but only when --output-format json --- Classification (per cycle #24 calibration): - Red-state bug? ✗ (behavior was reasonable, docs were incomplete) - Real friction? ✓ (docs/code divergence revealed by dogfood) - Evidence-backed? ✓ (test suite probed both modes, found the gap) Source: Jobdori cycle #29 proactive dogfood — in response to Clawhip nudge for pinpoint hunting. Found that text-mode errors return exit 2 but ERROR_HANDLING.md implied exit 1 was the parse-error contract universally.
This commit is contained in:
@@ -10,12 +10,26 @@ After cycles #178–#179 (parser-front-door hole closure), claw-code's error int
|
|||||||
|
|
||||||
Every clawable command returns JSON on stdout when `--output-format json` is requested.
|
Every clawable command returns JSON on stdout when `--output-format json` is requested.
|
||||||
|
|
||||||
|
**IMPORTANT:** The exit code contract below applies **only when `--output-format json` is explicitly set**. Text mode follows argparse conventions and may return different exit codes (e.g., `2` for argparse parse errors). Claws consuming claw-code as a subprocess MUST always pass `--output-format json` to get the documented contract.
|
||||||
|
|
||||||
| Exit Code | Meaning | Response Format | Example |
|
| Exit Code | Meaning | Response Format | Example |
|
||||||
|---|---|---|---|
|
|---|---|---|---|
|
||||||
| **0** | Success | `{success fields}` | `{"session_id": "...", "loaded": true}` |
|
| **0** | Success | `{success fields}` | `{"session_id": "...", "loaded": true}` |
|
||||||
| **1** | Error / Not Found | `{error: {kind, message, ...}}` | `{"error": {"kind": "session_not_found", ...}}` |
|
| **1** | Error / Not Found | `{error: {kind, message, ...}}` | `{"error": {"kind": "session_not_found", ...}}` |
|
||||||
| **2** | Timeout | `{final_stop_reason: "timeout", final_cancel_observed: ...}` | `{"final_stop_reason": "timeout", ...}` |
|
| **2** | Timeout | `{final_stop_reason: "timeout", final_cancel_observed: ...}` | `{"final_stop_reason": "timeout", ...}` |
|
||||||
|
|
||||||
|
### Text mode vs JSON mode exit codes
|
||||||
|
|
||||||
|
| Scenario | Text mode exit | JSON mode exit | Why |
|
||||||
|
|---|---|---|---|
|
||||||
|
| Unknown subcommand | 2 (argparse default) | 1 (parse error envelope) | argparse defaults to 2; JSON mode normalizes to contract |
|
||||||
|
| Missing required arg | 2 (argparse default) | 1 (parse error envelope) | Same reason |
|
||||||
|
| Session not found | 1 | 1 | Application-level error, same in both |
|
||||||
|
| Command executed OK | 0 | 0 | Success path, identical |
|
||||||
|
| Turn-loop timeout | 2 | 2 | Identical (#161 implementation) |
|
||||||
|
|
||||||
|
**Practical rule for claws:** always pass `--output-format json`. This eliminates text-mode surprises and gives you the documented exit-code contract for every error path.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## One-Handler Pattern
|
## One-Handler Pattern
|
||||||
|
|||||||
@@ -186,3 +186,57 @@ class TestCrossChannelConsistency:
|
|||||||
'Boolean fields must correlate with error block:\n' +
|
'Boolean fields must correlate with error block:\n' +
|
||||||
'\n'.join(failures)
|
'\n'.join(failures)
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class TestTextVsJsonModeDivergence:
|
||||||
|
"""Cycle #29: Document known text-mode vs JSON-mode exit code divergence.
|
||||||
|
|
||||||
|
ERROR_HANDLING.md specifies the exit code contract applies ONLY when
|
||||||
|
--output-format json is set. Text mode follows argparse defaults (e.g.,
|
||||||
|
exit 2 for parse errors) while JSON mode normalizes to the contract
|
||||||
|
(exit 1 for parse errors).
|
||||||
|
|
||||||
|
This test class LOCKS the expected divergence so:
|
||||||
|
1. Documentation stays aligned with implementation
|
||||||
|
2. Future changes to text mode behavior are caught as intentional
|
||||||
|
3. Claws consuming subprocess output can trust the docs
|
||||||
|
"""
|
||||||
|
|
||||||
|
def test_unknown_command_text_mode_exits_2(self) -> None:
|
||||||
|
"""Text mode: argparse default exit 2 for unknown subcommand."""
|
||||||
|
result = _run(['nonexistent-cmd'])
|
||||||
|
assert result.returncode == 2, (
|
||||||
|
f'text mode should exit 2 (argparse default), got {result.returncode}'
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_unknown_command_json_mode_exits_1(self) -> None:
|
||||||
|
"""JSON mode: normalized exit 1 for parse error (#178)."""
|
||||||
|
result = _run(['nonexistent-cmd', '--output-format', 'json'])
|
||||||
|
assert result.returncode == 1, (
|
||||||
|
f'JSON mode should exit 1 (protocol contract), got {result.returncode}'
|
||||||
|
)
|
||||||
|
envelope = json.loads(result.stdout)
|
||||||
|
assert envelope['error']['kind'] == 'parse'
|
||||||
|
|
||||||
|
def test_missing_required_arg_text_mode_exits_2(self) -> None:
|
||||||
|
"""Text mode: argparse default exit 2 for missing required arg."""
|
||||||
|
result = _run(['exec-command']) # missing name + prompt
|
||||||
|
assert result.returncode == 2, (
|
||||||
|
f'text mode should exit 2, got {result.returncode}'
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_missing_required_arg_json_mode_exits_1(self) -> None:
|
||||||
|
"""JSON mode: normalized exit 1 for parse error."""
|
||||||
|
result = _run(['exec-command', '--output-format', 'json'])
|
||||||
|
assert result.returncode == 1, (
|
||||||
|
f'JSON mode should exit 1, got {result.returncode}'
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_success_path_identical_in_both_modes(self) -> None:
|
||||||
|
"""Success exit codes are identical in both modes."""
|
||||||
|
text_result = _run(['list-sessions'])
|
||||||
|
json_result = _run(['list-sessions', '--output-format', 'json'])
|
||||||
|
assert text_result.returncode == json_result.returncode == 0, (
|
||||||
|
f'success exit should be 0 in both modes: '
|
||||||
|
f'text={text_result.returncode}, json={json_result.returncode}'
|
||||||
|
)
|
||||||
|
|||||||
Reference in New Issue
Block a user