fix(sieve): stop classing every undeclared exec exit as network (#562)

CHANGELOG.md

@@ -448,6 +448,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   `operator_config`, `trust`, `ignored_repository_settings`,
   `unknown_assertions`, `summary`, `results`, and `warnings` when present)
   instead of a bare list of results.
+- An `exec` step whose command exits with a code the step does not declare
+  reports error class `unexpected_exit` instead of `network` unless stderr
+  shows a connection error (name resolution, refused or reset connection,
+  unreachable host, TLS or certificate failure). Exit code 127 or "command
+  not found" reports `missing_tool`. The step's message gives the exit code,
+  the command, and the start of stderr. Before, `grep` exiting 2 on a missing
+  directory or `git` exiting 128 outside a repository was reported as a
+  network failure (#562).
 
 ### Fixed
 

docs/SECURITY_GUIDE.md

@@ -568,6 +568,8 @@ When reviewing a framework TOML or plugin change, treat any new `existence = tru
 
 A step that could not measure returns ERROR with a class and cause: `auth` (401/403), `rate_limit` (429 or a rate-limit 403), `unavailable` (5xx, transport failure, undeclared status), `missing_tool` (an absent binary or required MCP server), or `evaluation`. ERROR never concludes FAIL: later steps still run, and if none concludes the control ends ERROR. A platform response proves failure only when the `gh_api` step lists its status in `fail_on_status` (for example 404 for "no branch protection"), and a rate limit never does. A token that cannot read a setting therefore shows up as ERROR `[auth]`, not as a failing project. ERROR is non-compliant.
 
+An `exec` command that exits with a code its step does not declare concludes nothing either. Its class comes from the exit code and stderr: `rate_limit`, `auth`, `missing_tool` (exit 127 or "command not found"), `network` only when stderr shows a connection error, and otherwise `unexpected_exit`, whose message gives the exit code, the command, and the start of stderr.
+
 ### PASS Candidates Confirmed by the Operator
 
 Controls that need judgment of document content end `PENDING` (`pending.kind = "llm_judgment"`) after the deterministic steps. A judgment comes from the harness's model step or from a coding agent through the `submit_judgment` MCP tool; both follow the same rules:

docs/architecture/framework-design.md

@@ -1,8 +1,8 @@
 # Darnit Framework Design Specification
 
-> **Version**: 1.0.0-alpha.12
+> **Version**: 1.0.0-alpha.13
 > **Status**: Authoritative
-> **Last Updated**: 2026-10-04
+> **Last Updated**: 2026-10-06
 
 This specification defines the authoritative design of the Darnit framework, including the sieve orchestrator, TOML schema, built-in pass types, remediation actions, and plugin protocol.
 
@@ -440,6 +440,18 @@ These are the step type's declared `settings` (section 3.0.3); any other key fai
 
 **Broken measurements**: a command whose binary is not installed is ERROR, class `missing_tool`; a timeout is ERROR, class `timeout`. Neither is FAIL.
 
+**Undeclared exit codes**: an exit code in neither `pass_exit_codes` nor `fail_exit_codes` concludes nothing; the step is evidence only and carries a class taken from the command's exit code and stderr, first match wins (#562):
+
+| Evidence | Class |
+|----------|-------|
+| A rate-limit message (`api rate limit exceeded`, `secondary rate limit`, `abuse detection mechanism`) | `rate_limit` |
+| An authentication message (`http 401`, `bad credentials`, `requires authentication`, `gh auth login`, `authentication token`, `not logged in`) | `auth` |
+| Exit code 127, or stderr says the command was not found | `missing_tool` |
+| A connection error: name resolution, connection refused, reset, or timed out, network or host unreachable, a TLS or certificate failure, or git's `unable to access` | `network` |
+| Anything else | `unexpected_exit` |
+
+The step's message names the command, the exit code, and the start of stderr, so an operator can act on an `unexpected_exit` without rerunning the command.
+
 #### Scenario: CEL expression evaluated
 - **WHEN** an `exec` step has an `expr` field and the exit code gives PASS or FAIL
 - **THEN** the expression MUST be evaluated after the command, and the step result MUST follow the outcome rules of section 3.7
@@ -450,6 +462,11 @@ These are the step type's declared `settings` (section 3.0.3); any other key fai
 - **THEN** the handler MUST evaluate the exit code against `pass_exit_codes` and `fail_exit_codes`
 - **AND** an exit code in neither list MUST NOT give PASS or FAIL
 
+#### Scenario: Undeclared exit code with no identified cause
+- **WHEN** an `exec` command exits with a code in neither list and its stderr shows no rate limit, authentication failure, missing command, or connection error (for example `grep` exiting 2 on a missing directory, or `git` exiting 128 outside a repository)
+- **THEN** the step MUST carry class `unexpected_exit`, not `network`
+- **AND** its message MUST name the command, the exit code, and an excerpt of stderr
+
 ### 3.4 regex Handler
 
 **Purpose**: Regex-based content analysis
@@ -1236,7 +1253,7 @@ Result fields beyond `status`:
 |-------|-------------|
 | `authority` | `dispositive` (a step concluded within its effective set), `suggestive` (no step concluded, or a model finding), `asserted` (a person confirmed it) |
 | `concluded_by` | Step handler name, `llm_judgment`, `confirmation`, `inferred_from`, or `none` |
-| `error` | `{class, cause}`; present on every ERROR. Classes include `auth`, `rate_limit`, `unavailable`, `missing_tool`, `evaluation`, and the feature 036 classes |
+| `error` | `{class, cause}`; present on every ERROR. Classes include `auth`, `rate_limit`, `unavailable`, `missing_tool`, `evaluation`, `unexpected_exit` (#562), and the feature 036 classes |
 | `pending` | `{kind}`; present on every PENDING |
 | `candidate` | Present with `pending.kind = "confirmation"`: `verdict = "pass"`, `reasoning`, `cited_evidence`, `model`, `model_version`, `evidence_digest`, `source` (`harness` or `mcp_agent`) |
 | `confirmation` | Present on a PASS from a confirmed candidate: `confirmed_by`, `confirmed_at`, `expires_at` |
@@ -2301,6 +2318,7 @@ The following requirements have been superseded: by the handler dispatch archite
 
 | Version | Date | Changes |
 |---------|------|---------|
+| 1.0.0-alpha.13 | 2026-10-06 | Error class `unexpected_exit`: an `exec` step whose undeclared exit code has no identified cause no longer reports `network`; exit code 127 reports `missing_tool` (Sections 3.3, 5.2; #562) |
 | 1.0.0-alpha.12 | 2026-10-04 | Repository-level `.baseline.toml` is no longer read: one notice points at `darnit config migrate`, framework selection only by `--framework` or a tool argument (Sections 2.3, 10.5, 14.4, 15.1; Appendix C) |
 | 1.0.0-alpha.11 | 2026-10-04 | Close remaining false-PASS paths (feature 044): an expression that cannot be evaluated or is not boolean makes the step ERROR, expression names per step type with usable `project` values and a repository-aware `file_exists`, load-time expression reference check (Section 3.7); step registration declares `settings` and `expression_names`, plugins cannot replace a registered step type, and unknown control keys, unknown step keys, and unregistered step types fail loading (Sections 2.3, 3.0.3); reproducibility step types conclude only FAIL (Sections 3.0.1, 12); `expr_decides`, an expression that decides on a handler PASS (Sections 3.0.3, 3.7); `gh_api` `evidence_fields`, required for personal records (Section 3.8); `file_must_exist` replaced by the registered `file_exists` (Section 3.2); field tables corrected to what each handler reads (Sections 3.3-3.5) |
 | 1.0.0-alpha.10 | 2026-10-02 | Remediation safety (feature 043): plan/apply protocol and single writer (Section 4.2), `platform_setting` (4.5), exec `effects`/`offline` and no platform state from exec (4.4), `file_create.project_reference` (4.3), remediation policy, digest-bound approval, outcomes, re-check, run manifest, and version-control rules (Section 15); removed `api_call`, `requires_confirmation`, `dry_run_supported`, `dry_run_command` (Appendix C) |

packages/darnit/src/darnit/core/error_class.py

@@ -1,6 +1,7 @@
 """Environmental failure classification.
 
-Feature 036. See specs/036-tier2-error-class/contracts/error-class.md.
+Feature 036. See specs/036-tier2-error-class/contracts/error-class.md and
+its #562 addendum, contracts/error-class-addendum-562.md.
 
 When a sieve pass cannot run to completion -- the network is unreachable,
 the auth token expired, a subprocess timed out, a required binary is
@@ -16,31 +17,34 @@
 check ran and the repository does not comply" -- that stays a bare
 FAIL/WARN with no ``error_class``.
 
-=============  ===============================================================
-Value          Meaning
-=============  ===============================================================
-network        Host unreachable, DNS failure, TLS error, MCP server unusable,
-               or any non-zero subprocess exit whose stderr matched no
-               more-specific pattern.
-auth           HTTP 401, bad credentials, expired token, "requires
-               authentication", or MCP plugin signature-verification failure.
-timeout        Subprocess exceeded its ``timeout`` budget, or an MCP tool
-               call exceeded ``MCP_DEFAULT_TIMEOUT_SECONDS``.
-rate_limit     GitHub primary or secondary rate limit, or abuse-detection
-               throttle.
-not_found      A control names an MCP server the operator never configured.
-               (Before feature 041 this also covered an absent binary; step
-               results now report that as ``missing_tool``.)
-crashed        A handler raised an unexpected exception, or an MCP tool
-               returned unparseable output. The handler did not complete
-               cleanly.
-missing_tool   Feature 041. A tool the step needs (``gh``, a command's
-               binary, a required MCP server executable) is not installed.
-unavailable    Feature 041. The platform API answered with a 5xx or a status
-               the step did not declare, or the request never got a response.
-evaluation     Feature 041. The step ran but its result could not be
-               evaluated (for example a CEL ``expr`` error over a response).
-=============  ===============================================================
+===============  =============================================================
+Value            Meaning
+===============  =============================================================
+network          Host unreachable, DNS failure, connection refused or reset,
+                 TLS error, or MCP server unusable. An ``exec`` step reports it
+                 only when stderr shows a connection error (#562).
+auth             HTTP 401, bad credentials, expired token, "requires
+                 authentication", or MCP plugin signature-verification failure.
+timeout          Subprocess exceeded its ``timeout`` budget, or an MCP tool
+                 call exceeded ``MCP_DEFAULT_TIMEOUT_SECONDS``.
+rate_limit       GitHub primary or secondary rate limit, or abuse-detection
+                 throttle.
+not_found        A control names an MCP server the operator never configured.
+                 (Before feature 041 this also covered an absent binary; step
+                 results now report that as ``missing_tool``.)
+crashed          A handler raised an unexpected exception, or an MCP tool
+                 returned unparseable output. The handler did not complete
+                 cleanly.
+missing_tool     Feature 041. A tool the step needs (``gh``, a command's
+                 binary, a required MCP server executable) is not installed,
+                 or a command exited 127 or said "command not found".
+unavailable      Feature 041. The platform API answered with a 5xx or a status
+                 the step did not declare, or the request never got a response.
+evaluation       Feature 041. The step ran but its result could not be
+                 evaluated (for example a CEL ``expr`` error over a response).
+unexpected_exit  #562. A command exited with a code its step did not declare,
+                 and nothing in its output identified the cause.
+===============  =============================================================
 
 Two names are exported because ``typing.Literal`` is erased at runtime
 and enforces nothing on its own. ``ErrorClass`` gives static-analysis
@@ -74,6 +78,7 @@
     "missing_tool",
     "unavailable",
     "evaluation",
+    "unexpected_exit",
 ]
 
 # Runtime-checkable companion to ``ErrorClass``. See module docstring for
@@ -89,6 +94,7 @@
         "missing_tool",
         "unavailable",
         "evaluation",
+        "unexpected_exit",
     )
 )
 

packages/darnit/src/darnit/sieve/builtin_handlers.py

@@ -54,11 +54,12 @@
 # Feature 036: environmental-failure classification
 # =============================================================================
 
-# GitHub-only stderr patterns for v0 (clarify Q4). The `exec` handler sees
-# only stdout/stderr/exit-code -- it has no access to response headers -- so
-# classification is substring matching against `gh` CLI stderr shape. Other
-# exec targets (git, curl, syft, cosign) fall through to `network`; per-target
-# pattern packs are a follow-up if real audits show they are needed.
+# The `exec` handler sees only stdout/stderr/exit-code -- it has no access to
+# response headers -- so classification is substring matching against stderr.
+# The rate-limit and auth patterns follow `gh` CLI stderr shape (clarify Q4).
+# A failure no pattern identifies is `unexpected_exit`, not `network` (#562):
+# most undeclared exits (grep on a missing directory, git outside a
+# repository) never touch the network.
 #
 # Rate-limit is checked BEFORE auth: GitHub answers 403 for both rate limits
 # and permission failures, and the rate-limit body is the more specific signal.
@@ -74,22 +75,75 @@
     "requires authentication",
     "gh auth login",
     "authentication token",
+    "not logged in",
 )
 
 
-def _classify_exec_failure(stderr: str) -> ErrorClass:
-    """Classify a non-zero-exit subprocess failure from its stderr.
+_MISSING_TOOL_EXIT_CODE = 127
+
+_MISSING_TOOL_PATTERNS: tuple[str, ...] = ("command not found",)
+
+_NETWORK_PATTERNS: tuple[str, ...] = (
+    "could not resolve host",
+    "name or service not known",
+    "temporary failure in name resolution",
+    "nodename nor servname provided",
+    "no such host",
+    "connection refused",
+    "connection reset",
+    "connection was reset",
+    "network is unreachable",
+    "no route to host",
+    "connection timed out",
+    "failed to connect to",
+    "i/o timeout",
+    "error connecting to",
+    "tls handshake",
+    "ssl_connect",
+    "gnutls_handshake",
+    "ssl certificate problem",
+    "certificate verify failed",
+    "x509:",
+    "server certificate verification failed",
+    "unable to access 'http",
+)
+
+# git's "unable to access" also wraps an HTTP status the server sent back;
+# an answer from the server is not a connection failure.
+_HTTP_RESPONSE_PATTERNS: tuple[str, ...] = ("the requested url returned error",)
+
+_STDERR_EXCERPT_CHARS = 200
+
+
+def _classify_exec_failure(stderr: str, exit_code: int | None = None, executable: str | None = None) -> ErrorClass:
+    """Classify an undeclared-exit subprocess failure from its exit code and stderr.
 
     Only called on paths where the command did not complete as expected.
     Callers must NOT invoke this for a declared ``fail_exit_codes`` hit --
     that is a check that ran and concluded, not an environmental failure.
+    ``executable`` lets a shell's "<name>: not found" count as a missing tool
+    without reading every "not found" (an HTTP 404, say) as one.
     """
     haystack = (stderr or "").lower()
     if any(p in haystack for p in _GH_RATE_LIMIT_PATTERNS):
         return "rate_limit"
     if any(p in haystack for p in _GH_AUTH_PATTERNS):
         return "auth"
-    return "network"
+    if exit_code == _MISSING_TOOL_EXIT_CODE or any(p in haystack for p in _MISSING_TOOL_PATTERNS):
+        return "missing_tool"
+    if executable:
+        name = re.escape(os.path.basename(executable).lower())
+        if re.search(rf"(^|[\s:]){name}: not found\s*$", haystack, re.MULTILINE):
+            return "missing_tool"
+    if any(p in haystack for p in _NETWORK_PATTERNS) and not any(p in haystack for p in _HTTP_RESPONSE_PATTERNS):
+        return "network"
+    return "unexpected_exit"
+
+
+def _excerpt(text: str) -> str:
+    """The start of ``text`` on one ASCII line, for an operator-facing cause."""
+    line = " ".join((text or "").split())[:_STDERR_EXCERPT_CHARS]
+    return line.encode("ascii", "replace").decode("ascii")
 
 
 def _log_environmental_failure(
@@ -369,10 +423,14 @@ def exec_handler(config: dict[str, Any], context: HandlerContext) -> HandlerResu
         )
     else:
         # Undeclared exit code: we cannot tell whether the check concluded.
-        # Classify from stderr so the operator can distinguish a rate limit
-        # or expired token from a genuine non-compliance signal.
-        error_class = _classify_exec_failure(evidence["stderr"])
-        message = f"Command exited with unexpected code {proc.returncode}"
+        # Classify from exit code and stderr so the operator can distinguish
+        # a rate limit or expired token from a genuine non-compliance signal.
+        error_class = _classify_exec_failure(evidence["stderr"], proc.returncode, resolved_cmd[0])
+        excerpt = _excerpt(evidence["stderr"]) or "(no stderr)"
+        message = (
+            f"Command exited with unexpected code {proc.returncode}: "
+            f"{_excerpt(' '.join(resolved_cmd))}; stderr: {excerpt}"
+        )
         _log_environmental_failure(context.control_id, "exec", error_class, message)
         return HandlerResult(
             status=HandlerResultStatus.INCONCLUSIVE,