Skip to content

fix(sensors): harden provenance and diagnostics - #447

Open
JWThewes wants to merge 2 commits into
mainfrom
fix/364-v2-sensor-provenance
Open

JWThewes wants to merge 2 commits into
mainfrom
fix/364-v2-sensor-provenance

Conversation

@JWThewes

@JWThewes JWThewes commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Problem / Motivation

PR #364 identified a valid v2 defect: sensor harness failures could be reported as code-quality failures, while incomplete changed-file provenance could either suppress required sensors or inspect the wrong projects. The submitted branch was not safe to merge because it double-prefixed multi-repository paths, lost rename sources, treated unknown provenance as an empty known list, and accepted malformed successful sensor output.

This is a maintainer-owned replacement, rebuilt on current main rather than merging the original head.

Why it matters

Incorrect provenance can make blocking sensors fail open or hold unrelated stages. Unbounded or unredacted harness diagnostics can also persist credentials and create avoidable performance and UI risks.

What changed

Provenance and selection design

  • Models changed-file provenance explicitly as known or unknown, including the reason for unknown state.
  • Converts repository-relative paths to workspace-relative paths exactly once and preserves already-normalized multi-repository paths.
  • Preserves both source and destination endpoints for staged, retry, unpushed, and cross-project renames.
  • Handles missing/stale remote refs and aggregates multiple unpushed commits safely.
  • Carries provenance through commitAndPushAllrunStage → sensor selection and through orchestrator feedback persistence.
  • Preserves unknown provenance in the orchestrator consumer as changedFiles: null plus changedFileProvenance: { state: "unknown", reason }; unknown is never encoded or reported as []/none.
  • Runs blocking sensors when provenance is unknown, while skipping definitively inapplicable sensors.

Security and contract design

  • Gives sensor children an explicit minimal environment instead of inheriting repository credentials and unrelated process state.
  • Redacts credential-shaped diagnostic values before logging, persistence, rendering, and deduplication; dedupe keys are derived from redacted text.
  • Strictly validates verdictMode, scope, and safe relative projectConfig values.
  • Treats empty, malformed, schema-invalid, or unsupported stdout JSON as bounded INCONCLUSIVE, never PASS.

Bounded diagnostics

  • Replaces repeated stringify-and-pop trimming with deterministic bounded rendering and accurate represented-file omission counts.
  • Adds accessible frontend rendering for bounded harness diagnostics, omission metadata, redacted content, and empty details.

Production-shaped seam coverage

The replacement now exercises the real orchestration seam rather than only helper functions:

  • Git result → commitAndPushAllrunStage → sensor selection preserves both endpoints of a cross-repository rename, scopes the affected project files correctly, and rejects double-prefixing.
  • Exit 127 / tool unavailable becomes blocking INCONCLUSIVE, holds the stage, retains two changed files, and redacts both secrets before persistence and broadcast.
  • The original generic nonzero script failure (exit 1 with captured stderr) becomes blocking INCONCLUSIVE, holds the stage, retains the changed-file scope, and persists only redacted stderr.
  • Git diff failure remains explicit unknown provenance with its reason, widens inspection to all applicable files, and cannot be collapsed to an empty known list.
  • Exit 0 with invalid stdout JSON remains bounded INCONCLUSIVE, never PASS.

Attribution

The original valid sensor-provenance direction was contributed by Olivier Rossant in PR #364. This replacement preserves that credit while reworking the implementation to address the correctness, security, provenance, and performance gaps found during review.

Validation

Baseline SHA: 671502e5da02a2fe182d3043462ed75d17c7d564

Current head SHA: 93760487ead209376dbe93089ba6142222cdb33f

Validated with repository-pinned Node v22.22.1 and Vitest 4.1.10 (frontend Vitest 4.1.9):

Focused remediation suites

  • Orchestrator provenance consumer: 1/1 file, 14/14 tests passed.
  • Git-to-stage and production-orchestration seam: 1/1 file, 147/147 tests passed.
  • Assertions cover known-empty versus unknown provenance persistence, cross-repository rename scoping, no double-prefixing, exit 127, generic exit 1 with stderr, invalid successful JSON, changed-file scoping, stage hold behavior, and pre-persistence/pre-broadcast credential redaction.

Affected regressions

  • Backend: 88 files and 1,515 tests passed; 1 file / 3 explicitly container-runtime-dependent tests skipped; 0 failures.
  • Included suites: sensor runner 45/45, orchestrator replay 6/6, orchestrator 93/93, run-stage 147/147, v2 process 94/94, v2 execution plan 67/67, section integration 3/3, section 14/14, v2 sensor contract 42/42, and Git engine 89/89.
  • Frontend diagnostics and adjacent intent regression: 2/2 files, 30/30 tests passed (SensorDiagnostics 8/8; UnitLaneBoard 22/22).

Final quality gates

  • Root format: 834 files checked, passed.
  • Frontend format: 386 files checked, passed.
  • Root lint: 735 files checked, 0 errors and 16 pre-existing warnings outside the remediation scope.
  • Frontend lint: 375 files checked, 0 errors and 5 pre-existing warnings.
  • Frontend typecheck passed.
  • Frontend production build passed.
  • Secretlint passed with no findings.
  • Git diff check passed.

Local hook and container-runtime evidence

  • The committed Husky hook was not changed (sha256:03ec8be4211f2c0980329c3954e3b17c3c3089be23ab74bd202672f13cc6126c).
  • No supported local container runtime was available: Docker, Podman, Nerdctl, Finch, and Colima CLIs and conventional runtime sockets were absent; a direct Testcontainers probe failed with Could not find a working container runtime strategy.
  • Therefore the native final Husky command, npx vitest run --changed --passWithNoTests, is environmentally blocked and is not claimed green because the root Vitest setup unconditionally starts Gremlin and DynamoDB Local containers.
  • A temporary untracked adapter preserved the non-container hook phases and replaced only that unavailable final phase with the focused no-container suites. It passed lint-staged (oxfmt, oxlint, secretlint), audits at the configured threshold, 14/14 section tests, 147/147 run-stage tests, 1,515 backend tests (3 container-only skipped), and 30/30 frontend tests. The adapter was removed, the index restored, and the repository hook restored byte-for-byte.
  • Container-backed validation remains mandatory and must pass on this exact head in CI before the replacement is considered review-ready or PR fix(sensors): stop reporting harness failures as code-quality FAILs #364 is closed as superseded.

Residual risks

  • Local validation cannot make a green claim for the 3 container-runtime-dependent tests or the native container-backed Husky phase; CI is the required authority for both.
  • The root production audit reports one moderate qs advisory; the configured high-severity threshold passed, and the frontend audit reported 0 vulnerabilities.
  • No visual screenshot was captured for the bounded diagnostics UI; focused accessibility/rendering coverage, typecheck, and the production build passed.

Supersedes #364

no linked issue: this PR replaces an existing pull request rather than resolving a separately tracked issue.

Maintainer-owned hardened port of the valid sensor provenance work originally contributed by Olivier Rossant in PR #364.

This reworks the implementation rather than copying it unchanged, adding explicit provenance states, safe rename and history handling, strict verdict parsing, credential-safe bounded diagnostics, contract validation, and regression coverage.
Retain explicit known and unknown changed-file provenance in the orchestrator consumer, and add production-shaped regressions for Git-to-stage selection, generic harness failures, changed-file scoping, and diagnostic redaction.

Builds on and credits Olivier Rossant's original sensor-harness work in PR #364.
@jeromevdl jeromevdl added the bug Something isn't working label Sep 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants