Skip to content

Commit fd3f09f

Browse files
committed
fix(perf): step 7 — three defects the first Windows CI run exposed
The capture tool shipped describing its Windows path as unverified. One CI run on windows-latest made most of it verified and found two defects in it, plus a third in the controls. virtualization reported "systemd-detect-virt absent or unclear and no DMI identity under /sys" — on Windows. That is true of every Windows machine ever built, and it reads as though a probe ran when nothing Windows-specific had been tried at all. A stated reason naming the wrong operating system is worse than no reason, because "unavailable with a reason" is the contract that makes an absence trustworthy. _virtualization() and _power_policy() are now branched by platform, and _virtualization() queries Win32_ComputerSystem through CIM. The general rule is now a control. envcapture-reason-fits-platform requires every unavailable reason to name only mechanisms that exist on the platform it was produced on, and it caught a second instance while being written: the repaired _cpu_model() reason named /proc/cpuinfo and Win32_Processor in one sentence. cpu_model fell through a dead tool in silence. wmic is gone from recent Windows images, so the name arrived as platform.processor()'s family/model/stepping string rather than the processor's name — observed, but nowhere near the "cpu model exact" the ruling asks for. The CIM query runs first now. A green run printed a failure. control_guard_reports exercised the guard by calling it and deleting the entry from the failure list afterwards, which left a literal FAIL[envcapture-guard-fixture] in every passing CI log — the exact string this repository's mutation scorer keys on, and the sort of line that teaches a reader to skip past FAIL. The guard is split into a pure guard_failure() returning the message and a guarded() that reports it, so the probe asserts on a returned string and prints nothing. Fifteen mutations now, all caught by their named control. Two of them mutate the control file rather than the tool: the guard is the durable answer to a defect recorded eight times, so it needs an adversary of its own. What the Windows run did establish: the registry MachineGuid read, powercfg and GlobalMemoryStatusEx all work. What stays unverified is capture on a DEDICATED Windows host, and the new CIM query, which has executed nowhere yet. The windows-fixture control now says that instead of claiming the branches are unrun. No clock ran. No frozen source touched; the harness digest is unchanged at 562a7f7232da. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CYLNQy6tLXqsV1CbuNqsSb
1 parent 78460f4 commit fd3f09f

3 files changed

Lines changed: 205 additions & 52 deletions

File tree

‎docs/notes/p022-263a-step7-environment-capture.md‎

Lines changed: 61 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -140,21 +140,24 @@ so it cannot later be presented as host-taken.
140140
## Controls
141141

142142
```text
143-
envcapture-guard-reports a control that raises is reported, never fatal
144-
envcapture-schema the declared key set, enforced both ways
145-
envcapture-field-contract observed/unavailable, with no third state
146-
envcapture-identity-split provenance can never become identity
147-
envcapture-assigned-id an owner-assigned id cannot be 'unavailable'
148-
envcapture-drift provenance moves freely; identity never does
149-
envcapture-no-measurement no clock, no resource accounting, by AST
150-
envcapture-windows-fixture the schema holds off Linux; the capture path does not
151-
envcapture-frozen-untouched this addition moved none of the three frozen digests
152-
envcapture-ci-provenance a CI-taken manifest says so and cannot hide it
143+
envcapture-guard-reports a control that raises is reported, never fatal
144+
envcapture-schema the declared key set, enforced both ways
145+
envcapture-field-contract observed/unavailable, with no third state
146+
envcapture-identity-split provenance can never become identity
147+
envcapture-assigned-id an owner-assigned id cannot be 'unavailable'
148+
envcapture-drift provenance moves freely; identity never does
149+
envcapture-no-measurement no clock, no resource accounting, by AST
150+
envcapture-reason-fits-platform an 'unavailable' reason names THIS platform
151+
envcapture-windows-fixture the schema holds off Linux
152+
envcapture-frozen-untouched this addition moved none of the three frozen digests
153+
envcapture-ci-provenance a CI-taken manifest says so and cannot hide it
153154
```
154155

155-
Twelve mutations of the tool, each declaring in advance which control must catch
156-
it, and each scored on a `FAIL` line from **that** control rather than on a
157-
non-zero exit.
156+
Fifteen mutations, each declaring in advance which control must catch it, and
157+
each scored on a `FAIL` line from **that** control rather than on a non-zero
158+
exit. Two of the fifteen mutate the **control file**, not the tool: the guard is
159+
the durable answer to a defect recorded eight times, so it needs its own
160+
adversary.
158161

159162
## Two defects the campaign found, both real
160163

@@ -174,11 +177,49 @@ turns any escaping exception into a reported failure for that control and lets
174177
the run continue, and `envcapture-guard-reports` makes restoring the bare call a
175178
test failure rather than a story in a session log.
176179

177-
## What is NOT verified
180+
## What the first Windows CI run found, and what it cost
178181

179-
The Windows capture path. The registry read, `powercfg` and
180-
`GlobalMemoryStatusEx` branches have never executed. `envcapture-windows-fixture`
181-
exercises the **schema and the drift rule** on a synthetic non-Linux manifest and
182-
says so in its own success message; it is not evidence that Windows capture
183-
works. That stays unverified until a dedicated Windows host exists, which is the
184-
same gate the first clock waits behind.
182+
The Windows capture path was described as unverified, and one CI run on
183+
`windows-latest` (Server 2025) made most of it verified and found two defects.
184+
185+
| field | Windows result |
186+
|---|---|
187+
| `host_fingerprint` | observed — the registry `MachineGuid` read works |
188+
| `os_build`, `kernel` | observed — `Windows 2025Server 10.0.26100 SP0 …` |
189+
| `memory_bytes` | observed — `GlobalMemoryStatusEx` works |
190+
| `power_policy` | observed — `powercfg` returns the active scheme |
191+
| `cpu_model` | observed, but **weak** |
192+
| `virtualization` | **unavailable, for a Linux reason** |
193+
194+
**`virtualization` reported "systemd-detect-virt absent or unclear and no DMI
195+
identity under /sys" — on Windows.** That is true of every Windows machine ever
196+
built, and it reads as though a probe ran when nothing Windows-specific had been
197+
tried at all. A stated reason naming the wrong operating system is worse than no
198+
reason. `_virtualization()` is now platform-branched and queries
199+
`Win32_ComputerSystem` through CIM, and `envcapture-reason-fits-platform`
200+
enforces the general rule: an `unavailable` reason may only name mechanisms that
201+
exist on the platform it was produced on. Designing that control immediately
202+
caught a second instance — `_cpu_model`'s repaired reason named both `/proc/cpuinfo`
203+
and `Win32_Processor` in one sentence.
204+
205+
**`cpu_model` fell through a dead tool in silence.** `wmic` is gone from recent
206+
Windows images, so the model name arrived as `platform.processor()`'s
207+
`AMD64 Family 25 Model 1 Stepping 1, AuthenticAMD` — a family string, not the
208+
processor's name, and nowhere near the "cpu model exact" the ruling asks for. The
209+
CIM query now runs first.
210+
211+
**A green run printed a failure.** `control_guard_reports` exercised the guard by
212+
calling it and then deleting the entry from the failure list, which left a literal
213+
`FAIL[envcapture-guard-fixture]` in every passing CI log — exactly the string this
214+
repository's mutation scorer keys on, and the sort of line that teaches readers to
215+
ignore `FAIL`. The guard is now split into a pure `guard_failure()` that returns
216+
the message and a `guarded()` that reports it, so the probe asserts on a returned
217+
string and prints nothing.
218+
219+
## What is still NOT verified
220+
221+
Capture on a **dedicated** Windows host. A hosted runner is not one, and the CIM
222+
virtualization query added above has never executed anywhere, since this container
223+
is not Windows. `envcapture-windows-fixture` exercises the schema and drift rule
224+
on a synthetic manifest and now says exactly that rather than claiming the
225+
Windows branches are unrun.

‎scripts/step7/envcapture.py‎

Lines changed: 51 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -168,6 +168,33 @@ def _windows_machine_guid() -> str | None:
168168
return str(value) if value else None
169169

170170

171+
def _windows_cim(class_name: str, props: tuple[str, ...]) -> dict[str, str] | None:
172+
"""One CIM query for identity. No shell, no timing, no clock.
173+
174+
`wmic` is gone from recent Windows images — the first CI run of this tool on
175+
windows-latest (Server 2025) fell straight through it — so the model name
176+
arrived as `platform.processor()`'s family/model/stepping string instead of
177+
the processor's actual name. CIM is the supported replacement.
178+
"""
179+
if os.name != "nt":
180+
return None
181+
script = "; ".join(
182+
[f"$o = Get-CimInstance -ClassName {class_name} | Select-Object -First 1"]
183+
+ [f"Write-Output ('{prop}=' + $o.{prop})" for prop in props])
184+
for exe in ("powershell", "pwsh"):
185+
found = _tool([exe, "-NoProfile", "-NonInteractive", "-Command", script])
186+
if found is None or found[0] != 0:
187+
continue
188+
values: dict[str, str] = {}
189+
for line in found[1].splitlines():
190+
key, _, value = line.partition("=")
191+
if key.strip() in props and value.strip():
192+
values[key.strip()] = value.strip()
193+
if values:
194+
return values
195+
return None
196+
197+
171198
def _os_build() -> dict[str, object]:
172199
if os.name == "nt":
173200
release, version, csd, ptype = platform.win32_ver()
@@ -208,14 +235,15 @@ def _cpu_model() -> dict[str, object]:
208235
if value.strip():
209236
return observed(value.strip())
210237
if os.name == "nt":
211-
found = _tool(["wmic", "cpu", "get", "name"])
212-
if found and found[0] == 0:
213-
lines = [ln.strip() for ln in found[1].splitlines() if ln.strip()]
214-
if len(lines) > 1:
215-
return observed(lines[1])
238+
cim = _windows_cim("Win32_Processor", ("Name",))
239+
if cim and cim.get("Name"):
240+
return observed(cim["Name"])
216241
processor = platform.processor()
217242
if processor:
218243
return observed(processor)
244+
if os.name == "nt":
245+
return unavailable("no Win32_Processor Name through powershell or pwsh, and "
246+
"platform.processor() is empty")
219247
return unavailable("no model name in /proc/cpuinfo and platform.processor() is empty")
220248

221249

@@ -270,7 +298,21 @@ class _Status(ctypes.Structure):
270298

271299

272300
def _virtualization() -> dict[str, object]:
273-
"""Exact, or explicitly classified. Never a guessed 'probably bare metal'."""
301+
"""Exact, or explicitly classified. Never a guessed 'probably bare metal'.
302+
303+
Platform-branched, because an `unavailable` reason that names the other
304+
platform's mechanisms is a false statement about what was tried. The first
305+
Windows CI run of this tool reported "systemd-detect-virt absent ... no DMI
306+
identity under /sys" on Windows Server 2025, which is true of every Windows
307+
machine ever built and told the reader nothing.
308+
"""
309+
if os.name == "nt":
310+
cim = _windows_cim("Win32_ComputerSystem", ("Manufacturer", "Model"))
311+
if cim:
312+
return observed(f"Win32_ComputerSystem Manufacturer={cim.get('Manufacturer', '?')} "
313+
f"Model={cim.get('Model', '?')}")
314+
return unavailable("Win32_ComputerSystem is unreadable through powershell and pwsh, "
315+
"so the virtualization boundary is not classified on this host")
274316
found = _tool(["systemd-detect-virt"])
275317
if found is not None:
276318
# Exit 1 with "none" is the tool answering, not failing.
@@ -285,14 +327,14 @@ def _virtualization() -> dict[str, object]:
285327

286328

287329
def _power_policy() -> dict[str, object]:
288-
governor = _text("/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor")
289-
if governor and governor.strip():
290-
return observed(f"cpufreq scaling_governor={governor.strip()}")
291330
if os.name == "nt":
292331
found = _tool(["powercfg", "/getactivescheme"])
293332
if found and found[0] == 0 and found[1].strip():
294333
return observed(found[1].strip())
295334
return unavailable("powercfg /getactivescheme produced no active scheme")
335+
governor = _text("/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor")
336+
if governor and governor.strip():
337+
return observed(f"cpufreq scaling_governor={governor.strip()}")
296338
return unavailable(
297339
"no /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor on this host; CPU "
298340
"frequency policy is not observable here")

‎tests/test_step7_envcapture.py‎

Lines changed: 93 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@
1111
envcapture-assigned-id an owner-assigned id cannot be 'unavailable'
1212
envcapture-drift provenance moves freely; identity never does
1313
envcapture-no-measurement no clock, no resource observation, by AST
14+
envcapture-reason-fits-platform an 'unavailable' reason names THIS platform
1415
envcapture-windows-fixture the schema holds off Linux; the capture path does not
1516
envcapture-frozen-untouched this addition moved none of the three frozen digests
1617
envcapture-ci-provenance a CI-taken manifest says so and cannot hide it
@@ -329,6 +330,61 @@ def control_no_measurement() -> None:
329330
}
330331

331332

333+
# Mechanism names that exist on one platform only. A reason naming the other
334+
# platform's mechanisms is a false statement about what was tried.
335+
POSIX_ONLY = ("/sys/", "/proc/", "systemd-detect-virt", "scaling_governor", "cpufreq")
336+
WINDOWS_ONLY = ("powercfg", "Win32_", "MachineGuid", "GlobalMemoryStatusEx",
337+
"powershell", "pwsh")
338+
339+
340+
def foreign_mechanisms(reason: str, *, on_windows: bool) -> tuple[str, ...]:
341+
"""Written once, used twice: on the live manifest and on a synthetic reason."""
342+
foreign = POSIX_ONLY if on_windows else WINDOWS_ONLY
343+
return tuple(token for token in foreign if token in reason)
344+
345+
346+
def control_reason_fits_platform() -> None:
347+
"""An 'unavailable' reason must describe THIS platform's mechanisms.
348+
349+
The first Windows CI run of this tool reported `virtualization` unavailable
350+
because "systemd-detect-virt absent ... and no DMI identity under /sys" —
351+
true of every Windows machine ever built, and silent about the fact that
352+
nothing Windows-specific had been tried at all. A stated reason that names
353+
the wrong operating system is worse than no reason, because it reads as
354+
though a probe ran.
355+
"""
356+
on_windows = os.name == "nt"
357+
manifest = _live()
358+
problems: list[str] = []
359+
for name, field in manifest["identity"].items(): # type: ignore[union-attr]
360+
if field.get("status") != ec.UNAVAILABLE:
361+
continue
362+
found = foreign_mechanisms(str(field.get("reason", "")), on_windows=on_windows)
363+
if found:
364+
problems.append(f"{name} blames {', '.join(found)}")
365+
if problems:
366+
fail("envcapture-reason-fits-platform",
367+
f"on {'Windows' if on_windows else 'POSIX'}, "
368+
+ "; ".join(problems))
369+
return
370+
371+
# The scanner must actually catch the defect that prompted it, or its silence
372+
# on the live manifest proves nothing.
373+
regression = "systemd-detect-virt absent or unclear and no DMI identity under /sys"
374+
if not foreign_mechanisms(regression, on_windows=True):
375+
fail("envcapture-reason-fits-platform",
376+
"the scanner does not catch the exact Windows reason that prompted it")
377+
return
378+
if foreign_mechanisms(regression, on_windows=False):
379+
fail("envcapture-reason-fits-platform",
380+
"the scanner calls a POSIX reason foreign on POSIX")
381+
return
382+
ok("envcapture-reason-fits-platform",
383+
f"every unavailable reason on {'Windows' if on_windows else 'POSIX'} names a mechanism "
384+
"that exists here, and the Windows virtualization reason that shipped in the first "
385+
"version is still caught")
386+
387+
332388
def control_windows_fixture() -> None:
333389
problems = ec.validate(WINDOWS_FIXTURE)
334390
if problems:
@@ -341,9 +397,11 @@ def control_windows_fixture() -> None:
341397
fail("envcapture-windows-fixture", "a changed Windows power policy was not drift")
342398
return
343399
ok("envcapture-windows-fixture",
344-
"the schema and the drift rule hold on a non-Linux manifest. This does NOT exercise the "
345-
"Windows capture path: the registry, powercfg and GlobalMemoryStatusEx branches stay "
346-
"unrun until a dedicated Windows host exists, and are recorded as unverified")
400+
"the schema and the drift rule hold on a non-Linux manifest. This fixture does not run "
401+
"the Windows capture path; the CI selftest on windows-latest does, and its first run "
402+
"returned the registry fingerprint, powercfg and GlobalMemoryStatusEx but no "
403+
"virtualization, which is what added the CIM query. What stays unverified is capture on "
404+
"a DEDICATED Windows host, since a hosted runner is not one")
347405

348406

349407
def _git(*args: str) -> tuple[int, bytes]:
@@ -423,6 +481,23 @@ def control_ci_provenance() -> None:
423481
"so it cannot later be presented as taken on a dedicated measurement host")
424482

425483

484+
def guard_failure(control: Callable[[], None]) -> str | None:
485+
"""Run a control; return the message a failure should carry, or None.
486+
487+
Split from `guarded` so the probe below can exercise it WITHOUT printing a
488+
`FAIL[` line. The first version reported through `fail()` and then deleted
489+
the entry from the list — which left a literal `FAIL[envcapture-guard-fixture]`
490+
in every green CI log, exactly the string this repository's mutation scorer
491+
keys on. A green run must not print a failure.
492+
"""
493+
try:
494+
control()
495+
except Exception as exc: # the point is that nothing escapes
496+
return (f"the control raised {type(exc).__name__}: {exc}. A control that crashes "
497+
"before reporting is a traceback, not a finding")
498+
return None
499+
500+
426501
def guarded(name: str, control: Callable[[], None]) -> None:
427502
"""Report an escaping exception instead of dying of it.
428503
@@ -433,11 +508,9 @@ def guarded(name: str, control: Callable[[], None]) -> None:
433508
`envcapture-identity-split` — the control that owns exactly that defect —
434509
could say a word.
435510
"""
436-
try:
437-
control()
438-
except Exception as exc: # the point is that nothing escapes
439-
fail(name, f"the control raised {type(exc).__name__}: {exc}. A control that crashes "
440-
"before reporting is a traceback, not a finding")
511+
message = guard_failure(control)
512+
if message is not None:
513+
fail(name, message)
441514

442515

443516
def control_guard_reports() -> None:
@@ -446,33 +519,29 @@ def control_guard_reports() -> None:
446519
A scratchpad campaign that once watched a crash get reported is not a
447520
regression catcher; restoring the bare call in `run()` has to fail here.
448521
"""
449-
marker = "envcapture-guard-fixture"
450-
before_fails, before_passes = len(_FAILURES), len(_PASSES)
451-
452522
def always_raises() -> None:
453523
raise RuntimeError("a control that dies before reporting")
454524

455525
try:
456-
guarded(marker, always_raises)
526+
message = guard_failure(always_raises)
457527
except Exception as exc: # nothing may escape, including from the guard
458528
fail("envcapture-guard-reports",
459-
f"guarded() let {type(exc).__name__} escape, so one crashing control still "
529+
f"guard_failure() let {type(exc).__name__} escape, so one crashing control still "
460530
"kills every control after it")
461531
return
462-
recorded = [d for c, d in _FAILURES[before_fails:] if c == marker]
463-
# Take the fixture's own failure back out; it is a probe, not a finding.
464-
del _FAILURES[before_fails:]
465-
del _PASSES[before_passes:]
466-
if not recorded:
467-
fail("envcapture-guard-reports", "guarded() swallowed the exception without reporting it")
532+
if message is None:
533+
fail("envcapture-guard-reports", "a raising control was reported as clean")
468534
return
469-
if "RuntimeError" not in recorded[0]:
535+
if "RuntimeError" not in message:
470536
fail("envcapture-guard-reports",
471-
f"guarded() reported without naming the exception type: {recorded[0]!r}")
537+
f"the report does not name the exception type: {message!r}")
538+
return
539+
if guard_failure(lambda: None) is not None:
540+
fail("envcapture-guard-reports", "a clean control was reported as a failure")
472541
return
473542
ok("envcapture-guard-reports",
474-
"a control that raises is reported by name and the run continues, so the mutation that "
475-
"once killed the suite before its owning control could speak stays caught")
543+
"a control that raises is reported by name and the run continues, a clean one is not, "
544+
"and this probe prints no FAIL line of its own on a green run")
476545

477546

478547
def run() -> int:
@@ -483,6 +552,7 @@ def run() -> int:
483552
guarded("envcapture-assigned-id", control_assigned_id)
484553
guarded("envcapture-drift", control_drift)
485554
guarded("envcapture-no-measurement", control_no_measurement)
555+
guarded("envcapture-reason-fits-platform", control_reason_fits_platform)
486556
guarded("envcapture-windows-fixture", control_windows_fixture)
487557
guarded("envcapture-frozen-untouched", control_frozen_untouched)
488558
guarded("envcapture-ci-provenance", control_ci_provenance)

0 commit comments

Comments
 (0)