P-022 step 8: Rust-default cutover, rollback gate, and Python distribution removal #262

Parent: #250
Recommended tier: strong + owner decision

Status (2026-09-18) — Stage 3 COMPLETE and merged (PR #359 → 63148d0); Stage 4 open

Stage 1     DONE — opt-in Rust behind the launcher
Stage 2     DONE — Own.NET's own CI and dogfood on Rust
Stage 3     COMPLETE — three qualification points, kept apart:
            decision        f1d3681e96e31bba7a36dda09967d0af7c3e9b56
                            run 35312190310, 31/31 — established the decision
            implementation  b22543680da9e8fa6b2607435bb54a5437815bff
                            run 35320877147, 31/31 — after the repository-hygiene
                            commit untracked rust/target-fault (build output)
            terminal merge  1bf17e4b0f64f012909d5c7c762febf1253e198d
                            run 35322222621, 31/31 — the reconciliation-only head
                            of branch claude/p022-stage3-rust-cutover-q0n2q5
            public default: Rust on owen / own-check.sh / own-check.ps1 / Action
            rollback: explicit Python selection, tested — no silent fallback
            performance: DEFERRED BY OWNER — NOT MEASURED; #263 remains open
            merge to main: DONE — PR #359 merged 2026-09-18 as 63148d0b2bdd873d18234ac8fff782a3b6b78856
                            (a merge commit whose tree is 1bf17e4's tree; main was 70189a3)
Stage 4     OPEN / NOT STARTED — Python distribution removal, a separate PR
            after the observation period
P-037 A1    UNBLOCKED — not started; begins on a new branch from the
            Rust-default baseline (main at 63148d0)

The full terminal record, with the decision packet, the defects found and closed during qualification and the known differences carried forward, is the Stage 3 comment below (2026-09-18); the terminal merge candidate and its run are recorded in the second Stage 3 comment. The comments are the historical decision record; this body is the status surface.

CLI prerequisite satisfied — #261 / 261.B: the production Rust OwnIR executable landed via PR #347 (merged as 206e9c7) and #261 is closed completed; it was the sole CLI prerequisite of this cutover by owner decision C-5 (ratified 2026-09-08). #260 is reached. Not blocked by #345, the residual .own/dev CLI split out of #261.
Performance: DEFERRED BY OWNER at the Stage 3 decision (2026-09-18), not measured. #263 remains open and owes the baselines; the performance-gates section below is retained as #263's obligation, not as a Stage 3 predicate.
Also depends on: #256 and #259 (both reached)

Goal

Make an explicit, evidence-backed decision to switch Owen's default engine from Python to Rust, preserve a tested rollback path, and only later remove Python as a distribution dependency.

This is not an implementation catch-all. It is a cutover gate with measurable correctness, reliability, distribution and performance requirements.

Staged rollout

Stage 1: opt-in Rust — DONE

• Python remains default.
• Rust engine is explicitly selectable — by the launcher, never by the Rust executable itself (owner decision C-4 with P-022 step 7b: the production Rust OwnIR executable (own-cli ownir) — command, output and exit-code parity behind the existing launcher #261: the executable presents one engine and knows nothing of Python).
• Compare mode is available.
• Dogfood and selected CI jobs exercise Rust.

Stage 2: repository CI/dogfood Rust default — DONE

• Rust is default in Own.NET's own CI and internal dogfood.
• Python remains an explicit fallback/reference mode.
• All compare gates continue to run.

Stage 3: public Rust default — COMPLETE (decision f1d3681, run 35312190310; terminal merge candidate 1bf17e4, run 35322222621; PR #359 merged as 63148d0)

• Owen public package/Action uses Rust by default.
• A documented escape hatch can select Python during the observation window.
• No silent fallback from Rust failure to Python success.

Stage 4: remove Python distribution dependency — OPEN / NOT STARTED

Separate PR after an observation period:

• remove vendored Python core from the package;
• remove runtime Python discovery/download/cache requirements;
• deprecate/remove OWEN_PYTHON and legacy OWN_PYTHON according to a documented compatibility plan;
• retain Python reference code and oracle fixtures in the repository while they remain valuable for differential testing.

Launcher rulings (ratified 2026-09-08 with #261's decision packet)

• The Rust executable behind the launcher is own-cli ownir (P-022 step 7b: the production Rust OwnIR executable (own-cli ownir) — command, output and exit-code parity behind the existing launcher #261) — one engine. Engine selection, the staging above and the escape hatch are the launcher's, explicit and documented; no silent fallback from a Rust failure to a Python success at any stage.
• An unexpected Rust child exit code outside the legal set maps to the public internal-error path (exit 5), and the raw child status is retained in the evidence/report. A catchable panic in the child is one actionable stderr diagnostic and exit 70 (mapped onto the internal-error path as today); an uncatchable death is a visible hard failure, never findings and never a clean exit, with no particular OS exit number contracted.
• SIGINT/cancellation: the Windows and Linux reference behaviour is measured first; 130 is not invented as a universal contract. (Measured at Stage 3: Linux signal 2 with no exit-code contract; Windows 0xC000013A / STATUS_CONTROL_C_EXIT; the universal 130 explicitly not contracted.)
• stdin is not part of the production OwnIR contract.

Known differences recorded ahead of the packet (2026-09-08, from #261's 261.B measurement)

Named here so they enter the decision packet's Known differences: line as recorded facts rather than being discovered at cutover. Status at Stage 3 (2026-09-18): the invalid-UTF-8 tail, V1 and V2 were closed in the reference first, before the default moved (invalid UTF-8 → rc 2 on both engines at byte parity; V1 to the JSON door; V2's literal top-level -0 refused, with the residual narrowing CLI-B2); V3 verified not reopened; V4's reopen predicate verified not fired. Carried forward: WIN-ABC, CLI-B1, CLI-B2, V4, CANCEL-TRACEBACK (Linux-only) and ACTION-BUILD (a declared temporary distribution cost until the first published artifact). The paragraphs below are kept as the record of what was known before the packet.

• Windows console/pipe encoding — a deliberate behavior change at cutover, not a regression. On a piped Windows stdout the Python reference encodes with cp1252, translates line endings to CRLF, and dies with UnicodeEncodeError on non-ASCII output that the Linux reference renders. The Rust executable emits canonical UTF-8 on both platforms. So a Windows user's bytes change at cutover — cp1252 / CRLF (and the occasional UnicodeEncodeError) today, canonical UTF-8 tomorrow. Three claims, kept apart:
  • A — canonical reference parity: claimed. Rust reproduces the Linux/UTF-8 reference byte for byte.
  • B — Rust portability: claimed. Rust on Linux and Rust on Windows emit identical bytes.
  • C — native-Windows Python parity: NOT claimed. It does not exist today, because the Windows reference is not byte-portable; "better" here still means "different", so it is a behavior change, not folded into a parity claim.
• Invalid UTF-8 input — CLOSED before the cutover (2026-09-18, Python-first, byte parity). As recorded before the packet: the Python reference exited 70 on a UnicodeDecodeError (its load() caught OSError/JSONDecodeError only, so a decode failure escaped to the exit-70 catch-all), and a Python-first hygiene fix was owed before public cutover — UnicodeDecodeError → OwnIRError → rc 2. It landed in the reference first; both engines now report rc 2 with a message that states the offending byte and offset, so this is byte parity rather than a declared boundary.

Parser-domain rulings V1–V4 (owner-ratified 2026-09-08, surfaced by #261's Version census)

The ownir_version gate exposes classes where CPython's permissive json and serde_json decode the same source differently — V1–V3, acceptance / kind / sub-branch divergences, not message wording — plus one representation-only boundary, V4, where the two sides' Unicode tables are independently versioned. They are ruled here:

• V1 — non-finite constants NaN / Infinity / -Infinity: declared Python-reference defect. CPython json.loads accepts them, so the reference reaches the Version door (got nan / got inf / got -inf); serde_json rejects them at the JSON door (kind Json). The Rust JSON refusal is not a P-022 step 7b: the production Rust OwnIR executable (own-cli ownir) — command, output and exit-code parity behind the existing launcher #261 parity failure and the parser is not taught non-standard JSON. Python-first hygiene owed before public cutover: reject them at the JSON door → OwnIRError / rc 2 with a frozen message. Closed before the cutover (2026-09-18).
• V2 — literal -0: declared cross-parser encoding defect. CPython reads -0 as the integer 0, so {"ownir_version": -0} is accepted as v0 (measured); serde_json reads it as the float -0.0, so the Rust door refuses it wrong-type. This is the ambiguity P-022 step 7a: add dual-engine shadow mode and zero-diff reproduction artifacts #260 already froze a refusal for (tests/fixtures/repro/domain_negative_zero.facts.json; manifest reason "the literal -0: this reference reads it as the integer 0, serde_json as the float -0.0. Refused rather than reconciled"). Python's acceptance is not emulated in Rust for P-022 step 7b: the production Rust OwnIR executable (own-cli ownir) — command, output and exit-code parity behind the existing launcher #261 — an exact owner-declared exception to C-1, scoped to the top-level scalar ownir_version value the type check reads. A -0 inside a wrong-type container is not this exception and needs no emulation: {"ownir_version": [-0]} renders got [0] byte-identically on both sides — the reference's own repr reads the nested -0 as the integer 0 — and P-022 step 7b: the production Rust OwnIR executable (own-cli ownir) — command, output and exit-code parity behind the existing launcher #261's Version census pins that case as byte parity (negative_zero_below_the_top_level_matches_the_reference), not a declared difference. Python-first hygiene owed before public cutover: reject the literal top-level -0 at the OwnIR input boundary, consistent with P-022 step 7a: add dual-engine shadow mode and zero-diff reproduction artifacts #260's frozen domain. Closed before the cutover (2026-09-18): the literal top-level -0 is refused; the residual difference is CLI-B2, a narrowing of the ratified boundary from accept-versus-reject down to one token's spelling inside a refusal both sides make.
• V3 — integral version value beyond i64/u64: reproduced (byte parity). Such a value is an int to Python and takes the Version mismatch branch; serde_json's lossy f64 would otherwise send the port to wrong-type. P-022 step 7b: the production Rust OwnIR executable (own-cli ownir) — command, output and exit-code parity behind the existing launcher #261's repair-2 reproduced the mismatch branch narrowly — without enabling serde_json/arbitrary_precision — for oversized values of both signs, byte-identical to the reference in the Version census (Group 1, the byte denominator). Accepted at P-022 step 7b: the production Rust OwnIR executable (own-cli ownir) — command, output and exit-code parity behind the existing launcher #261's 261.B head (PR feat(own-cli): the production Rust OwnIR executable, behind the unchanged launcher (#261 261.B) #347, merged 2026-09-08 as 206e9c7): this is a completed parity fact, no longer an open divergence. Verified not reopened at both signs at Stage 3.
• V4 — str.isprintable() is answered from a version-dependent Unicode table: declared reference-version exception. Unlike V1–V3 this is not an accept/reject divergence and does not touch the parser: serde_json remains the sole parser deciding acceptance. It is a representation / diagnostic boundary — when a rejected ownir_version is a string, its repr escapes or prints each code point by str.isprintable(), which each side answers from the Unicode table it was built with. The supported CPython reference matrix is itself version-dependent — Python 3.11 links UCD 14.0.0, 3.12 links 15.0.0, 3.13 links 15.1.0 — so for a code point whose classification differs between those releases there is no single "what Python does". No single static Unicode-property table can be byte-identical to every supported CPython reference version on such code points; a pin would buy parity with one interpreter and silently lose it against another, which is a change to the P-022 reference contract rather than a technical fix — so V4 is not described as "unrepairable from Rust". V4 stays excluded from the byte-parity denominator but remains separately declared and measured. Its reach in the frozen fixture family is zero, evidenced first by the green Python-oracle matrix on 3.11/3.12/3.13 (three different Unicode tables, all green) and corroborated by a whole-plane sweep. Any exact mismatch count is a specific two-version measurement and must be labelled as such — e.g. the 15 097 code points measured for CPython 3.11.15 / UCD 14.0.0 against unicode-properties 0.1.4 / UCD 17.0.0 — never presented as a version-independent size of V4. Reopens if the supported Python matrix changes, the Rust Unicode snapshot changes, a divergent code point enters the frozen fixtures, or this difference reaches accept/reject/verdict behaviour or any previously byte-parity production output. The same is_printable helper is carried by own-syntax, so the boundary is a repository-wide tail to consolidate (one measurement, one place) rather than anything P-022 step 7b: the production Rust OwnIR executable (own-cli ownir) — command, output and exit-code parity behind the existing launcher #261 introduces. Reopen predicate verified not fired at Stage 3 (matrix still 3.11/3.12/3.13, snapshot still unicode-properties 0.1.4).

Correctness gates

• P-022 step 7a: add dual-engine shadow mode and zero-diff reproduction artifacts #260 fast and broad compare matrices report zero unexplained differences.
• Five pinned OSS repositories report zero unexplained differences.
• Full diagnostic/message/Evidence/SARIF parity is green (report was struck at P-022 step 5b: port the SARIF projection with canonical parity #256 and is not a surface here).
• The production OwnIR executable's command/output/exit-code parity (P-022 step 7b: the production Rust OwnIR executable (own-cli ownir) — command, output and exit-code parity behind the existing launcher #261) is green. The residual .own/dev CLI (P-022 step 7b residual: Rust CLI for the .own and dev surfaces — cfg, summaries, explain, .own check; emit behind #257 #345) is not a gate of this cutover (C-5).
• No severity or diagnostic-count drift is accepted without a separate Python-first decision.

Reliability gates

• malformed OwnIR and malformed source inputs do not panic;
• Rust crashes are visible and produce reproduction artifacts;
• cancellation/interruption behavior is tested against the measured reference (the SIGINT ruling above);
• deterministic reruns produce identical normalized output;
• memory/resource limits for hostile or very large inputs are documented;
• Windows and Linux clean-machine paths are covered.

Distribution gates

• Rust binary is packaged for all supported platforms;
• public Owen.Cli install and Action use work without undeclared runtimes;
• package upgrade/uninstall/reinstall tested;
• rollback engine selection is documented and tested during the observation stage;
• release workflow tests the actual packed artifact, not a project build.

Performance gates — DEFERRED BY OWNER at Stage 3; #263 open

Establish and publish baselines for:

• process startup;
• OwnIR parse;
• bridge/lowering;
• analysis;
• CLI/SARIF rendering;
• end-to-end C# project/solution run;
• peak RSS/allocations where practical.

#263 produces these baselines. At the Stage 3 decision (2026-09-18) the owner deferred them: the packet's three performance fields read exactly DEFERRED BY OWNER — NOT MEASURED and are kept rather than deleted, because a packet missing the line is a packet nobody can notice is missing it. Do not require a theatrical multiplier. Rust must meet an explicit budget and must not materially regress the user-visible path. Profile before optimizing; JSON and rendering may dominate once analysis becomes cheap.

Cutover decision packet

The PR/ADR must include:

Observation window:
Fast compare result:
Five-repo compare result:
Large-solution result:
Windows packaging result:
Linux packaging result:
Startup delta:
End-to-end delta:
Peak memory delta:
Known differences:
Rollback command/config:
Python-removal timing:

The Known differences: line begins from the section recorded above (the Windows A/B/C behavior change, the invalid-UTF-8 tail, and the parser-domain rulings V1–V4), plus anything the observation window adds. Delivered at Stage 3: generated from a measurement ledger, never typed — docs/generated/p022-stage3-packet.md from docs/evidence/p022-stage3-cutover.json, held by tests/test_stage3_packet.py to mutation-proved rules; zero OWED rows; the three performance fields deferred as above.

Rollback

• Rollback must be a documented engine selection or package patch, not a hidden automatic fallback.
• A Rust failure must remain observable.
• Immutable release tags are never moved.
• A broken release is corrected by a new patch release.

Guardrails

• No new diagnostics.
• No semantic cleanup mixed into cutover.
• No removal of Python reference/oracle tests merely because distribution no longer needs Python.
• No public Rust default before the evidence packet is approved.
• No Python dependency removal in the same PR that first changes the default.
• No engine selection inside the Rust executable, and no fallback anywhere (C-4).

Acceptance

Stage 3 — met on the decision candidate f1d3681 (run 35312190310, 31/31), re-confirmed on the implementation candidate b225436 (run 35320877147, 31/31) and on the terminal merge candidate 1bf17e4 (run 35322222621, 31/31)

• Explicit owner-approved cutover decision exists — met (2026-09-18, recorded in the Stage 3 comment).
• Rust is public default only after all gates pass — met for the correctness, reliability and distribution gates on the terminal run; the performance gates are deferred by owner decision with IDE foundation: establish cold/warm/incremental latency and memory baselines #263 open, not passed.
• Rollback path is tested and documented — met (explicit Python selection; no silent fallback).
• Observation-period results are recorded — met in the generated decision packet, zero OWED rows.
• Public Owen install/Action behavior remains correct on supported platforms — met on Linux and Windows; ACTION-BUILD accepted as a declared temporary distribution cost until the first published own-cli artifact.
• Merge of the terminal merge candidate 1bf17e4 into main — done (PR P-022 Stage 3: Rust is the public default engine (cutover, rollback gate, decision packet) #359, merged 2026-09-18 as 63148d0, a merge commit whose tree is 1bf17e4's tree).

Stage 4 — open

• Python distribution removal occurs in a later, separately reviewable PR, after the observation period.
• The Python reference code and oracle fixtures are retained while they remain valuable for differential testing.