Skip to content

ci: retain sanitized failure context before pytest interruption - #6355

Open
loopx-agent wants to merge 1 commit into
mainfrom
codex/ci-failure-locations-20261013
Open

loopx-agent wants to merge 1 commit into
mainfrom
codex/ci-failure-locations-20261013

Conversation

@loopx-agent

Copy link
Copy Markdown
Collaborator

Interrupted Python shards retain failed node names and phase timings, but can lose the failed source location when cancellation prevents the final traceback/JUnit. This leaves admission, settlement and process-retirement failures indistinguishable. A bounded real-pytest fixture on main b0b2acffb8fea7d85b20dd4f0139c1ce0616e5a7 reproduced the gap: six failed reports produced no immediate failure-context records.

Extend the existing #6318 TimingObserver with separate LOOPX_PYTEST_FAILURE JSON lines, flushed when the controller receives a failed report. Reuse its emitter and hashed identity. Keep ordinary phase durations and subtest accounting unchanged; nested failure locations are separate diagnostics, not additional timing phases. No workflow, dependency, timing reader, test selection, timeout, assertion or required gate is changed.

The small Python CI helper retains at most eight repository-relative source positions, a known exception class and literal-redacted assertion expressions. It omits messages, parameter values, locals and captured IO from the new JSON. Source reads/cache/output fields are bounded. Python 3.14 interpolation source text is cleared before rendering; known exception identities and public names are frozen; dynamically constructed names, source symlink escapes and malformed metadata cannot inject payload text. Optional metadata/context/output errors preserve the original failed pytest report, complete traversal and JUnit. The existing timing reader ignores the separate prefix.

Validation on the final source (Python 3.14.8 / Node 24.21.0, synthetic fixtures, one outer pytest process): 76 passed across the new failure-context suite, all existing timing observer/reader cases, real four-way collection/coverage union, early failed-node reporting, and required-aggregate rejection. This covers serial/two-worker q/v, setup/call/teardown/subtest outcomes, default-off JUnit parity, hard interruption before final JUnit, four diagnostic fault sites, type-name spoofing/rename, template literals, bounded JSON and path escape. Full CI Ruff inputs plus both CI scripts, diff hygiene and canonical/public-outbound scans of all four paths passed. Python 3.11 source syntax passed; its local interpreter lacks pytest, so full 3.11 execution is not claimed and awaits hosted CI.

The initial failure-context oracle failed on unchanged main as expected. During review, a new rename-negative fixture initially failed because pytest monkeypatch tried to delete the type's __name__ slot during restoration; explicit try/finally restoration fixed the fixture while retaining the original privacy assertion. The final 76-case batch passed. Independent internal read-only review found no unresolved P1/P2; it did not execute tests or grant formal approval.

Limits: this is diagnostic context, not authenticated log evidence or complete cause attribution. Existing pytest progress/summary output retains its original behavior. Unknown source/classes or diagnostic failure remain missing evidence; collection failures, worker death before report delivery and unseen reports at cutoff are not covered. This does not fix or explain the earlier descendants failure, shorten the full suite or qualify full hosted CI, release capacity or installed adoption. The existing CI observation boundary remains the owning contract. The related refactor pass reused that observer rather than adding a second plugin or modifying the reader. Frontend/Lark entry points are unaffected by this internal CI adapter.

中文:复用已合入的 timing observer,在 controller 收到失败报告后立即保留脱敏异常类、源码位置和断言结构;子测试保留失败位置而不重复计时。四文件范围不改 workflow、原断言、完整测试集、超时或退出失败语义。最终有界 76 PASS,完整 Python 3.11/hosted CI 与旧 descendants 原因仍未验证;不以诊断记录、同节点基线失败或本地通过豁免门禁。

Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant