Skip to content

test(python): make notebook speed-of-light xfail non-strict - #1222

Closed
thekevinbot wants to merge 2 commits into
mainfrom
claude/1221-notebook-xfail-nonstrict
Closed

thekevinbot wants to merge 2 commits into
mainfrom
claude/1221-notebook-xfail-nonstrict

Conversation

@thekevinbot

Copy link
Copy Markdown
Collaborator

Sets strict=False on the xfail marker of use_case_notebook_test.py::describe_notebook_speed_of_light::it_matches_native_rows_within_the_bar. The test hovers around its 1.1x bar, so with strict=True a fast run counts as XPASS(strict) and fails just e2e-attest-python for unrelated PRs. The reason string is unchanged, and the other speed-of-light tests are untouched.

Fixes #1221
Related: #1202

Changelog / Migrations

  • Changelog fragment added under <root>/<pkg>/changelog.d/ for each changed package (or: skip-changelog trailer on a commit with reason): test-only, so the file is exempt; a skip-changelog: line is also on the commit
  • Migration fragment added under <root>/<pkg>/migrations.d/ (or: not required -- additive/bugfix only): not required

E2E Verification

  • Ran e2e suites locally for every affected SDK
  • Python SDK e2e: pass
  • TypeScript SDK e2e: N/A
  • Rust core e2e (if applicable): N/A
  • packages/python/e2e-attestations/<branch>.json receipt written if packages/python changed (just e2e-attest-python)
  • packages/ts/e2e-attestations/<branch>.json receipt written if packages/ts changed (just e2e-attest-ts): N/A
  • Command(s) run: cargo build -p dirsql --features cli, just e2e-attest-python
  • Result summary: 38 passed, 3 xfailed, 1 xpassed. The xpass is now tolerated rather than failing the run.

The test straddles its 1.1x bar locally, so a fast run XPASSes and strict xfail fails e2e attestation for unrelated PRs.

skip-changelog: test-only marker change
@thekevinbot

Copy link
Copy Markdown
Collaborator Author

Closing: #1212 deletes this marker outright (the test passes at parity there), and the decision on the five speed-of-light tests is strict xfail. #1218's python receipt is written after #1212 merges.

@thekevinbot thekevinbot closed this Oct 2, 2026
@thekevinbot
thekevinbot deleted the claude/1221-notebook-xfail-nonstrict branch October 2, 2026 15:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Notebook speed-of-light e2e is flaky as strict xfail; make it non-strict

1 participant