Skip to content

perf: rerun a per-table hook over known paths on the calling thread - #1516

Merged
thekevinbot merged 4 commits into
mainfrom
bot/1514-watch-rerun-inline-1015
Oct 16, 2026
Merged

thekevinbot merged 4 commits into
mainfrom
bot/1514-watch-rerun-inline-1015

Conversation

@thekevinbot

@thekevinbot thekevinbot commented Oct 15, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #1514

A per-table hook rerun over paths that are already known and fit one invocation runs on the calling thread again, without counting CPUs. Since the streaming dispatcher (#1502), every watch rerun spawned a dispatcher thread and a worker thread and read the cgroup CPU files before spawning the hook, which put the Python e2e watch_speed_test over its 1.1x bar. Paths still arriving from a walk keep the streaming pool.

Paired local measurement, 600 alternating samples vs inotifywait (median per-pair ratio, loaded host):

build median ratio
main 654c4f8 1.161
before #1502 (aa0f3aa) 0.961, 1.028
this PR 1.018, 0.949

watch_speed_test on this PR, five runs: medians 0.889, 0.976, 1.062, 1.042, 0.935, all passing.

Changelog / Migrations

  • Changelog fragment added under <root>/<pkg>/changelog.d/ for each changed package (or: skip-changelog trailer on a commit with reason)
  • Migration fragment with the same slug added under <root>/<pkg>/migrations.d/ for each changed package (every subsection but Summary _None._ when nothing breaks)

E2E Verification

  • Ran e2e suites locally for every affected SDK
  • Python SDK e2e: pass (impacted on-file and watch tests)
  • TypeScript SDK e2e: pass
  • Rust core e2e (if applicable): N/A (cargo test --lib on_file:: pass)
  • packages/python/e2e-attestations/<branch>.json receipt written if packages/python changed (just e2e-attest-python)
  • packages/ts/e2e-attestations/<branch>.json receipt written if packages/ts changed (just e2e-attest-ts)
  • Command(s) run: testing-conventions e2e attest 'uv run python -m pytest tests/e2e/{watch_speed,on_file_query,fanout,declared_table_name,query_contract,server_speed,jsonl_parse_speed,frontmatter_parse_speed,use_case_notebook,use_case_transcripts,sigint_interrupt}_test.py -x -q'; just e2e-attest-ts
  • Result summary: Python 24 passed; TypeScript 18 passed.

@thekevinbot
thekevinbot merged commit b304387 into main Oct 16, 2026
89 checks passed
@thekevinbot
thekevinbot deleted the bot/1514-watch-rerun-inline-1015 branch October 16, 2026 11:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python e2e watch_speed_test over its 1.1x bar on main

1 participant