Skip to content

A refresh updates a keyed row in place - #1542

Merged
thekevinbot merged 4 commits into
mainfrom
bot/1540-update-in-place-1019
Oct 20, 2026
Merged

thekevinbot merged 4 commits into
mainfrom
bot/1540-update-in-place-1019

Conversation

@thekevinbot

@thekevinbot thekevinbot commented Oct 19, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #1540

A config table's watch refresh (reconcile_batch_rows) now compares and writes only the columns the on-file command emits, and matches rows by the table's key:

  • Key: the PRIMARY KEY, else the first unique, non-partial index (a UNIQUE constraint or CREATE UNIQUE INDEX, in index-name order) whose columns the new rows all fill non-NULL. Read from pragma_table_info.pk / pragma_index_list / pragma_index_info; expression indexes are skipped. A generated INTEGER PRIMARY KEY the command doesn't emit is NULL in every row, so it is passed over.
  • Keyed: a matched row that differs is UPDATEd by rowid (rowid and non-emitted columns kept) and emits update; vanished keys are deleted, new keys inserted (never paired into an update).
  • Unkeyed: the multiset compare as before, but over emitted columns only, so a table with a generated id no longer rewrites every row on each refresh.
  • An unchanged refresh writes nothing. Owner range is re-fitted as before.

UPSERT was not used: the old rows are needed for the events anyway, and a matched-row UPDATE by rowid is simpler than ON CONFLICT per candidate key.

Not in this PR (follow-up): per-file SDK hooks (handle_upsert) still delete and reinsert a file's rows; the --persist startup refill of a config table (stream_table_rows) still deletes and reinserts.

Docs: hooks.md (refresh semantics), config.md and howto/search-indexes.md (keyed tables need an AFTER UPDATE trigger).

E2E Verification

  • Ran e2e suites locally for every affected SDK
  • Python SDK e2e: pass
  • TypeScript SDK e2e: pass
  • Rust core e2e (if applicable): pass (cargo test --workspace --features cli)
  • packages/python/e2e-attestations/<branch>.json receipt written if packages/python changed (just e2e-attest-python)
  • packages/ts/e2e-attestations/<branch>.json receipt written if packages/ts changed (just e2e-attest-ts)
  • Command(s) run: testing-conventions e2e attest 'uv run python -m pytest tests/e2e/watch_speed_test.py tests/e2e/on_file_query_test.py tests/e2e/server_speed_test.py -q -p no:cacheprovider' (packages/python); testing-conventions e2e attest 'pnpm test:e2e' (packages/ts); cargo test --workspace --features cli
  • Result summary: all pass. Watch speed vs inotifywait, 5 reruns, final medians 0.86-1.06x. Manual: dirsql server on the experiments example; renaming a emits one update, ids unchanged; adding 0.json emits one insert with id 3; touching an unchanged file emits nothing.

Changelog / Migrations

  • Changelog fragment added under <root>/<pkg>/changelog.d/ for each changed package (or: skip-changelog trailer on a commit with reason)
  • Migration fragment with the same slug added under <root>/<pkg>/migrations.d/ for each changed package (every subsection but Summary _None._ when nothing breaks)

@thekevinbot
thekevinbot merged commit 43649b8 into main Oct 20, 2026
90 checks passed
@thekevinbot
thekevinbot deleted the bot/1540-update-in-place-1019 branch October 20, 2026 11:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A refresh updates a changed row in place instead of replacing it

1 participant