Repository navigation
sqlitedb: announce and time the one-time search index build - #99
Merged
Merged
Conversation
…DID-matlab f35bbb4) Opening a database without doc_data_doc_idx / doc_data_field_value builds them, silently - and for a large dataset on a slow or synced drive that takes minutes, so the open appears to hang. _ensure_indexes now prints to stderr, before each index it builds, its name, the database, the approximate number of rows (max(rowid)) and that this happens once and can take minutes; afterwards the time it took; and a build that failed. A database that already has the indexes opens without printing anything. Test: test_indexes_added_when_older_database_opened checks the messages on the first open and silence on the next. Bridge: sqlitedb -> f35bbb4. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Dv7SxVR8MCu3MKpUvwwUia
…hing to port) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Dv7SxVR8MCu3MKpUvwwUia
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #98. This ports VH-Lab/DID-matlab's
f35bbb4..5b7220b, on the DID-matlab follow-up PR to #229.Problem
When an older database is opened,
_ensure_indexesbuildsdoc_data_doc_idxanddoc_data_field_value, and it does so silently.For a large dataset on a slow or synced drive, that build takes minutes, and the open looks like it has hung. This was reported in DID-matlab on a real 100,000-document dataset.
Change
_ensure_indexesnow prints to stderr:max(rowid), which costs nothing), and that this happens once and can take minutes.The text matches DID-matlab's. A database that already has the indexes opens without printing anything.
Tests
test_indexes_added_when_older_database_openednow checks two things:Local results:
black --checkandruff checkare clean.pytest: 817 passed.bin/check_bridge_coverage.py(all six checks, against the DID-matlab branch): 0 problems.Bridge
The
sqlitedbentry now points at5b7220b(the newest commit on the DID-matlab PR branch); itsdecision_logrecords the port. That commit only reaches DID-matlabmainwhen the DID-matlab PR is merged, so merge that PR first, with a merge commit.Generated by Claude Code