Repository navigation
fix: v1.6.1 I/O safety, lifecycle recovery and extraction recall - #279
Merged
Merged
Conversation
added 18 commits
October 1, 2026 09:11
The default build prune hid Java, Kotlin and Scala packages named build
(src/<sourceSet>/{java,kotlin,scala}/**/build). Exempt exactly those
directories in the scan and the watcher's structural tier; module build
output and the other defaults inside them stay pruned (upstream #1642).
The scan matched root .gitignore lines literally: anchored rules, globs and ** never matched, and dir rules stopped at the root. The watcher read the same file differently, matching every dir rule at any depth. Both now share one matcher built on the ignore crate: a leading or inner / anchors a rule, a slash-less rule applies at any depth, * and ** glob, and nothing is re-included below an ignored directory. A JVM package named build therefore stays indexed and watched under Android's /build and Spring Initializr's build/ plus !**/src/main/**/build/, while a bare build/ hides it (upstream #1642).
Resolution and the framework pass read files by path with no guard, so an import that followed package metadata into a dependency archive decoded it whole, and a FIFO blocked the pass. Stat first: only a regular file within the project's indexing.max_file_size is read, the read itself is bounded, and the rejection is cached like any read. The resolver hands the configured limit to every context it builds, so index and sync read the same files (upstream #1553).
Every indexing pass read a file in full before checking its size and decoded it strictly, so one oversize blob was read whole and one byte that is not UTF-8 failed the entire index, leaving it unreadable. One shared reader now serves index, sync, migration, the MCP freshness probe and resolution: it stats before reading and never holds more than the limit plus one byte; a file over the limit is recorded by a size stamp in place of its content hash, so a same-size rewrite is no change and crossing the limit is; an MPEG transport stream named .ts is video, neither indexed nor counted, and a tracked file that becomes one leaves the index; other bytes decode with U+FFFD as upstream does. Extraction version 15 -> 16 (upstream #1910).
A close tag with whitespace before its bracket (</select >) let a statement run on and swallow the next one (upstream #1222). The rest of upstream's #1182 forms were also missing: commented-out statements and includes are now ignored (CDATA stays data), attribute values may use either quote style, a namespace-less <sqlMap> qualifies statements by their Map.statement ids, iBatis maps add <statement> and <procedure>, and databaseId joins the signature. A vendor-split pair written on one line keeps both nodes through the #1349 id allocator. KEEP-RUST: a self-closing <sqlMap .../> reference is never a statement-map root.
A {% liquid %} body holds tags without braces of their own, so render,
include, section and assign written there were never extracted. Scan
whole braced tags one at a time and the bare lines of each liquid body,
anchored at the line start, so tag-like strings and inline comments are
not read twice; {% comment %} and {% raw %} regions are skipped in both
spellings (upstream #1906).
The orphan sweep's resolver statistics were dropped, so a sync that healed an interrupted index on unchanged files read as a no-op. Carry the attempted, resolved and unresolved counts in SyncOutcome, print them after the sync line, and add them to the diagnostics payload (upstream #1360).
MCP root discovery counted any .codegraph/codegraph.db as an index, so a stray, empty or schema-less file in an ancestor captured the upward walk for every project below it and the sub-project scan never ran. Require the published namespace state as well, the authority the CLI's ancestor walk already uses; it reads only the state slots and never opens SQLite (upstream #1895). Test fixtures now publish their namespace.
…I/O error When Windows-native and WSL CodeGraph open the same .codegraph index under /mnt/<drive>/, SQLite's locking and -shm memory do not hold across the 9p/DrvFs bridge and WSL fails with a bare disk I/O error. With CODEGRAPH_DIR unset on a WSL Windows drive, the index root is now chosen per project: an existing .codegraph-wsl/ wins, an existing .codegraph/codegraph.db is kept, and a fresh index goes in .codegraph-wsl/, which init reports. A SQLite I/O error on a shared default index carries the CODEGRAPH_DIR instruction in the CLI and in MCP open failures. Every other host is unchanged (upstream #995).
A transient Windows sharing violation (an antivirus scan or indexer holding daemon.pid open) aborted the lock publish, and the post-bind socket rewrite truncated the lock in place, so a reader could see a partial record. Both now rename a complete temp record over the lock, re-check that the lock still holds this daemon's record before every attempt, and retry only Windows sharing errors with a 25 ms to 400 ms backoff over about 1.6 s (upstream #2053, #1773).
Under Git Bash, MSYS2 or Cygwin, install.sh stopped at an unsupported OS. Recognise those shells, print the exact PowerShell command and exit non-zero before downloading anything; Linux and Darwin are unchanged (upstream #1294).
Every target's detect() reads its agent's config, and install --refresh detects every target to skip the ones codegraph was never installed into, so an unparseable config of an unmanaged agent got a .backup from the read alone. Reading is now pure; the backup is made where a config is replaced (write_json_file) and before a write path skips one it cannot parse (upstream #1870).
The event loop took the pending paths and the owed full sync before an attempt and dropped them when it failed, so a directory removal whose full sync failed never reconciled and its files kept their nodes. Keep both and retry after the debounce doubled per consecutive failure, capped at 30 s (upstream #1964). An index-lease timeout behind a long foreground index is now lock contention, backed off in place, instead of a sync failure counted toward disabling auto-sync.
…alth Lock contention past the budget stopped the watcher for good, so after one long foreground index the daemon answered from a frozen index with no notice. The watcher now pauses as RECOVERING instead: it keeps collecting changes and retries a full reconcile every 30 s until one commits. Persistent sync failures and exhausted watch resources still stop it as DISABLED. MCP answers carry a RECOVERING or DISABLED whole-index banner, status reports the state, and while auto-sync is not healthy search, callers, callees and impact name any indexed file that changed since its last sync instead of answering from the frozen graph (upstream #876, #1959).
Portuguese/Spanish como and German wie are everyday words, so treating them as structural keywords fired the HIGH tier on ordinary prompts. Drop them from the unconditional list; another structural keyword, a verified code token or indexed prose segments still decide the tier (upstream #1654).
…tion
The JS dynamic-import form skipped any argument starting with a quote
or backtick, so import(`./locales/${lang}.js`) and import('./x/' + y)
read as literal imports and explore never noted the runtime boundary.
A call is now dropped only when its argument is exactly one complete
string literal, with escape-aware template scanning (upstream #1967).
Import resolution probes every extension of a specifier from every file that imports it, and each probe queried the store and stat'd the disk. Memoize the answer for the pass in both contexts, bounded at 200,000 entries as upstream's fileExistsMemo is; neither the borrowed store nor the tree changes within a pass.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1fe6c5cd94
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
+89
to
+90
| /// Every occurrence of `tag`, in both spellings Liquid allows (upstream | ||
| /// #1906). Inside a `{% liquid %}` tag each body line is a tag without |
There was a problem hiding this comment.
Document these shipped extraction changes in docs/embedded-extraction.md: it still says Liquid only recognizes braced regex forms and MyBatis only recognizes namespaced <mapper> roots with five statement types, omitting {% liquid %} blocks, suppressed raw/comment regions, namespace-less iBatis maps, and statement/procedure. Users consulting the canonical reference therefore get behavior that directly contradicts this implementation, and the repository requires extraction-rule changes to update their canonical documentation alongside the code.
AGENTS.md reference: AGENTS.md:L96-L99
Useful? React with 👍 / 👎.
This was referenced Oct 1, 2026
Merged
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fifth PR of the v1.6.1 alignment, following #278: the I/O-safety, lifecycle-recovery and remaining extraction-recall rows. Dispositions and landing state are tracked in
docs/upstream-sync/V1_6_1_AUDIT_2026-09-30.md; tracked parity inUPSTREAM.mdstaysv1.6.0until the release gate.Reading files — extraction version 15 → 16
indexing.max_file_sizeis recorded by a size stamp instead of a content hash, so a same-size rewrite is not a change and crossing the limit is. An MPEG transport stream named.tsis video: not indexed, not counted, and a tracked file that becomes one leaves the index.init/indexand left the index unreadable. Bytes that are not UTF-8 now decode with U+FFFD, as upstream'stoString('utf8')does.file_existsanswer is memoized for the pass in both contexts.Scope
build(#1642):src/<sourceSet>/{java,kotlin,scala}/**/buildstays indexed and watched; module output and the other defaults inside it stay pruned..gitignorenow follows git's rules in scan and watch. The scan matched its lines literally (anchored rules, globs and**never matched; dir rules stopped at the root) while the watcher matched every dir rule at any depth. One shared matcher on theignorecrate replaces both, so Android's/buildand Spring Initializr'sbuild/+!**/src/main/**/build/keep a JVMbuildpackage, a barebuild/hides it as in git, and the watcher handles exactly what the scan indexes. Behavior change: projects whose root.gitignoreuses globs, anchored paths or nested-matching directory rules now index fewer files — the ones git ignores.Extraction recall
<sqlMap>qualifies statements by theirMap.statementids; iBatis adds<statement>/<procedure>;databaseIdjoins the signature; a vendor-split pair on one line keeps both nodes through the #1349 allocator. KEEP-RUST: a self-closing<sqlMap resource="…"/>is never a statement-map root (upstream would read the config file's tail as one).{% liquid %}blocks are extracted;{% comment %}/{% raw %}regions are skipped in both spellings.import(\./x/${y}`)andimport('./x/' + y)` are runtime imports.Lifecycle
index) no longer stops the watcher for good. It pauses as RECOVERING, keeps collecting changes and retries a full reconcile every 30 s until one commits. MCP answers carry a RECOVERING or DISABLED banner, status reportsAuto-sync:, and while auto-sync is not healthysearch/callers/callees/impactname any drifted file instead of answering from the frozen graph.Resolved N pending references (M unresolved).codegraph.dbwithout a published namespace no longer captures the upward walk; the CLI already used the same state authority./mnt/<drive>withCODEGRAPH_DIRunset, a fresh index goes in.codegraph-wsl/(an existing.codegraph/codegraph.dbis kept); a SQLite I/O error on a shared default index explains the fix.daemon.pidpublishes rename a complete temp record, re-check ownership, and retry sharing violations for up to ~1.6 s; the socket rewrite no longer truncates the lock in place.install.shunder Git Bash/MSYS2/Cygwin points at the PowerShell installer (#1294);detect()no longer leaves a.backupbeside a config it cannot parse (#1870).como/wiealone are no longer structural keywords.Golden impact
All 19 re-indexable corpora regenerated with this build are byte-identical. MCP structural fixtures are unchanged.
Verification
-D warnings,cargo fmt --check,scripts/guardrail.sh,docs-checkandoxfmt --checkclean; the pre-push gate re-runs them on the pushed head..gitignorerule and parity cases,resolution_file_reads,source_reads,mybatis_mappers,liquid_blocks,sync_recovery_report, the MCP root, auto-sync and stamp-freshness cases, the WSL default, the daemon lock replace, the watcher retry and recovery cases, the prompt-hook and dynamic-import cases, and the Windows-shell installer harness.multi_segment_dir_rule_matches_on_boundariesexpected.gitignorea/b/to prunex/a/b; an inner/anchors a rule in git, in upstream and in the old scan, so it now expectsx/a/bto be watched. MCP root-discovery test fixtures now publish their namespace state, which is what makes a directory an index.BEGIN_COMMIT_OVERRIDE
fix(scan): keep JVM packages named build under source roots
fix(scan): read the root .gitignore with git's rules in scan and watch
fix(resolve): read no file extraction would skip
fix(extract): bound every source read and skip MPEG-TS clips named .ts
fix(extract): read MyBatis and iBatis mappers as upstream does
fix(extract): read Liquid tags inside {% liquid %} blocks
fix(sync): report the pending references a healing sync resolved
fix(mcp): a database file without a published index is not a root
fix(wsl): keep WSL off Windows' index on /mnt drives and explain the I/O error
fix(daemon): retry a Windows sharing violation when replacing daemon.pid
fix(install): point Windows shell users at the PowerShell installer
fix(installer): keep detection read-only for configs it cannot parse
fix(watch): keep a failed sync's batch and owed full reconcile
fix(watch): recover auto-sync after lock contention and report its health
fix(prompt-hook): require corroboration for everyday como and wie
fix(explore): flag dynamic imports built from a template or concatenation
perf(resolve): answer each file-existence probe once per pass
END_COMMIT_OVERRIDE
🤖 Generated with Claude Code