diff --git a/docs/upstream-sync/UPSTREAM.md b/docs/upstream-sync/UPSTREAM.md index 18f6738a..7c8409d4 100644 --- a/docs/upstream-sync/UPSTREAM.md +++ b/docs/upstream-sync/UPSTREAM.md @@ -13,14 +13,18 @@ `1.2.0 → 1.4.1` subset landed across `v0.28.3`–`v0.39.0`; see the 2026-07-17 CLOSEOUT entry) - **This project version:** see `Cargo.toml` (`codegraph-rs`, independent line) -- **Last sync:** `2026-08-20` — the post-`v1.5.0` **unreleased** range (104 commits - on upstream `main` past the `v1.5.0` tag) was triaged and six tiers shipped - across `v0.43.2` → `v0.46.1`; see the 2026-08-20 entry. Tracked parity stays - `1.5.0` — none of these releases is a port of an upstream *release* range, and - upstream's newest tag is still `v1.5.0` -- **Next sync starts here:** `git ls-remote --tags` alone is not enough. Diff - `v1.5.0..origin/main` too — this round found 104 unreleased commits behind an - unchanged tag +- **Last audit:** `2026-08-27` (refreshed `2026-08-31`) — upstream `v1.6.0` + plus its one following + docs-only commit was audited through + `6a056ec5db35172f9dc348f87b54ea415aa5169e`. The exact new range after the + previously closed 104-commit batch is 28 commits and leaves 15 PORT families; + see `V1_6_TRIAGE_2026-08-27.md`. Tracked parity therefore remains `1.5.0` + until that backlog lands +- **Previous exact boundary:** `c6aaa20358cd6adcd04b87bdef8e5803ad146f3a` + (the 104th commit after `v1.5.0`; merge of upstream PR #1516) +- **Next discovery starts here:** diff + `6a056ec5db35172f9dc348f87b54ea415aa5169e..origin/main`; do not re-triage + the v1.6 backlog — use `V1_6_TRIAGE_2026-08-27.md` - **Upstream repo:** https://github.com/colbymchenry/codegraph > **1.4.1 → 1.5.0 sync: COMPLETE** (codegraph-rs `v0.42.1` → `v0.42.4`). Every @@ -73,6 +77,55 @@ ## Sync log +### 2026-08-31 — implementation baseline refreshed at codegraph-rs `v0.48.1` + +Before implementation began, both repositories and the three load-bearing open +issues were refreshed against their live GitHub state: + +- codegraph-rs `main` and the latest published release are both `v0.48.1` + (`e5350b3c6eb6312cfae86f2353375e4bfb5e9a3f`); +- upstream's latest release remains `v1.6.0`, and upstream `main` is still + exactly `6a056ec5db35172f9dc348f87b54ea415aa5169e`; +- the diff after the audit boundary is empty, so the 15 PORT families below do + not need re-triage; +- #1626 (Python aliases), #1616 (Lua function expressions), and #1617 (custom + Lua loaders) remain open. The first two stay in the implementation backlog; + #1617 stays explicitly deferred. + +This refresh changes only the Rust implementation/release baseline. It does not +rewrite the 2026-08-27 audit evidence, and tracked upstream parity remains +`1.5.0` until the portable `v1.6.0` backlog lands. + +### 2026-08-27 — `v1.6.0` release range AUDITED; 15 PORT families OPEN (tracked parity stays `1.5.0`) + +Upstream released `v1.6.0` on 2026-08-26. The earlier post-`v1.5.0` audit +covered exactly 104 commits through `c6aaa20358cd6adcd04b87bdef8e5803ad146f3a`; +that count was re-verified before this round. The fresh range through current +upstream head `6a056ec5db35172f9dc348f87b54ea415aa5169e` is exactly 28 commits +(27 through the release tag plus one docs-only follow-up), 100 files, ++6900/−575. + +The complete classification, local source evidence, open-issue triage, and +recommended implementation waves are recorded in +[`V1_6_TRIAGE_2026-08-27.md`](V1_6_TRIAGE_2026-08-27.md). + +The highest-priority item is native tree-walker depth protection: upstream +demonstrated a process-level stack-overflow crash on deeply nested source, and +the Rust walker still recurses without a guard. The graph-correctness wave then +contains Erlang arity identity, two Rust method-resolution fixes, JS/TS +object-literal namespaces, inherited TS aliases, C++ `.h` detection, Python +import aliases, Lua function-expression callables, and live watcher-scope +refresh. Retrieval/UX work follows. + +One measured performance regression is also now pinned: the Rust database +already has `idx_nodes_lower_name`, but its case-insensitive exact-name query +uses `COLLATE NOCASE` and scans `nodes`; `lower(name) = lower(?)` seeks the +expression index. That upstream change is a real PORT item, not an +already-present optimization. + +No source port was implemented in this audit. The next upstream discovery diff +starts after `6a056ec5`; implementation work starts from the linked report. + ### 2026-08-20 — post-`v1.5.0` unreleased range TRIAGED and CLOSED, six tiers SHIPPED (`v0.43.2` → `v0.46.1`; tracked parity stays `1.5.0`) `git ls-remote --tags` still reports `v1.5.0` as upstream's newest tag, so a diff --git a/docs/upstream-sync/V1_6_TRIAGE_2026-08-27.md b/docs/upstream-sync/V1_6_TRIAGE_2026-08-27.md new file mode 100644 index 00000000..906b7302 --- /dev/null +++ b/docs/upstream-sync/V1_6_TRIAGE_2026-08-27.md @@ -0,0 +1,179 @@ +# Upstream v1.6.0 triage — 2026-08-27 + +> **2026-08-31 refresh:** implementation now starts from codegraph-rs +> `v0.48.1` (`e5350b3`). Upstream `main` still equals the audited boundary +> `6a056ec5`, with no intervening commits. Issues #1626, #1616, and #1617 remain +> open with their dispositions below unchanged. This note updates the +> implementation baseline only; the measured 2026-08-27 audit evidence remains +> historical and unchanged. + +## Outcome + +Upstream `colbymchenry/codegraph` released `v1.6.0` on 2026-08-26. +The previous Rust-port audit ended at upstream commit +`c6aaa20358cd6adcd04b87bdef8e5803ad146f3a` (the 104th commit after +`v1.5.0`). The exact new range audited here is: + +```text +c6aaa20358cd6adcd04b87bdef8e5803ad146f3a + ..6a056ec5db35172f9dc348f87b54ea415aa5169e +``` + +That range contains: + +- 28 commits total; +- 27 commits through the `v1.6.0` release commit + `dfccdf62547fcd76d343344d823a0e1998d3a89f`; +- one docs-only commit after the tag; +- 100 changed files, +6900/−575; +- 20 behavioral commit families after changelog/release-only commits are + removed. + +Disposition of those 20 behavioral families: + +- **PORT: 15** +- **ALREADY-HAVE / N/A mixed bundle: 1** +- **N/A: 3** +- **DEFER: 1** + +Tracked parity remains `1.5.0` until the PORT backlog below lands. The next +upstream discovery audit starts after `6a056ec5`; implementation work should use +this report rather than re-triaging the same commits. + +Official upstream release: + + +## Important release-note boundary + +The `v1.6.0` release notes aggregate the whole post-`v1.5.0` development +period, not only the 28 commits in this audit. The preceding 104-commit range +was already triaged and closed in this repository across `v0.43.2`–`v0.46.1`. +That earlier work includes Copilot installer support, generated-file +classification, most Explore budgeting/selection fixes, unions, Rust unit +structs, Tauri IPC, and source freshness checks. They are not reopened here. + +## Commit-family classification + +| upstream | change | Rust evidence | disposition | +| -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `238dbc5`, `ccb0295` | Pin exact file paths and extensionless kebab-case basenames in Explore; supplement plain words with camelCase identifier segments; allow variable/constant seeds | `find_relevant_context` still starts with one `search_nodes(query, limit=8)` result set. It has no indexed-path extraction/pinning or identifier-segment supplement. Variables/constants can only appear if ordinary search happens to rank them. | **PORT — P1.** Add deterministic query-path matching, explicit pinned roots/allocation, kebab basename matching, and segment-vocabulary seed supplementation. Preserve current freshness and whole-source contracts. | +| `26045b3`, `d8f2eea` | Repair upstream dual-kernel retry materialization, V8 spread-result failure, and stripped-source salvage visibility | The Rust port has one native extraction path, no JS↔native kernel result transport, no V8 argument spreading, and no stripped-comment retry mode. | **N/A.** Do not transplant the transport/retry implementation. | +| `81e1f4a` | Daemon identity, crash recovery, skipped-file persistence, parse timeout, dropped-index repair, and JS cache hardening bundle | The Rust daemon already uses owner-bound socket/control behavior and stale-socket cleanup rather than trusting a reused PID; rebuild state is fail-closed behind a permanent lease; V8/JS regex-cache items do not apply. The skipped-file marker and parse-worker timeout designs do not map directly to the current native extractor. | **ALREADY-HAVE / N/A.** Keep as a bundle-level non-port. Revisit only from a Rust-native failure measurement, especially if unchanged oversized/failed files are observed to churn. | +| `340d4b0` | Remove catastrophic backtracking from the Swift Vapor route regex | The Rust port has no Swift Vapor framework resolver, and Rust's `regex` engine does not implement catastrophic backtracking. | **N/A.** A future Swift resolver must be designed against Rust regex semantics rather than porting this patch. | +| `cc9ce09` | Use Git `-uall` so status sees source files inside a wholly new directory | Rust extraction/status scans the filesystem directly, and the watcher explicitly registers newly created directories. It does not depend on Git's collapsed untracked-directory output. | **N/A.** | +| `a740291` | Resolve extensionless ES imports to `.xsjs` / `.xsjslib` | Language detection already maps both suffixes to JavaScript, but `extension_resolution(Language::JavaScript)` contains only `.js/.jsx/.mjs/.cjs` and index forms. | **PORT — P2.** Add both suffixes to JS-family import candidate ordering with positive and same-name-collision negatives. | +| `9219967` | Make case-insensitive exact-name supplements seek the name index | The database already has `idx_nodes_lower_name ON nodes(lower(name))`, but `nodes_by_exact_name_nocase` uses `name = ? COLLATE NOCASE`. Measured with `EXPLAIN QUERY PLAN` on this repository: the current query does `SCAN nodes`; `lower(name) = lower(?)` does `SEARCH nodes USING INDEX idx_nodes_lower_name`. | **PORT — P1.** Change the query shape and pin the planner behavior with an `EXPLAIN QUERY PLAN` regression test while preserving result ordering/content. | +| `474f051` | Follow `tsconfig`/`jsconfig` `extends` chains, package configs, cycles, config-relative `baseUrl`, and `tsconfig.base.json` fallback | `load_project_aliases` checks only root `tsconfig.json` then `jsconfig.json`, parses one file, and reads only that file's `compilerOptions`. | **PORT — P1.** Implement deterministic recursive config loading with cycle guards and inherited-option origin tracking. | +| `bc89480` | Project-local Codex installer | `CodexTarget::supports_location` accepts only `Global`, writes only `~/.codex/config.toml` and `~/.codex/AGENTS.md`, and explicitly rejects local MCP installation. | **PORT — P2.** Write `./.codex/config.toml` plus the repository `AGENTS.md`; keep local skill support and MCP support as separate capabilities. | +| `1d9de88` | Ranking-only `deprioritize` path patterns | `IndexingConfig` has `ignore_dirs`, `ignore_paths`, `exclude`, and `include`, but no ranking-only path setting. Search/Explore scoring has no project-configured path penalty. | **PORT — P2.** Add TOML/JSON-compatible project configuration and deterministic search/Explore scoring. It must never remove nodes or change extraction/goldens. | +| `12f7a59` | Attribute methods in generic/lifetime/reference/qualified Rust `impl` blocks to the implementing type | No corresponding impl-type normalization or regression coverage is present in the Rust extractor. Current generic extraction still depends on the generic walker shape. | **PORT — P1, golden-sensitive.** Add Rust fixtures for generic, lifetime, reference, qualified, inherent, and trait impls; preserve node-id stability for unaffected methods. | +| `7963672` | Resolve `self.field.method()` through the field's declared type | No field-owner/type lookup path exists for Rust call resolution; source search found no equivalent implementation or tests. | **PORT — P1, golden-sensitive.** Preserve the receiver chain during extraction and resolve only statically safe wrappers/references. External, generic, and container field types must remain unresolved rather than guessed. | +| `278a8ed` | Resolve calls to methods inside exported object-literal namespace constants | The resolver has no object-literal containment/import path for JS-family methods. | **PORT — P1, golden-sensitive.** Cover defining-file and imported namespace calls, ambiguity, and same-name non-members. | +| `0d17dfd` | `install --init` and non-interactive `init --yes` | `Install` already has `--yes`, but no `--init`; `Init` has no `--yes`. | **PORT — P2.** Reuse one init implementation, retain broad-root/home safety refusals, and ensure install never indexes unless `--init` is explicit. | +| `838006c` | Guard deeply recursive native walkers against process stack overflow | `TreeSitterWalker::visit_node` and `visit_named_children` recurse directly with no depth/stack guard. There is no fallback parser in this port, so a direct copy is impossible, but the process-crash risk remains. | **PORT — P0 reliability.** Add a Rust-native bounded traversal failure that records one file error and continues the project. Do not catch process aborts or weaken deterministic output for ordinary files. | +| `cf1b0e3` | Reload watcher scope when project config or `.gitignore` changes, then fully reconcile | `ProjectWatcher::start` builds one immutable `WatchPolicy`; closures capture the initial include/exclude lists; the event loop never reloads them. | **PORT — P1.** Adapt the trigger to this port's resolved `.codegraph/config.toml`, extension config, and root `.gitignore`; atomically replace policy/watch targets and run one full sync. Incremental sync must re-check the current scope. | +| `d618d94` | Recognize a plain `struct Derived : Base` anywhere in `.h` as C++ | `looks_like_cpp` lacks the plain derived-struct form and examines only `prefix_8k(source)`. | **PORT — P1, golden-sensitive.** Use comment/string-masked full-source detection with C negatives; extend the C-family golden corpus. | +| `a5c2709` | From an unindexed launch root, adopt exactly one indexed child project and emit useful diagnostics | MCP `roots/list` can adopt the first indexed root supplied by a capable client, but `resolve_project_arg_with(None, ...)` only probes the configured default. There is no bounded launch-root down-scan for clients that do not provide roots. | **PORT — P2.** Add a deterministic bounded scan that adopts only exactly one candidate; zero/multiple candidates remain unselected and are listed in diagnostics. Keep explicit `projectPath` authoritative. | +| `c382225` | Register the documented `codegraph context` CLI command | The underlying deterministic context engine exists as `explore`, and the MCP surface already emits the same source/relationship pack. The upstream command adds a second public CLI contract (`--format`, `--max-nodes`, `--no-code`) rather than fixing graph correctness. | **DEFER.** Decide whether compatibility consumers such as Memorix justify an alias/new output contract. Do not fork a second context engine. | +| `41c1075` | Erlang arity-qualified function identity/resolution and binary-literal arity counting | `visit_erlang_fun_decl` merges adjacent clauses using only the function name, marks exports by name, and creates unqualified nodes. | **PORT — P1, golden-sensitive.** Use `module::fun/arity`-style internal qualified names while preserving the node-id formula; make specs, exports, calls, fun values, and behaviour dispatch arity-aware. | + +## Open issue triage after v1.6.0 + +| issue | local assessment | disposition | +| ---------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ | +| [#1626](https://github.com/colbymchenry/codegraph/issues/1626) Python aliased module imports | The Rust resolver has no demonstrated scope alias binding for `import x as y` / `from p import x as y` when resolving `y.member()`. | **PORT — P1, golden-sensitive.** Add both forms plus plain-import controls and ambiguity negatives. | +| [#1616](https://github.com/colbymchenry/codegraph/issues/1616) Lua function expressions in table fields/keys | `LuaSpec::function_types()` contains only `function_declaration`; function expressions assigned into tables cannot create enclosing callable nodes. | **PORT — P1, golden-sensitive.** Include `M.f = function`, nested table keys, and `local f = function` call targeting. | +| [#1617](https://github.com/colbymchenry/codegraph/issues/1617) custom Lua module-loader functions | This is project-specific configurable synthesis, not a regression in standard `require`. | **DEFER.** Design only with a real target project and a deterministic path/root contract. | +| [#1620](https://github.com/colbymchenry/codegraph/issues/1620) Explore dedup leaks across subagent/compaction contexts | This port does not implement upstream's connection-persistent source suppression. | **N/A.** Do not add that state without a host-supplied context identity/reset protocol. | +| [#1577](https://github.com/colbymchenry/codegraph/issues/1577) interrupted orphan sweep under-resolves | The described Node implementation's orphaned-ref sweep does not exist here. Rust rebuilds are fail-closed, and sync-vs-force byte equality is a hard gated invariant. | **ALREADY-HAVE by stronger invariant / architecture.** Add a crash-injection regression only if an equivalent partial-state path is found. | +| [#1553](https://github.com/colbymchenry/codegraph/issues/1553) daemon OOM and reused PID | V8 heap OOM is N/A. The Rust daemon already has socket/control ownership and stale socket cleanup rather than trusting PID existence alone. | **N/A / ALREADY-HAVE.** | +| [#1539](https://github.com/colbymchenry/codegraph/issues/1539) oversized WAL after forced kills | WAL healing, status visibility, and journal-size handling were already shipped in the Rust port. | **ALREADY-HAVE.** | +| [#1549](https://github.com/colbymchenry/codegraph/issues/1549) old-Git untracked behavior | The Rust scan path does not depend on that Git behavior. | **N/A.** | +| [#1618](https://github.com/colbymchenry/codegraph/issues/1618) GDScript support request | The Rust port has GDScript extraction plus a concrete Godot resolver and golden corpus. | **ALREADY-HAVE / AHEAD.** | + +## Recommended implementation waves + +### Wave 0 — process safety + +1. `838006c`: bounded tree traversal / deep-nesting failure containment. + +Required proof: + +- an adversarial deeply nested fixture does not abort the process; +- the bad file produces a visible extraction error; +- other files still index; +- ordinary fixture goldens remain byte-identical; +- sync and `index --force` remain byte-identical. + +### Wave 1 — graph correctness + +1. Erlang arity identity/resolution (`41c1075`). +2. Rust generic impl ownership (`12f7a59`). +3. Rust `self.field.method()` (`7963672`). +4. JS/TS object-literal namespaces (`278a8ed`). +5. TS config inheritance (`474f051`). +6. C++ plain-derived `.h` detection (`d618d94`). +7. Python aliased module imports (#1626). +8. Lua table/function-expression callables (#1616). +9. Watcher live-scope refresh (`cf1b0e3`). + +Each extraction/resolution item must grow the relevant byte-stable golden corpus +or add a new documented corpus when none exists. Watcher work must prove a +running watcher removes newly excluded files, admits newly included files, and +does not re-add either from stale queued events. + +### Wave 2 — retrieval and large-index behavior + +1. Explore path pinning / camelCase / variable seeds (`238dbc5`, `ccb0295`). +2. Exact-name index seek (`9219967`). +3. Ranking-only `deprioritize` (`1d9de88`). + +Required proof: + +- pinned paths survive tight output budgets; +- unmatched prose containing hyphens is not treated as a file; +- scores/output remain deterministic; +- the exact-name query uses `idx_nodes_lower_name`; +- `deprioritize` changes ranking only, never extraction, graph rows, or direct + exact-path retrieval. + +### Wave 3 — CLI and project discovery + +1. `.xsjs` / `.xsjslib` import candidates (`a740291`). +2. Project-local Codex installation (`bc89480`). +3. One-shot bootstrap flags (`0d17dfd`). +4. Exactly-one indexed subproject adoption (`a5c2709`). + +The `context` command remains deferred until an external compatibility need is +confirmed. + +## Likely Rust touch points + +- Explore/search: `crates/codegraph-mcp/src/engine.rs`, + `crates/codegraph-graph/src/query/`, + `crates/codegraph-store/src/queries.rs`, + `crates/codegraph-core/src/config.rs`. +- Extraction: `crates/codegraph-extract/src/walker.rs`, + `crates/codegraph-extract/src/engine.rs`, + `crates/codegraph-extract/src/lang/{rust,erlang,lua,python}.rs`. +- Resolution: `crates/codegraph-resolve/src/{path_aliases,import_resolver,name_matcher}.rs` + and snapshot-context equivalents. +- Watch: `crates/codegraph-watch/src/{watcher,policy,sync}.rs`. +- MCP project selection: `crates/codegraph-mcp/src/roots.rs` plus both server + front ends. +- Installer/CLI: `crates/codegraph-cli/src/main.rs`, + `crates/codegraph-cli/src/installer/targets/codex.rs`. +- Golden-sensitive fixtures: `crates/codegraph-bench/fixtures/`, + `reference/golden/`, and `docs/equivalence.md`. + +## Audit evidence + +- Fresh upstream clone at audited head: + `6a056ec5db35172f9dc348f87b54ea415aa5169e`. +- `git rev-list --count v1.5.0..c6aaa20` returned `104`, proving the previous + audit boundary. +- `git rev-list --count c6aaa20..6a056ec5` returned `28`. +- `git rev-list --count c6aaa20..dfccdf6` returned `27`. +- Local CodeGraph index was current at `codegraph-rs` `0.47.2` with 407 files, + 11,956 nodes, 37,876 edges, and zero pending changes. +- Local worktree was clean before this documentation update.