Cross-file LSP pass regressed 16.6x (21.5s → 358s on java) and is 78% of the 0.10.x indexing slowdown

[DeusData]

Version

0.10.5 (regression against 0.9.0)

Platform

macOS arm64 (48 GB, 18 CPU); measured with the shipped release binaries of both versions

Summary

The cross-file LSP pass regressed 16.6x between v0.9.0 and v0.10.5 — from 21.5 s to ~358 s on a Java corpus — and it accounts for roughly 78% of the total java indexing regression. It contributes 25,885 edges, about 0.5% of the graph, so the cost/benefit is roughly 14 ms per edge.

Measurement

Corpus: 46,477-file Java tree. Shipped binaries, same host, isolated HOME/CBM_CACHE_DIR per run, sequential (no concurrent load).

	v0.9.0	v0.10.5
baseline	99.8 s	~530 s
CBM_DISABLE_LSP_CROSS=1	78.3 s	~172 s
cross-LSP pass alone	21.5 s	~358 s (16.6x)
rest of the pipeline	78.3 s	~172 s (2.2x)

Reproducibility: the 0.10.5 cuts were run twice — xlsp-off 175.0 s and 169.1 s; lsp-off 522.0 s and 525.2 s. Baselines 477.1 s (first sweep) and 534.5 s (recheck), so ~10% run-to-run variance, well inside the effect size.

Edge counts barely move, which is what makes the cost hard to justify as-is:

	edges
0.10.5 baseline	5,646,309
0.10.5 CBM_DISABLE_LSP_CROSS=1	5,620,424

Note also that CBM_LSP_DISABLED=1 (disabling LSP entirely) barely changes wall time (522 s vs 530 s baseline) — so the cost is specifically the cross-file pass, not per-file LSP.

Reproduction

for cut in baseline xlsp-off; do
  W=$(mktemp -d)
  ENVV=""; [ "$cut" = "xlsp-off" ] && ENVV="CBM_DISABLE_LSP_CROSS=1"
  time env HOME="$W" CBM_CACHE_DIR="$W/.cache" $ENVV ./codebase-memory-mcp cli index_repository \
    --repo-path /path/to/java-corpus --mode full --name "$cut" --persistence true
done

Where to look

src/pipeline/pass_lsp_cross.c gained +628 lines between the tags, and src/pipeline/lsp_resolve.h +774, which is the natural place to start. pipeline.c already labels this "the most expensive phase", and its comment notes the old sequential implementation cost ~520 s on Kubernetes — so the parallel rewrite was supposed to be the cure here.

This may be the same root as the recorded LSP-resolve audit finding (short-name lookups bypassing QN-only buckets, and no negative memoisation), which would explain why a language with many short, heavily-overloaded method names — Java — is hit hardest.

Cross-corpus context

From the same A/B sweep (11 corpora, both versions), java is by far the worst but the regression is broad:

corpus	v0.9.0	v0.10.5	ratio
java (46,477 files)	79.5 s	477.1 s	6.0x
csharp (58,656)	444.3 s	1210.8 s	2.7x
typescript (81,397)	20.7 s	52.0 s	2.5x
linux (94,769)	243.8 s	508.5 s	2.1x
go (31,323)	38.4 s	60.2 s	1.6x

Peak RSS moved only +5–15% across all of them, so this is a throughput regression, not a memory one (the memory side is #1654, which is Linux-specific and separate).

[DeusData]

Narrowed to a single phase. CBM_PROFILE=1 on the same Java corpus (v0.10.5, full index, 693,100 nodes / 5,646,235 edges):

pass	elapsed
parallel_resolve	362,160 ms
parallel_extract	17,789 ms
dump_and_persist	14,571 ms
semantic_edges	8,129 ms
registry_build	3,751 ms
lsp_cross_prepare	2,857 ms
tests / similarity / configlink / decorator_tags / route_match / complexity / githistory / structure / k8s	< 2,400 ms each

parallel_resolve is 87% of the run, and its 362 s matches the ~358 s that CBM_DISABLE_LSP_CROSS=1 removes — consistent with pipeline.c's own comment that with cross-LSP off "all_defs stays NULL and the fused resolver simply no-ops cross-file resolution". So the cost is inside the fused resolver's cross-file path, not in lsp_cross_prepare (2.9 s) and not in extraction (17.8 s).

Note the shape: preparing the cross-file inputs is cheap; consuming them is not. That points at the per-candidate lookup rather than at collection — which lines up with the earlier LSP-resolve audit finding (short-name lookups bypassing the QN-only buckets, with no negative memoisation). Java is the worst case for exactly that: many short, heavily-overloaded method names, so a short-name lookup fans out across a large candidate set and a miss costs the same as a hit every time it recurs.

Suggested first probes for whoever picks this up:

1. Instrument parallel_resolve for candidate-set size per lookup and hit/miss ratio — if misses dominate and repeat, a negative memo is the cheap win.
2. Check whether short-name lookups are hitting a QN-only bucket path and falling back to a scan.
3. Compare against a language with long, unique names (Go finished in 60 s here) to confirm the fan-out hypothesis rather than assuming it.

[DeusData]

Root cause found and proven. Correcting my earlier comment on this issue, which pointed at the wrong thing.

It is the per-file registry build, not a lookup

Cross-file LSP for Java runs, per file:

cbm_registry_init(&reg, arena);
cbm_java_register_lsp_defs(arena, &reg, defs, def_count);
cbm_registry_finalize_into(&reg, &idx_arena);   // build hash indexes
// ... then parse + walk the AST

Go, Python, C/C++, C# and TS skip this: cbm_pxc_dispatch_file hands them a shared prebuilt registry built once for the whole run. Java, Kotlin and PHP have no such registry (cbm_pxc_registry_for_lang has no case for them), so they take the cbm_pxc_run_one fallback and rebuild from scratch for every file.

That would be fine if the per-file def set were bounded. It is not:

	1/4 corpus	full corpus	growth
defs_total	179,033	689,216	3.85x
defs_per_file	1,292	5,031	3.89x
us_per_file	55,210	207,448	3.76x
us_per_file_per_kdef	308	300	flat

defs_per_file grows in lockstep with the corpus. The module filter reduces by a constant factor (~0.73%), not to a constant size — so each file's work is proportional to the entire corpus, and the pass is files x corpus_defs = O(n^2).

Note filter_failed=0 throughout. The filter always "succeeds", which is exactly why this stayed invisible: nothing reported how much it actually left behind.

Against v0.9.0, same corpus, same command

	v0.9.0	v0.10.5	ratio
files visited	40,542	40,542	1.00
files processed by cross-LSP	29,642	29,661	1.00
defs_total	413,302	689,216	1.67x
cross_lsp	110,344 ms	6,195,780 ms	56x
resolve_calls	195,070 ms	176,637 ms	0.91x
wall	61.5 s	419.9 s	6.8x

Identical file counts, so the widened eligibility condition (calls.count > 0 -> semantic_sites > 0 || pending_lsp_site) is not the cause. resolve_calls even got faster. The whole regression is cross-LSP per-file work: 3.7 ms/file -> 208.9 ms/file.

Hypotheses tested and rejected

Recording these so nobody re-walks them:

1. The uncapped tail-match scan in cbm_pipeline_lsp_target_node_policy. Real and genuinely quadratic by counter (242M candidate visits, n^2.04), but indexing it away cut visits 281x and changed wall time by 0% — those visits cost ~1ns each.
2. Java overload selection scanning reg->func_count per method declaration (java_lsp.c:3468). Converting it to the existing func-QN index bought ~1%.
3. The module filter failing for Java — it never fails (filter_failed=0).
4. Eligibility widening pulling in more files — identical file counts (29,642 vs 29,661).
5. Backpressure / memory budget — that block is byte-identical between the tags.

Suggested fix

Give Java, Kotlin and PHP the shared prebuilt registry + per-file overlay that Go/Python/C/C#/TS already use. Cost becomes O(corpus) once plus O(file) per file. The code already anticipates this — pass_lsp_cross.c says "PHP falls through to the per-file build path below until its overlay variant lands" — so this is finishing an acknowledged gap, not new architecture.

Failing that, the filter needs to bound the set by size rather than by factor, which would trade graph quality for time and is the worse option.

This is not Java-only

Raw registry scans (reg->func_count / reg->type_count) that run per file exist across eight language LSPs — ts_lsp.c has 7, php_lsp.c 7, py_lsp.c 3, rust_lsp.c 2, java/go/cs/perl the rest. Each is files x corpus_defs for the same reason. They are individually cheap (see rejected hypotheses 1 and 2), but they are the same anti-pattern the project's own LSP checklist bans as "no registry tail-scan", and they are worth sweeping once the registry work above lands.

Diagnostics

The counters used here now ship (branch feat/scaling-probe): parallel.resolve.cross_lsp_cost (us_per_file next to defs_total) and parallel.resolve.perfile_registry (defs_per_file, filter_failed). Two greps on two differently sized repos reproduce the table above without an A/B of release binaries.

[DeusData]

Resolved by #1681 (merge commit 41d240a), landing in v0.10.6. The full arc, validated per-language against same-host v0.9.0 and v0.10.5 baselines: shared cross-LSP Java registry + two-phase Java/C# builds (java 477→86 s, csharp 1211→450 s with +48.6% edges), the usages-walker ancestry rewrite, a per-node TS expression-eval memo (typescript 37.5→~24 s), registry short-name indexes, coverage-rebuild dedup, and a deterministic O(n²) complexity guard suite (counter ratios, never wall time) so this class of regression trips a unit gate from now on.