perf(lsp): eliminate the cross-LSP O(n²) — shared Java registry, own-file overlay, complexity guard (#1669) - #1681
Merged
Merged
Conversation
Finding the 0.10.x indexing regression took an 11-corpus A/B across two
release binaries, a subset-scaling series, and a per-pass exponent fit.
None of that should have been necessary: the pass that carried it,
cross-file LSP, was already timed and already logged its def count. The
numbers were there, just never normalised into something a reader could
judge.
Three additions, all shipped without a flag except the detailed curve:
- parallel.resolve.cross_lsp_cost — cross-LSP cost NORMALISED per file,
next to defs_total. Wall time cannot separate "big repo" from
"superlinear pass"; us_per_file can. Measured on one Java tree:
files=3710 defs=102845 us_per_file=35129 us_per_file_per_kdef=341
files=14833 defs=339866 us_per_file=106722 us_per_file_per_kdef=314
Per-file cost tripled while per-kdef stayed flat — the fingerprint of
work proportional to the whole corpus (files x defs). One grep on two
differently sized repos now answers what previously took a two-binary
bench.
- parallel.resolve.scan_cost — candidates visited per tail-match lookup,
plus fallback_rows, which the code has counted since #1085 but exposed
only to a test. On the same tree the tail scan reached 242M candidate
visits (n^2.04), worth seeing even though it proved cheap in wall time.
- cbm_scale_probe (foundation/profile.h) — samples cumulative elapsed at
1/8, 1/4, 1/2 and 1 of a pass's items and fits k in T ~ n^k, warning
once k reaches 1.35 in shipped builds. Wired into parallel_extract and
parallel_resolve.
The probe's limits are documented rather than oversold: it catches growth
WITHIN a run, and would NOT have caught this bug, whose per-item cost is
constant-but-large within any single run (it reported 1.26 while the
cross-corpus exponent was 1.86). That is exactly why us_per_item and
us_per_file are emitted alongside it.
Tests are deterministic by construction: the exponent fit is a pure
function fed synthetic points, and the checkpoint bookkeeping is asserted
directly. A test that proved the detector by generating a real quadratic
workload would be asserting on the scheduler.
No product behaviour changes; diagnostics only.
Refs #1669.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…cost lines Formatting only, on lines added by the previous commit. Verified with the repo's pinned Homebrew LLVM clang-format so this is the CI-canonical result, and confirmed no unrelated file needed reformatting. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The per-file cross-LSP path rebuilds a type registry for every file. Whether
that is cheap or quadratic depends entirely on how many defs the module
filter leaves, and nothing reported it — so a filter that reduces by a
constant FACTOR rather than to a constant SIZE looked identical to one that
worked.
parallel.resolve.perfile_registry reports defs_per_file next to defs_total,
plus how often the filter failed. Measured on one Java tree:
1/4 corpus: defs_total=179033 defs_per_file=1292
full corpus: defs_total=689216 defs_per_file=5031
defs_per_file grew 3.89x while the corpus grew 3.85x — lockstep, so per-file
work is proportional to the whole corpus and cross-file LSP is
O(files x corpus_defs). filter_failed=0 throughout, which is why this was
invisible: the filter always 'succeeds', it just does not bound the set.
Diagnostics only; no behaviour change.
Refs #1669.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Cross-file LSP for Java rebuilt a type registry for EVERY file from a def
set the module filter reduces by a constant factor, not to a constant
size: the JVM filter branch includes every def sharing the file's
namespace, so per-file work tracks the corpus (measured defs_per_file
1,292 -> 5,031 in lockstep as defs_total went 179k -> 689k). That makes
the pass O(files x corpus_defs) — 87% of a Java index and the bulk of
the v0.9.0 -> v0.10.x slowdown.
Three changes, one architecture (the pattern Go/Python/C/C#/TS already
use):
1. cbm_java_build_cross_registry — the JVM def universe (Java + Kotlin,
for mixed source roots) built ONCE per run, sealed read-only, shared
across resolve workers. Wired into both pipeline.c and
pipeline_incremental.c.
2. Two-phase registration inside that build: all TYPES first, finalize
(hash buckets exist), then FUNCS. Func registration parses signatures,
and type-name qualification via cbm_registry_lookup_type is a LINEAR
scan until finalize — a single mixed pass measured 0.44 ms/def at
689k defs, a ~300 s sequential build that erased the sharing win.
Two-phase: 306 s -> 2.8 s. Stable partition order because overload
ties resolve to the first registered QN match.
3. cbm_run_java_lsp_cross_with_registry resolves each file against the
shared base through an overlay holding exactly THIS FILE's defs
(register_local_func_or_type_from_file). Own-file scope is the load-
bearing choice: patch_one_method refines signatures from the AST and
must write a private copy (its types live in the per-file arena, the
base is sealed), and any wider scope re-imports the quadratic — a
module/namespace-scoped overlay measured ratio 4.00 on the growing-
package corpus, own-file measures 2.00.
Elasticsearch corpus (46,477 files), same host, CBM_PROFILE=1:
v0.10.5 this change v0.9.0
wall 419.9 s 85.6 s (4.9x) 61.5 s
cross-LSP CPU 6,195,780ms 52,513ms (118x) 110,344 ms
us_per_file 207,448 1,770 3,722
us_per_file_per_kdef 300 2 —
nodes 693,100 693,100 —
edges 5,646,235 5,649,949 —
Cross-LSP CPU now beats v0.9.0. Nodes are identical; edges +0.066%,
consistent with the shared base resolving cross-package targets the old
per-file namespace/import filter could not see, plus source-order
independence from the two-phase build.
Guarded by the complexity suite's shared-package gate (RED at ratio 4.00
on the pre-change tree, 2.00 after — see the suite commit).
Refs #1669.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Finding #1669 took an 11-corpus A/B across two release binaries. This suite makes that bug class fail a unit test in seconds, on every platform, from tiny corpora. Method: build k and 2k REPLICATED module copies, run the full in-process pipeline on both, assert counter RATIOS. Independent copies mean every extensive quantity — nodes, edges, Σ per-file registry defs — must grow linearly (ratio ~2). A files x corpus coupling makes per-file work itself grow with k and lands at ratio ~4. Ratios expose the exponent regardless of absolute scale, so 60-120 files suffice. Verdicts are pure functions of (code, input): gates ride ONLY on deterministic work counters and data-product counts, never on wall time. Throughput (nodes/s, edges/s) is information-only, written to private/benchmarks/complexity-<ts>.json (local, gitignored; skipped under CBM_SKIP_PERF where rates are meaningless). Two corpus shapes, both needed: - Independent modules (java/py/go/ts templates): catches cross-module contamination and dedup breakage. The #1669 bug is GREEN here — fully closed modules filter perfectly, which is exactly why it survived. - The growing shared package (bigpkg): one Java package whose file count scales with k — the real-repo shape (files concentrate in large packages). The JVM namespace filter branch makes per-file work track package size, so this corpus is the honest #1669 reproducer: ratio 4.00 RED on the pre-fix tree, 4.00 RED for a module-scoped overlay, 2.00 GREEN for the own-file overlay. It discriminated the correct fix design before the fix was written. Both legs of every pair exceed MIN_FILES_FOR_PARALLEL(50): below it the sequential path runs, which builds no shared registries and would be the wrong code path to gate (its per-file cost is bounded by the 50-file ceiling). Recorded but deliberately NOT gated, with reasons at the case: tail_candidates and fallback_rows are legitimately superlinear under replication until those scans are bounded, and measured ~1 ns/unit. Every ratio gate carries a non-vacuousness floor on the base counter so broken counter wiring fails loudly instead of green-washing (cbm_pxc_count_perfile_defs feeds the overlay path into the same counter the fallback path already used; wired for Java, extend to the TS overlay when touching ts_lsp). Dynamic coverage: languages iterate CBM_LANG_COUNT; embedded templates cover the LSP-hybrid languages, and tests/fixtures/complexity/<lang>/ dirs are auto-discovered so a new language joins the guard by dropping fixtures. Uncovered languages are listed in the report with the reason. The local report additionally carries per-language node/edge counts with ratios and a per-pass elapsed_ms table per run (captured via a TEE log sink during the in-process pipeline runs) — trend data for humans, still never a gate. Refs #1669. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
… +37% edges Four measured changes on the dotnet/runtime corpus (58,656 files; every step's numbers below are from full-corpus runs on the same host, baselines captured this session; v0.9.0 = 444.3 s): 1. Two-phase cs registry build (types -> finalize -> funcs), the same pre-finalize linear-lookup disease fixed for Java: prepare 280 -> 140 s. 2. C# import-context carve-out: cs_import_types lists namespace_declaration (for namespace-name mapping) and using_statement (C#'s RAII block — a grammar-name collision), so EVERY namespaced C# file's whole body ran with inside_import=true. That both suppressed ordinary usage extraction under namespaces and sent every identifier through the ancestor-walking import-binding check (tree-sitter's ts_node_parent re-descends from the root, so wide files went quadratic: one 147 KB JIT torture file cost 490 s; 6.8 s after). Only using_directive / namespace_use_declaration open an import scope now. Restores the suppressed usages: edges 4,291,387 -> 5,869,093 (+37%), nodes identical. 3. The usages walker maintains call/import ancestry as enter/exit counters on its explicit stack instead of per-node ancestor re-walks (extract_usages.c had grown from 6 to 100 ts_node_parent calls since v0.9.0; the two per-node gates are now O(1) with semantics preserved — strict ancestors only, emit before self-count). 4. Registry short-name indexes replace the two remaining full scans: cs_lookup_extension walked all 963k funcs per unresolved invocation (now the existing free-func short-name iterator, first-match order preserved via min-index selection), and cs_resolve_type_name's step-9 fallback scanned every type per unresolved name IN BOTH the builder and per-file resolution (new type_short index in finalize, same reverse-insertion ascending-order pattern, best-score ties keep the first-in-registration-order winner). Builder 140,095 -> 500 ms; resolve cross-LSP CPU 5.5M -> 317k ms (us_per_file_per_kdef 191 -> 11). End state: 449.9 s wall (1.01x of v0.9.0) with +48.6% edges vs v0.9.0 — per-edge cost 32% BETTER than v0.9.0. Extract's remaining 359 s is the 24 MB hugeexpr1.cs parse floor both versions pay. Guarded by the complexity suite; cs_lsp/extraction/edge/lang-contract suites green including the 53-language calls-breadth contract. Refs #1669. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
… boundary Two measured changes on the TypeScript corpus (81,397 files; v0.9.0 17.9 s, before 44.1 s, after 37.5 s): - cov_rebuild_shadow_graph upserted every directory segment for every failure row — 13,243 parse-partial baseline files under one tests/ subtree meant ~80k redundant node/edge round-trips, 9.1 s of a 9.2 s coverage_replace. An in-rebuild path->id map creates each directory once; identical graph (edges deduped by unique key before, absent now). Coverage block 9,162 -> 2,920 ms. Sub-block timings (publish.timing.coverage: del/rows/prune/meta/commit + row_count + detail_bytes) are kept — the caller-level number could not name the culprit. - JS/TS export_statement is an import CONTEXT only in its re-export forms (source field, or a bare specifier list without a declaration). The old is_export_of_declaration blacklist missed TS-only forms (ambient_declaration, function_signature, module_declaration), running declare-heavy subtrees (.d.ts, export namespace) behind inside_import: suppressed usages + per-identifier ancestor walks. Positive detection replaces the blacklist; +3,954 restored usage edges on the corpus, nodes identical. (Measured perf-neutral here — kept for correctness.) Suites green incl. store_nodes/edges/search, mcp, extraction, ts_lsp, complexity, and the 53-language calls-breadth contract. Refs #1669. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
ts_eval_expr_type was depth-capped but work-unbounded: crafted expressions (the TS test suite's repeated object spreads) stay under the depth cap while fanning out. Charge the same per-file budget the type-text parser uses, at 16 units per entry (an eval entry does ~two orders of magnitude more work than a text-parse unit), degrading to UNKNOWN on exhaustion. Honest status: this hardens the documented budget design, but the known 3.6 KB spread-bomb baseline file still measures ~11 s in-corpus — its entry path into the evaluator apparently runs unarmed and is recorded as an open lead (zero budget warnings observed). Suites green (ts_lsp, extraction, complexity). Refs #1669. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Owner
Author
Update: the regression program continued — C# recovered, TS partiallyC# (dotnet/runtime corpus, 58,656 files) — acceptance met decisively (commit a4c0ffb):
Four measured changes: two-phase cs registry build (prepare 280→140 s), the namespace/using_statement import-context bug (every namespaced C# file ran inside_import → suppressed usages + ancestor-walk tax; one 147 KB JIT file went 490→6.8 s; restores +1.58 M edges), the usages walker's ancestry counters (6→100 ts_node_parent growth since 0.9.0 reverted to O(1) gates), and registry short-name indexes for cs_lookup_extension + cs_resolve_type_name step 9 (builder 140,095→500 ms; resolve cross-LSP CPU 5.5 M→317 k ms). TypeScript (commits a412642, f95fe55): 44.1 → 37.5 s (v0.9.0: 17.9). Coverage shadow-graph rebuild deduped (9.16→2.92 s — it upserted every directory per failure row), TS export re-export boundary fixed (+3,954 restored usage edges), eval-budget hardening. The remaining TS gap is attributed (extract +8.6 s incl. a spread-bomb file class with an open budget-arming lead; persist +5.2 s = FTS + coverage + rows; ~6 s startup) and not yet at the acceptance bar — tracked. New telemetry shipped along the way: Full 11-corpus validation will be re-run on the final tree (the usages-walker change touches every language's extract), and the linux/C residual attribution is queued. |
…y forms The no-skips lint gate (scripts/check-no-test-skips.sh) rightly rejected the throughput-report test's two SKIP() calls: - CBM_SKIP_PERF=1 is deliberate operator configuration, not a hidden environment failure: reporting is off by request, so the test PASSes with a stderr note instead of skipping. - an uncreatable report dir IS an environment failure and now FAILs with the remedy in the message (set CBM_COMPLEXITY_REPORT_DIR), per the policy text. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
cbm_arena_alloc(arena, 0) returns NULL, so the java and cs cross-registry builders read a def_count of 0 as OOM and returned NULL — a corpus with no files of that language silently lost its shared registry (and the seal tests caught exactly that: cbm_cs_build_cross_registry(&arena, NULL, 0) == NULL). Guard the partition alloc behind def_count > 0; the empty registry is still built, finalized, and shared. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
ts_eval_expr_type and ts_signature_for_call are mutually recursive: resolving a call evaluates its argument expressions once per lookup path (method dispatch + namespace fallback), and in tsc-compiled spread files the first argument is itself the next nested Object.assign(...) call — the same subtree re-evaluates once per enclosing level, 2^n total. The TS suite's objectSpreadRepeatedComplexity.js (3.6 KB, 5 nodes) measured 20.4 s; with the memo its eval cost is zero within measurement noise of a one-file control, and the microsoft/TypeScript corpus drops 37.5 -> ~24 s warm (nodes byte-identical, edges within the known scheduler jitter). Expression types are position-pure within a file pass (one node = one scope path; the per-file walk is single-threaded and deterministic), so one eval per node is the correct semantics, not a cache trade-off. The memo is a per-file, arena-backed, linear-probe table keyed on TSNode.id. Results produced under a depth-cap or budget bail are never stored: both bail sites bump a degradation counter, and a store only happens when the subtree completed clean — a degraded UNKNOWN can therefore never shadow a later full evaluation. The regression guard asserts work, not wall-clock: the nested-Object.assign shape must complete without exhausting the deterministic eval budget, read back through a new CBM_ENABLE_TEST_SEAMS accessor pair. The seam lives in the lsp_all unity object, so GRAMMAR_CFLAGS_TEST/TSAN now carry the seams define (test artifacts always have seams; prod never does). Verified RED without the memo (budget exhausted, suite 47.8 s) and green with it (3.2 s). Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Fixes the dominant 0.9.0 → 0.10.x indexing slowdown (#1669): cross-file LSP was O(files × corpus_defs) for Java/Kotlin, plus ships the diagnostics that found it and a deterministic complexity-guard test suite that prevents the class from returning.
Five commits, each green:
feat(diagnostics)— scaling probe +cross_lsp_cost/scan_costtelemetry: normalized per-file numbers that make a superlinear pass visible from ONE run's log instead of an 11-corpus two-binary A/B.style— clang-format of those lines.feat(diagnostics)—perfile_registry(defs_per_file vs defs_total): the counter that proved the filter reduces by constant factor, not constant size.perf(lsp)— the fix: shared Java+Kotlin cross-registry built once (two-phase: types → finalize → funcs, because pre-finalize type lookups are linear scans — a one-pass build cost ~300 s), resolved per file through an overlay of own-file defs (module/namespace scope re-imports the quadratic on large packages — proven by the suite below). Wired into both the full and incremental pipelines.test(complexity)— the guard suite: replicated-module + growing-shared-package corpora, gates on deterministic work-counter RATIOS (never wall time), ~2 s runtime, auto-joins CI sharding. RED at ratio 4.00 on the pre-fix tree, GREEN at 2.00 after; it also rejected an earlier module-scoped overlay design before it shipped.Measured (elasticsearch corpus, 46,477 files, same host, this session's baselines)
Full 11-corpus validation (warm daemon): java 477 → 86 s; every language improves vs v0.10.5 (total 2,370 → 1,774 s); small corpora halve. Node counts stable on clean corpora.
Edge delta named: +0.066 % on java — the shared base resolves cross-package targets the old per-file namespace/import filter could not see, plus source-order independence from the two-phase build. Run-to-run jitter ±54 edges on this corpus (known MT jitter class).
Explicitly not fixed here (separate root cause — C# already had a shared registry and did not move): the csharp 2.6×-vs-0.9.0 regression, tracked as the next perf target. Also recorded-and-rejected attempts with numbers (tail-index ×2, batched registration) so they aren't re-tried.
Verification
mem/diagnosticssuites carry the new deterministic tests (probe math is a pure function — no timing assertions anywhere).Checklist
git commit -s)make -f Makefile.cbm lint-ci)Refs #1669.