fix: indexing OOM on large repos + resume/reliability + faster embedding

[dvcdsys]

Summary

Indexing a large external repo (reproduced with github.com/microsoft/vscode) drove cix-server RSS to 9–100 GB and stalled at 0 files processed, taking the host down with it. This PR fixes the OOM and bundles the related indexing reliability/throughput work — including a follow-up round of fixes for a session-leak + embedding-backpressure deadlock surfaced while indexing vscode under the Voyage provider.

Root cause (the OOM)

Heap profiling (-inuse_space) showed the live heap was almost entirely gotreesitter parser tables, node arenas and GLR scratch. Parse's error-recovery/snippet machinery calls NewParser repeatedly, and NewParser rebuilds the grammar's full LR tables (~175 MB for TypeScript) every call; the copies piled up in process-global parser pools faster than the GC could reclaim them. The isolated chunker never reproduced it (~1.8 GB) — only the live server's runtime did.

The pinned gotreesitter predated the upstream fixes for exactly this. Bumping to v0.20.2 (bounded recovery sub-parses, recovery-parser pooling, GLR merge caps, arena reuse) keeps the vscode index stable at ~2.5 GB and progressing.

Root cause (the stuck-error deadlock)

While indexing vscode under Voyage, a single transient embedding queue saturated, retry after 5s (ErrBusy) failed the whole run. Two compounding defects turned a retryable backpressure signal into a multi-minute outage:

1. Session leak — repoindexer.IndexDir released the indexer session only on success (FinishIndexing) or explicit force-stop. Any mid-run error left the session active until the idle-timeout reaped it, so every queue retry and every manual Sync bounced off ErrSessionConflict ("session already active") in the meantime. The failure that triggered the retry also blocked it.
2. ErrBusy treated as fatal — the HTTP/CLI path honours ErrBusy via 503 + Retry-After, but the in-process driver propagated it as a permanent walk failure with no retry.

Fixed by releasing the session on abort (FailIndexing) and by riding out ErrBusy with hinted backoff in-process. A related Voyage planning-headroom fix removes the bisect noise that aggravated the saturation.

What's in here (commit-by-commit)

1. fix(deps) — bump gotreesitter v0.20.2 (the OOM fix).
2. feat(config) — dashboard-tunable embed batch size (index_embed_batch_chunks).
3. feat(indexer) — parallel cross-file embedding pipeline; idle-based 10-min session TTL (no more silent wall-clock kill of long indexes); resume-on-restart via stored file SHAs + ForceFull; orphaned-job recovery + ReconcileStuckProjects; cancellable jobs context for clean shutdown.
4. feat(server) — opt-in localhost pprof endpoint (CIX_PPROF_ADDR).
5. fix(indexer) — release session when an in-process run aborts mid-walk. New Service.FailIndexing(projectPath, runID) removes the active session immediately and marks the run failed, without flipping projects.status (repojobs owns the terminal state) or setting a user-cancel tombstone. IndexDir defers it on every abort path; idempotent on success / already-removed sessions. Retries now re-enter via reconcile-resume instead of ErrSessionConflict.
6. fix(indexer) — ride out embedding-queue backpressure instead of failing. flush() now backs off the ErrBusy Retry-After hint (capped 30s, total bounded 5m) and retries the same batch; non-busy errors still return at once. Voyage defaultMaxTokensPerBatch 100K → 80K: estimateTokens divides bytes by 2 but dense code runs ~1.4 bytes/token (a ~43% undercount), so the old 17%-headroom target shipped estimated-95K batches as real-122K POSTs that tripped Voyage's 120K cap and forced adaptive bisect+retry on every hot file. 80K keeps the worst case (~114K) under the cap.

Verification

• vscode index: ~2.5 GB stable (was 9–100 GB / stalled).
• Full server test suite: 40 packages pass, 0 failures.
• New regression tests: TestFailIndexing_ReleasesSessionForRetry (orphaned session released for retry, run marked failed, idempotent) and TestIndexDir_RetriesEmbeddingQueueSaturation (rides out an ErrBusy-then-OK embedder and still indexes the file).

⚠️ Notes for reviewer / deploy

• Re-index required — the newer grammars change chunk output for some languages.
• Known caveat: gotreesitter ≥ v0.19.0 has a C-grammar regression where an enum corrupts the surrounding parse, so functions in such files degrade from function chunks to generic module chunks (content still indexed/searchable). Documented + tracked via skipped TestChunkFile_C_EnumRegression; worth filing upstream.

🤖 Generated with Claude Code

[dvcdsys]

Pushed two follow-up commits (963df02, d507e17) fixing a session-leak + embedding-backpressure deadlock found while indexing vscode under Voyage:

• Session leak — a mid-run abort (e.g. transient embedding queue saturated) left the indexer session active until the idle-timeout, so every retry/Sync bounced off ErrSessionConflict. IndexDir now releases the session on abort via FailIndexing.
• ErrBusy was fatal in-process — flush() now backs off the Retry-After hint and retries the batch (the HTTP/CLI path already did this via 503).
• Voyage headroom — defaultMaxTokensPerBatch 100K → 80K so the byte→token estimator's ~43% undercount on dense code stops tripping Voyage's 120K cap and forcing bisect+retry.

Two new regression tests; full suite still 40 pkgs / 0 failures. Body updated with the details.