feat(indexing): add dryRun mode to index_codebase for parse-only token totals

[dkhokhlov]

Summary

index_codebase accepts a new dryRun: true flag (CLI: index --dry-run) that parses the real file set, builds the embedding text for every indexable chunk, and sums estimateTokens over
those texts without requesting embeddings or writing to the index. It returns the file count, chunk count, and the token total a force index would consume — a read-only preflight.

Motivation

A live index_codebase run reports progress as a moving chunk percentage whose denominator grows during parse, so the percent dips backwards (60% → 36% → 62%) and is unfit as a progress gauge.
The dry-run total is a fixed, monotonic denominator: percent = tokensUsed * 100 / dryRunTotal climbs cleanly to ~100% for a force index and tops out below 100% for an incremental (cached
chunks counted in the total but not re-embedded).

It also gives a cost/throughput preview before committing GPU/time: e.g. ~/.claude/projects → 970 files / 72,488 chunks / 76.5M tokens in ~13 s, with no ollama call.

Behavior

• Implemented as a standalone read-only Indexer.dryRunCost() that reuses the exact index() pipeline: collectFiles → parseFiles(linesPerChunk) → fallbackToTextOnMaxChunks →
  selectIndexableChunks → createEmbeddingTexts → estimateTokens. Same chunking as a real index, so the chunk count and token sum match a force index exactly (same provider).
• No side effects: no DB writes, no embedding requests, no withIndexMutationLease, opens the DB read-only. (Provider detection may still contact the provider's tags endpoint, e.g. ollama
  /api/tags + /api/show; no embeddings are requested and no writes occur.)
• Result is a new kind: "dryrun" member of the index result union, formatted by formatDryRunEstimate in src/utils/cost.ts.
• chunksCount counts source chunks (matches forceIndex's indexedChunks); tokensToEmbed sums per embedding text (matches tokensUsed). A chunk that splits into multiple embedding texts
  still counts as one chunk.
• Plumbed through operations.ts, execute-common.ts, contracts.ts (SharedIndexCodebaseArgs), the MCP/opencode/pi tool schemas, and the CLI (parseIndexArgs + --dry-run + usage).

Exactness of the token total (important)

The dry-run sums the local estimateTokens(text) = ceil(len/4). This equals the live "Tokens used" counter only for providers that report usage on the same basis — ollama counts
ceil(len/4), so for ollama the dry-run total is exact. For providers that report a server tokenizer count (OpenAI, Gemini, custom), the dry-run value is an estimate, not exact.

It is also an upper bound in two cases even for a matching provider:

• a force index on a shared global index can reuse cached embeddings from other projects (not all chunks are re-embedded), and
• an incremental index counts cached chunks in the total that are not re-embedded.

In both cases a progress percent against this total tops out below 100%. The output states this explicitly.

Tests

• tests/dryrun-index.test.ts (new): integration test with a mocked ollama provider — asserts no embedding calls are made and the index is left unindexed; a subsequent force index reaches
  stats.tokensUsed == dryRun.tokensToEmbed and stats.indexedChunks == dryRun.chunksCount; idempotent across repeated dry-runs. Includes a split-chunk regression test: one oversized source chunk
  that splits into multiple embedding texts still keeps chunksCount == indexedChunks and tokensToEmbed == tokensUsed, with more embedding requests than chunks.
• tests/cost.test.ts: formatDryRunEstimate formatting + the provider-conditional / upper-bound wording assertions.
• tests/mcp-cli-index.test.ts: parseIndexArgs --dry-run parsing + the dryRun: false field in the existing CLI arg/execution assertions.

typecheck + lint clean; full vitest suite green (one pre-existing watcher flake is unrelated to this change).

[Helweg]

Thanks for the contribution. I exercised the native build and the affected dry-run, token-cost, and MCP CLI suites locally: 50 tests pass. Before this is ready to merge, could you please address these items?

1. The public dry-run wording says it avoids calling the embedding provider. A fresh run can still perform provider detection, including provider metadata discovery. Please narrow the guarantee to no embedding requests and no index writes, or avoid provider detection in dry-run mode.
2. chunksCount increments per generated embedding text. A source chunk can split into multiple embedding texts, so this does not necessarily match the indexed source-chunk count. Please either report the metric as embedding texts or count source chunks once, and add a split-chunk regression test.
3. Please document the index_codebase dryRun option and CLI index --dry-run, and add the user-visible change to CHANGELOG.md.
4. The PR description says a CI wrapper uses this mode, but no such change is included. Please either include or remove that claim.

[dkhokhlov]

@Helweg
updated PR text and narrowed the guarantee to "no embedding requests" (detection still runs for maxTokens), chunksCount now counts source chunks with a split-chunk regression test, added docs + CHANGELOG, and removed the CI-wrapper reference from the description.