Skip to content

perf: stop the code tab forking git per tree entry - #421

Merged
espadonne merged 6 commits into
trunkfrom
perf/code-tab-git-forks
Sep 2, 2026
Merged

espadonne merged 6 commits into
trunkfrom
perf/code-tab-git-forks

Conversation

@espadonne

@espadonne espadonne commented Sep 2, 2026 •

Copy link
Copy Markdown
Contributor

Phase 3 (CPU) of the availability campaign: the repo home / code tab is
the most-crawled page on the site and forked git O(entries) per
anonymous request.

  • de3ab45a — every git subprocess in internal/repos/git now goes
    through one helper that counts forks (repogit.ForkCount()) and can
    carry process-wide policy. Read-only calls on the code-tab paths get
    a 30 s deadline (repogit.ReadTimeout); blob streaming and
    pack/transport are excluded.
  • 909ba217 — repogit.EntryLastCommits: one streamed
    git log --name-only walk per directory, killed mid-stream once
    every entry resolves, bounded at 2,000 commits. Entries the walk
    can't resolve fall back to the old per-path git log -1, so
    rendered output is unchanged.
  • 6c0a803b — internal/web/handlers/repo/treecache: OID-keyed LRU +
    singleflight over the last-commit map, rev-list --count, the
    ls-tree -r language aggregate and the log -n 500 contributor
    tally. 2,048 / 512 entries, 10 min TTL. Invalidation is structural —
    a push changes the commit OID, so it changes the key; no hook to
    forget.
  • eeba23c1 — wires it through repo.Deps.TreeCache, built once in
    internal/web/repo_wiring.go (same shape as httpcache.PageCache).
  • 98219e85 — handler-level fork-count regression test.
  • a6eb19f8 — docs/internal/caching.md, docs/internal/code-tab.md,
    sitrep item ticked.

Fork counts (measured, cold anonymous root-tree render)

Directory Before After (cold) After (warm cache)
6 entries 15 10 6
81 entries 90 10 6

O(N) → O(1) in entry count. "Before" measured on this branch with the
walk disabled, so it is the same fixture and the same counter.

Tests

  • internal/repos/git — single-walk resolver: root + nested dirs,
    renames, never-touched entries, walk-bound stragglers, parity
    against per-path git log -1, fork count constant at 5 and 60
    entries, bad ref
  • internal/web/handlers/repo/treecache — hit/miss, OID-change
    invalidation, key independence, TTL, LRU eviction, errors not
    cached, singleflight collapse, nil receiver
  • internal/web/handlers/repo — TestCodeTree_*: forks constant
    in entry count + ceiling, warm cache drops forks with identical
    body, post-push render is not stale
  • make ci green end to end against Postgres 16 (golangci-lint
    2.13 + gofumpt, all boundary/spdx/unused/migration/systemd lints,
    verify-api-docs, full go test ./..., build), rebased onto
    current trunk

@mfwolffe
mfwolffe force-pushed the perf/code-tab-git-forks branch from 728afef to a6eb19f Compare September 2, 2026 07:02
@espadonne
espadonne merged commit b0b846b into trunk Sep 2, 2026
1 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants