Skip to content

Fix crash in the GC roots server on fd reuse - #672

Merged
edolstra merged 1 commit into
mainfrom
fix-gc-server-thread-race
Oct 7, 2026
Merged

edolstra merged 1 commit into
mainfrom
fix-gc-server-thread-race

Conversation

@edolstra

@edolstra edolstra commented Oct 7, 2026 •

Copy link
Copy Markdown
Collaborator

Motivation

Fixes a std::terminate() crash (SIGABRT) in nix-daemon during garbage collection, seen in Sentry (issue 7679852086, 3.23.0).

Context

The GC roots server spawned a thread per client and only afterwards inserted it into the connections map keyed by fd. If a client disconnected immediately, the thread's cleanup handler ran before the insert, leaving a stale entry for a closed fd. When a later client reused that fd number, the duplicate-key insert destroyed a joinable std::thread, which calls std::terminate().

The fix holds the connections lock while starting and registering the thread, so the cleanup handler can't run before the entry exists. See the commit message for details.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Bug Fixes
    • Improved the reliability of background storage cleanup when multiple cleanup connections start or close concurrently. Connection tracking is now handled safely during these operations, reducing the risk of inconsistent cleanup state. No user-facing behavior or settings have changed.

The GC roots server started a thread per client connection and only
afterwards inserted it into the `connections` map keyed by the client
fd. The client thread's cleanup handler erases its own entry from
that map, so if a client disconnected immediately, the cleanup could
run before the insert. The cleanup then found nothing to erase, the
fd was closed, and the server inserted a finished-but-joinable thread
under a now-closed fd number.

When a later client got the same fd number from accept(), the server
tried to insert under an existing key. std::map::insert doesn't
overwrite, so the temporary pair holding the new (joinable)
std::thread was destroyed, which calls std::terminate(). This showed
up in Sentry as a SIGABRT in nix-daemon with no exception in flight,
and another thread blocked on the `connections` lock in its cleanup
handler.

Fix this by holding the `connections` lock while starting the client
thread and inserting it. The cleanup handler can't run until the
entry exists, and the fd is only closed after the cleanup, so fd
numbers can no longer collide. Also assert that the insert didn't hit
an existing key.

Assisted-by: Claude Fable 5.1 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Oct 7, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Essentials
  • Run ID: ab673640-1a06-4a90-89ef-ecd64689a039
📥 Commits

Reviewing files that changed from the base of the PR and between 1ef9034 and f5ed833.

📒 Files selected for processing (1)
  • src/libstore/gc.cc

Included review availability: This review used your included allowance. 4 included reviews remain after this review. Your included PR review attempts over the past 7 days set your current allowance at 5 reviews per hour.


📝 Walkthrough

Walkthrough

The GC socket server now holds the connections lock while starting a client thread and registering it. It asserts that registration succeeds. The updated comment describes the race between thread cleanup and registration.

Changes

GC Connection Registration

Layer / File(s) Summary
Client thread registration
src/libstore/gc.cc
The handler holds the connections lock during thread startup, registers the thread through that lock, and asserts insertion succeeded. The comment describes the cleanup race.

Priority: ➖ Normal

Estimated code review effort: 2 (Simple) | ~8 minutes

Change: Bug fix

Suggested reviewers: lisanna-dettwyler

Merge Risk: ⚪ Minimal · up to f5ed8

The change synchronizes client registration with cleanup, preventing the stale-entry race behind the reported crash. No unresolved merge risk remains.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 1 functions across 1 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the main change: fixing a crash in the GC roots server caused by file descriptor reuse.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@edolstra
edolstra enabled auto-merge October 7, 2026 11:43
@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown

@github-actions
github-actions Bot temporarily deployed to pull request October 7, 2026 11:51 Inactive
@edolstra
edolstra added this pull request to the merge queue Oct 7, 2026
Merged via the queue into main with commit e7b95e8 Oct 7, 2026
98 of 104 checks passed
@edolstra
edolstra deleted the fix-gc-server-thread-race branch October 7, 2026 15:35

This branch was previously deployed

1 inactive deployment
pull request — f5ed8334 Deployed Oct 7, 2026 by github-actions[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants