computeFSClosure(): Recurse rather than loop

[edolstra]

Motivation

Fixes an issue with #539: with stores like binary caches that don't implement a queryPathInfos() that returns all paths at the same time, we shouldn't wait until all paths have been received but do a recursive call on the incoming path immediately.

Context

Summary by CodeRabbit

• Refactor

  • Improved filesystem closure processing by batching newly discovered paths and scheduling follow-up work more efficiently.

• Bug Fixes

  • Enhanced closure expansion reliability by ensuring discovered paths and related derivations are consistently scheduled.
  • Strengthened error propagation so failures during closure expansion surface predictably.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: ba1c356c-fc5b-4b62-ba6a-0057ab7164e6

📥 Commits

Reviewing files that changed from the base of the PR and between 7656214 and 5601270.

📒 Files selected for processing (1)

• src/libstore/misc.cc

🚧 Files skipped from review as they are similar to previous changes (1)

• src/libstore/misc.cc

---

📝 Walkthrough

Walkthrough

The non-flipDirection branch of Store::computeFSClosure in src/libstore/misc.cc is refactored from an iterative while-loop coroutine into a recursive doPaths coroutine. It batches path lookups via queryPathInfos, maintains shared done/required sets, and schedules subsequent iterations through asio::co_spawn.

Changes

Closure Traversal Refactor

Layer / File(s)	Summary
Recursive batched doPaths coroutine src/libstore/misc.cc	The while-loop-based traversal is replaced with a doPaths coroutine that batches unique paths, queries queryPathInfos, updates out using shared done/required sets, computes next todo from references and derivation outputs/derivers, and recursively schedules itself via asio::co_spawn, with exceptions routed through nested spawn handlers.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

• DeterminateSystems/nix-src#514: Both PRs refactor the same Store::computeFSClosure coroutine-based path expansion and batching logic in src/libstore/misc.cc.

Suggested reviewers: cole-h

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly describes the main change in computeFSClosure: switching from a loop to recursion.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch compute-fs-closure-recursive-async

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/libstore/misc.cc`:
- Around line 93-101: The top-level asio::co_spawn completion handler in doPaths
currently assigns ex unconditionally, which can overwrite an exception
previously stored by detached child tasks. Update that root completion lambda to
mirror the existing guarded handlers by only setting ex when the passed
std::exception_ptr is non-null, so errors raised by doPaths and its spawned
children are preserved.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 725a35a7-9f7d-4b50-a0fd-e1d78ff1fd81

📥 Commits

Reviewing files that changed from the base of the PR and between 90f7dca and 7656214.

📒 Files selected for processing (1)

• src/libstore/misc.cc

[github-actions]

🚀 Deployed on https://6a4d4184e285cba724209174--determinate-nix.netlify.app