Repository navigation
tine: run buck2 without a persistent daemon - #289
Merged
Merged
Conversation
daandemeyer
reviewed
Oct 10, 2026
daandemeyer
reviewed
Oct 10, 2026
daandemeyer
reviewed
Oct 10, 2026
| return f"{digest} {namespace}" | ||
| import json | ||
|
|
||
| return json.dumps({"dev": sorted(targets), "gitdirs": gitdirs}, sort_keys=True) |
Member
There was a problem hiding this comment.
It's an identity, shouldn't we just hash it?
Member
Author
There was a problem hiding this comment.
mount_namespace_marker() is a confusing name after this rework. It's not just a compared identity, it actually transports the mount table into the sandbox for namespace_mount_targets() (.buck/tine-mounts.toml may have changed at this point) and namespace_gitdirs(). I renamed it to {en,de}code_mount_table().
daandemeyer
approved these changes
Oct 10, 2026
The persistent buck2d is a bit annoying: - It reads its configuration once at startup, drifts from the mounts and cache address of later invocations, and outlives the command that started it. - It caused and still causes us a lot of trouble with inotify; which doesn't pay off at all at the scale we are currently operating on (≤ thousands, not millions of files). - It gets confused when mixing buck commands on the host and in a container. Move to running buck daemon-less, with `--no-buckd`. Our pinned buck2 fork has enough fixes to work correctly with that. That will let us drop our notify-rs fork, our buck2 fork's inotify patches, and quite a lot of tine.py magic (in the following commits). Signed-off-by: Martin Pitt <martin@amutable.com>
A resident daemon read the cache shim's address once at startup, so a changed address had to kill and restart it, and every command first compared the published address against the configuration to decide that. Daemon-less, each command's own daemon reads the current configuration, so writing the new address into the generated block needs no further ceremony. Signed-off-by: Martin Pitt <martin@amutable.com>
The private tmpfs config fragment existed so that Buck would hash the mount digest into its daemon constraints and replace a daemon whose mounts no longer matched. Without a resident daemon the only readers are tine's own re-entries and nested invocations. Environment is inherited/propagated between processes exactly like namespace membership, so this can be simplified a lot now: Carry the mounted paths and Git directories in `$TINE_MOUNTS` and drop the fragment, the tmpfs, the mount digest and the config-key reservations. `read_generated_buckconfig()` is not used in production any more, move it to the tests that assert on generated blocks. Signed-off-by: Martin Pitt <martin@amutable.com>
With latest buck2 (facebook/buck2#1557), daemon-less invocations serialize per isolation dir. That would make a completing shell queue behind a running build, and hang the tab keypress; before the serializing, the new buck2 would have killed the running build. Give completion its own an isolation dir to avoid that. Measured: against a live buck run, the completion invocation took 0.46s. Signed-off-by: Martin Pitt <martin@amutable.com>
Member
Author
|
@daandemeyer your previous approval still sticks, but it changed (well, simplified) quite a bit now. |
martinpitt
marked this pull request as ready for review
October 10, 2026 20:08
daandemeyer
approved these changes
Oct 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The persistent buck2d is a bit annoying:
It reads its configuration once at startup, drifts from the mounts and
cache address of later invocations, and outlives the command that
started it.
It caused and still causes us a lot of trouble with inotify; which
doesn't pay off at all at the scale we are currently operating on
(≤ thousands, not millions of files).
It gets confused when mixing buck commands on the host and in a
container.
Move to running buck daemon-less , i.e. pass
--no-buckdeverywhere.Our pinned buck2 fork has enough fixes to work correctly with that. That
will let us drop our notify-rs fork, our buck2 fork's inotify patches,
and quite a lot of tine.py magic (in the following commits).
Signed-off-by: Martin Pitt martin@amutable.com
Plus some prerequisites and tine.py cleanup for stuff that we don't need any more without the daemon.