Skip to content

multicall: add the busybox-style single binary, for 8 tools - #264

Open
metif12 wants to merge 8 commits into
vlang:mainfrom
metif12:multicall-single-binary
Open

metif12 wants to merge 8 commits into
vlang:mainfrom
metif12:multicall-single-binary

Conversation

@metif12

@metif12 metif12 commented Oct 17, 2026

Copy link
Copy Markdown
Contributor

The other build style: one binary containing several tools, dispatched by
name, alongside the per-tool binaries build.vsh already emits.

What this is

src/multicall/main.v holds a map from tool name to that tool's run
function. It dispatches on argv[0] first, so a hard link or symlink named
after a tool runs it with no sub-command, the way busybox does. A tool name
as argv[1] works too, which is how the earlier coreutils dispatcher was
invoked. build.vsh compiles the multicall and hard links each converted
tool's name to it, so bin/pwd and bin/multicall are one executable. That
is what GNU's --enable-single-binary=hardlinks does.

A tool joins by being converted from module main with its own fn main()
to a module exposing pub fn run(args []string) int. That conversion is the
part worth doing: both build styles then call the same function, so they
cannot drift apart, which is the usual way a multicall rots — the single
binary quietly implements something the per-tool binaries do not.

Eight of 83 tools so far

arch, hostid, logname, pwd, true, tty, whoami, yes. The
remaining tools need the same mechanical conversion, and the list of what is
left is the tools map in src/multicall/main.v against bin/.

false is deliberately excluded: V rejects module false because false is
a keyword and a module name must match its directory, so the tool cannot be
imported into a multicall without renaming its directory. bin/false still
builds on its own and is unaffected.

Two bugs fixed on the way

whoami did not compile. The conversion added module whoami to
src/whoami/lib_default.c.v but deleted the body of whoami() with it,
leaving fn whoami() !string { uid := os.geteuid() and no closing brace:

src/whoami/lib_default.c.v:15:1: error: unexpected eof, expecting `}`

lib_windows.c.v, the other variant, kept its body, so this only showed up
off Windows. Restored; the module whoami header is now the only difference
from main.

argv[0] dispatch never matched on Windows. os.base(os.executable())
carries the extension, and the map is keyed on bare names:

$ copy multicall.exe pwd.exe && pwd.exe
pwd.exe: no tool given, and pwd.exe is not itself a tool

Every per-tool name missed. Dispatch on argv[1] was fine, which made it look
like a missing tool rather than a matching bug. Now stripped before the
lookup.

Verified

All eight converted tools compile, and v fmt -verify is clean on all 17
files. Dispatch, with the binary copied to each name:

true.exe            exit 0, no output          (argv[0])
multicall.exe true  exit 0                    (argv[1])
notatool.exe        'no tool given, and notatool.exe is not itself a tool'
                    'available: arch, hostid, logname, pwd, true, tty, whoami, yes'

Not verified, and one known gap

  • The full build.vsh run, and the hard-link layout it produces. I built the
    multicall and the tools directly instead.
  • Anything on Unix. The .exe strip is a no-op there, but I have no Unix to
    check that on.
  • src/arch and src/whoami tests. Both already fail on pristine main,
    identically, so this is not a regression — but the conversion makes them
    stay failing in a new way. common/testing's rig builds a tool's
    directory as a standalone binary, and a converted tool is module arch
    rather than module main, so there is no main module to build. When those
    two suites are fixed, the rig will need to build the multicall instead. The
    other six converted tools have no tests, so nothing else is affected yet.
    Every remaining tool converted this way will hit it, so it is worth fixing
    before the remaining 75, not after.
  • CI is red for every PR in this repo because V's own bootstrap fails before
    any of this compiles; see tcc: error: unresolved reference to 'remainder' when compiling v_stage.exe v#30551, #30598 and #30753. So nothing here
    has a green signal behind it, and the checks above were run by hand.

GNU style already works: build.vsh emits one binary per tool into bin/. What it
does not do is the other style - one binary containing several tools, dispatched
by name. This starts it.

src/multicall/main.v holds a tools map from name to the tool module's run
function, and dispatches on argv[0] first, so a hard link or symlink named after
a tool works with no sub-command, as busybox does. A tool name as argv[1] also
works, which is how the earlier coreutils dispatcher was invoked.

A tool joins the multicall by being converted from module main with its own
`fn main()` to a module exposing `pub fn run(args []string) int`. That conversion
is the point: both build styles then call the same function, so they cannot drift
apart, which is the usual way a multicall rots - the single binary quietly
implements something the per-tool binaries do not.

build.vsh compiles the multicall and hard links each converted tool's name to
it, so bin/true and bin/coreutils are one executable. This is what GNU's
--enable-single-binary=hardlinks does and what busybox ships.

`false` is deliberately excluded. V rejects module false because false is a
keyword and a module name must match its directory, so the tool cannot be
imported into a multicall without renaming its directory. bin/false still builds
on its own and is unaffected.

Verified directly: the multicall compiles, and `arch` prints AMD64, whoami
prints the user, hostid reaches its platform stub, and an unknown name lists
what is available. Not verified here: the full build.vsh run, because src/touch
fails to compile on this machine's V with C.SetFileTime already declared - and
it fails the same way on pristine main, so that breakage is not from this change.
The src/arch and src/whoami test suites also fail here, and also fail identically
on pristine main. CI on Linux is the path that exercises both.

This is a vertical slice of 4 of the 83 tools. The remaining 79 need the same
mechanical conversion, and the list of what is left is src/multicall/main.v's
tools map against bin/.

(cherry picked from commit b949342662de311edd49ae4ea3d32704e08e20cf)
pwd had no exit() calls and no app.quit, so its conversion is mechanical: module
main to module pwd, os.args to the args parameter, fn main to pub fn run returning
int. That is the shape every remaining tool needs, and the ones that do not look
like this are the ones using common.CoreutilInfo.quit, which is @[noreturn] and
so cannot simply become a return.

(cherry picked from commit 5373e30761509849294a31dd3d5cd69d69b55a81)
tty is the first tool that cannot be converted mechanically. It routes its
"extra operand" failure through common.CoreutilInfo.quit, which is @[noreturn], so
inside the multicall binary that exit would end the whole process and take every
other tool with it. There is no way to turn a noreturn into a return at the call
site.

So common gains quit_status, which reports the identical message and advice but
returns the status. It is additive: quit stays as it is, and the 47 tools still
calling it are untouched. Only a converted tool calls quit_status, and it does so
as 
eturn app.quit_status(...).

The other wrinkle is that tty's args() function both parsed arguments and could
terminate the process, so it now returns a pair: the status from a failed parse,
and the settings from a successful one. parse(args) returning (int, Settings) is
the shape any tool that both parses and can fail will need.

tty_default.c.v and tty_nix.c.v carry module tty now, the same as every other
file in a converted tool's directory: V requires one module per directory, and a
file with no module declaration defaults to main, which is what made the multicall
reject it with "bad module definition".

bin/tty is now a hard link to bin/coreutils rather than its own executable, which
is why build.vsh lists tty in multicall_tools. Verified both ways: the multicall
dispatches 	ty and prints "not a tty" with status 1 when standard input is not a
terminal, and an unknown name still lists the six available tools.

6 of 83 converted. Remaining on this path: the 46 other tools using app.quit(),
each of which needs the same quit_status treatment, and the ones using a bare
exit() need their returns threaded through the same way pwd and tty show.

(cherry picked from commit 836639c668f7d9428078b1080da3a0d2437a00ca)
yes is straightforward once the pattern exists: module main to module yes,
os.args to the parameter, and the error block inside fp.finalize() returns 1
instead of exiting. Its write loop never terminates, so the trailing return is
unreachable and is there only because the multicall wants a status from every
tool. Verified through the binary: coreutils yes multicall-works repeats the
string until killed.

Two tools are now ruled out with the reason recorded next to false, rather than
left for someone to rediscover:

false, because V rejects module false - false is a keyword and a module name
must match its directory. An alias cannot fix that either.

sync, because vlib ships a module named sync for Mutex, RwMutex and WaitGroup, and
V resolves import sync to the standard library before it looks in this tree. I
tried import sync as sync_tool and the alias still resolved to vlib/sync, which
is how it was found: the error named a module with no 
un. A tool whose name
collides with a stdlib module cannot join the multicall without renaming its
directory, so bin/sync keeps its own binary.

The generalisation from those two: a tool can only join the multicall if its name
is neither a V keyword nor an existing vlib module. That is a property of the
name, not of the tool, so it can be checked once per tool instead of discovered
by a failed build.

7 of 83 converted. The remaining ones on this path are the ones using app.quit(),
which now have quit_status to call, plus the ones whose names have to clear the
same two checks first.

(cherry picked from commit eb6f485a2fd38640e7442bbb200eccf0421adb61)
I had reported that fmt, test, base32 and base64 were also excluded, on the
strength of comparing our tool names against vlib directory names. That was wrong
and this corrects it.

The check that matters is not "is there a directory by this name in vlib" but
"does import <name> resolve to vlib". Tested each one:

  fmt, test, base32, base64   resolved to our own module. The error each one
                              produced was "is defined as module main", the same
                              hostid gave before its platform files carried the
                              module name - which means V found us and the tool
                              simply is not converted yet.

  sync                       resolved to vlib. import sync as probe_sync
                              compiled with nothing worse than an unused-import
                              warning, which is a different failure from the
                              other four and is what settles it.

So the real exclusion list is three tools: true and false, because they are V
keywords and a module name must match its directory, and sync, because vlib owns
that name. Everything else can be converted.

This matters for the remaining count: 80 of 83 are reachable, not 79, and the
blocker for each is the mechanical conversion rather than its name.

(cherry picked from commit 08f7bb05e167d8bff6595c9c10ff1ab5b867e1bd)
logname is the first tool that carried its own error_exit/success_exit helpers
rather than using common, so those go away and the two paths return a status
instead. It is also the first conversion to surface a real bug.

flags_common takes an args parameter, documented as "should usually be os.args",
and then ignored it - it built the flag parser from os.args directly. That is
harmless while the tool is only ever its own binary, and wrong the moment anything
calls it with a different argv, which is precisely what the multicall does. The
parser now uses the argument it was given. The comment says why, because the
signature already claimed this and reading it would not have revealed that the
body disagreed.

Verified through the binary: coreutils logname prints the login name, and an
unknown name still lists the eight available tools.

8 of 83 converted, 80 reachable once the three excluded names - true, false and
sync - are accounted for.
416742a truncated src/whoami/lib_default.c.v mid-function. It deleted the
uid check, the getpwuid call, the return, and the closing brace, leaving

    fn whoami() !string {
        uid := os.geteuid()

which does not parse:

    src/whoami/lib_default.c.v:15:1: error: unexpected eof, expecting `}`

so the non-Windows variant of whoami could not be compiled. The conversion
added `module whoami` to the top of the file, which it needed, but the body
went with it. lib_windows.c.v, the other variant, kept its body.

Restoring the body leaves the header as the only difference from main, which
is what the conversion actually needed. Verified: the file parses, and the
whoami module compiles.

Worth noting against the commit message of 416742a, which claims "whoami
prints the user" was verified. It could not have been on this variant, since
the variant did not compile; that check can only have run against the
Windows one.
Dispatch on argv[0] is the point of this layout - bin/pwd is a hard link to
the multicall and runs pwd with no sub-command, as busybox does. It did not
work on Windows at all:

    $ copy multicall.exe pwd.exe
    $ pwd.exe
    pwd.exe: no tool given, and pwd.exe is not itself a tool

os.base(os.executable()) returns the last component of the path, which on
Windows carries the extension, and the tools map is keyed on bare names, so
`pwd.exe` never matched `pwd`. Every per-tool name missed. Dispatch on
argv[1] was unaffected, which is why the failure looked like a missing tool
rather than a matching bug.

Now strip a trailing .exe before the lookup. The check is case-insensitive
because the filesystem may hand back `PWD.EXE`, and it is length-guarded by
the ends_with test so a name that is exactly `.exe` cannot slice to a
negative index.

Verified on Windows, with the binary copied to each name:

    true.exe            exit 0, no output        (dispatched to true)
    notatool.exe        'no tool given, and notatool.exe is not itself a tool'
    multicall.exe true  exit 0                  (argv[1] path still works)

Not verified on Unix, where os.base never carries an extension and the
strip is a no-op.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant