Cut startup cost, keep command arguments whole, tighten up conf and creds - #36
Merged
Merged
Conversation
keycmd runs in front of every command a user puts through it, so its own import time is latency paid on each invocation. Three imports cost more than the whole of the rest of the package, and none of them is needed on every run, so each moves into the function that needs it: - keyring, which is over half of the import time and drags in urllib, ssl, email and importlib.metadata behind it, is only reached for once there is a credential to look up - pprint, whose own cost is mostly dataclasses, is only reached for by the new vlog_pretty, under --verbose - subprocess is only reached for on the windows path, which cannot replace its own process; the posix path never spawns one That takes `import keycmd.cli` from 94ms to 27ms above bare interpreter startup. A run that does look up a credential still pays for keyring, as it must, and lands where it did before; --version, --help and a config without keys no longer pay for it at all. test_cli_import_stays_lean fails if one of the three wanders back up to module level. Alongside that, two bits of duplication: - load_conf walked the file system twice, once for .keycmd and once for pyproject.toml, over exactly the same ground. It now walks once, over a walk_up generator that replaces find_file and its two overloads, along with the first_only flag that made them necessary. - get_env spelled out the b64 and format handling, and the log line that reports it, once for keys and again for aliases, and carried both options through a five field KeyData tuple of which the alias path discarded two. A key and an alias differ in where the credential comes from, not in what happens to it on the way into the environment, which is now what expose_conf says. Verbose output is unchanged, save for one addition: which backend keyring settled on, which decides where the credentials came from and was not reported anywhere. The README example of verbose output had drifted from what keycmd prints, and is refreshed to match. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016numkAkNoMbJRUgVWy63g7
Every shell but cmd takes its command as a single string after -c, and run_cmd built that string by joining the arguments with spaces. The shell then split it back into words, so anything the first split had held together came apart: `keycmd mytool 'hello world'` reached mytool as two arguments, and an argument containing a quote, like `it's`, was a shell syntax error rather than an argument at all. The two ways of invoking keycmd want opposite things here, and argv is what tells them apart: - one argument is a command line already. `keycmd 'echo $SECRET'` is the form the README recommends, and every example in it uses, and the shell is there precisely to interpret it. It is handed over as typed, so variables, pipes and redirections all keep working. - several arguments are an argv vector. Each is quoted, so the shell reassembles exactly the words that were passed in. Quoting is per dialect: shlex for posix shells, and doubled quotes for powershell, which also reads a quoted command name as a string to print and so needs the call operator once its first word ends up quoted. cmd already kept its arguments separate after /C and is untouched. This is a behavior change for the multi-argument form, in that shell syntax in an argument is no longer interpreted a second time: `keycmd echo '$SECRET'` now prints the name rather than the value. The single argument form is the one the README documents, and is unaffected. Both are now spelled out under Usage. The two tests that reached for the multi-argument form to check that the environment arrives, in test_cli and test_shell, were relying on that second round of expansion, and now use the documented single argument form to test the same thing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016numkAkNoMbJRUgVWy63g7
test_run_cmd_quoting asserts the command line keycmd builds, which says what quote() produces but not whether the shell reads it back the way it was meant. It also only exercised a space and a quote, where the point of quoting is everything else a shell might be tempted to read as syntax. test_run_cmd_preserves_argv now runs nineteen argument shapes through every installed shell for real, and compares the argv that arrives with the argv that was passed: quoting of both kinds, expansions, command substitution, globs, command separators, redirections, tabs, newlines, the empty string, and one argument with the lot in it. cmd is handed its arguments separately rather than as a command line keycmd quoted, and what quotes them on the way is the windows runtime, which knows nothing of cmd's own metacharacters. The arguments carrying those are skipped for cmd, through a Shell.carries predicate alongside the other dialect differences in conftest, rather than asserted as though keycmd could deliver them. Also records what quote() assumes: everything that is not powershell is quoted the posix way, which holds for every shell the suite runs against, but not for csh, fish or nu, which shellingham can also detect and which spell quoting their own way. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016numkAkNoMbJRUgVWy63g7
pwsh was listed among the windows shells, so the suite only ever reached for it on windows, and the posix jobs ran three shells that all quote the same way. pwsh installs on every platform keycmd supports, and is the one shell taking -c that does not quote the way a posix shell does, which makes it the shell most worth running and the one that was least run. It moves to a candidate list that is appended on either platform. The fixture still gates on which(), so a machine without it loses the coverage rather than the run, and all three runners in CI ship it. That puts 24 tests on pwsh, the round trip battery among them, and they pass on 7.6.4: every argument shape survives, including the ones where powershell and posix disagree, and the call operator turns out to be load bearing rather than defensive, since without it pwsh answers a quoted command name with a ParserError. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016numkAkNoMbJRUgVWy63g7
The round trip battery ran in full against cmd and windows powershell and found five arguments neither can carry, all of them limits of the windows command line rather than of keycmd's quoting: - windows powershell passes arguments to a native command the way it always has, which drops an embedded double quote and an empty argument however they are written. Powershell 7.3 fixed this, and every pwsh case passed on the same runner, so pwsh stays held to the whole battery. - cmd reaches a command through a command line, and a command line is a line, so a newline in an argument ends it early. Nothing quoting can do on keycmd's side lifts either, and the escaping that works around the powershell one is version dependent in a way the shell name does not reveal: it would fix 5.1 and break the pwsh that currently passes. So Shell.carries states what each windows shell can deliver, next to the exit statuses and variable syntax it already states, and quote() records the limitation where the quoting is. The posix shells and pwsh are unchanged and still assert all nineteen. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016numkAkNoMbJRUgVWy63g7
Which shells a run covers depends on what is installed on the machine, and the only place that showed was the ids of the tests that failed. A run where they all pass could not be told apart from one where a shell was missing and quietly contributed nothing, which is exactly the question the windows failures raised: powershell failed five cases and pwsh failed none, and reading that as pwsh passing rather than pwsh being absent took inference from an absence. The header pytest already prints for the keyring backend now names the shells too, so every run says what it covered. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016numkAkNoMbJRUgVWy63g7
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
keycmd runs in front of every command a user puts through it, so its own import time is latency paid on each invocation. This moves the expensive imports off the startup path, fixes a quoting bug found while measuring, and cleans up two bits of duplication.
Verbose output is unchanged apart from one addition, verified by diffing a real end-to-end run against the pre-refactor code.
Startup time
import keycmd.cligoes from 94ms to 27ms above bare interpreter startup (keycmd --versionend to end: 107ms → 40ms). Three imports cost more than the whole of the rest of the package, and none of them is needed on every run, so each moves into the function that needs it:keyringurllib,ssl,email,importlib.metadata)pprintdataclasses)--verbose, through the newvlog_prettysubprocessA caveat on that number, so it is not read as more than it is: a run that does look up a credential still pays for
keyring, as it must, and lands within ~1ms of where it was.keyringtransitively importssubprocessanddataclassesanyway, so for those runs the change relocates the cost rather than removing it. The full win lands on--version,--help, and a config without keys.test_cli_import_stays_leanfails if one of the three wanders back up to module level.argparseis the largest remaining import at ~9ms, and is left alone deliberately: the README documents its--helpoutput verbatim, so hand rolling the parser would mean maintaining that text by hand and losing prefix abbreviation, for 9ms.The dominant per-invocation cost is keyring's backend discovery
Worth recording, because it dwarfs everything above. keyring works out which backend to use by loading every backend registered by every installed package, on each invocation:
PYTHON_KEYRING_BACKENDpinned: ~2mskeycmd should not override a user's keyring configuration, so this is a README note rather than a code change — a new "Startup time" section under the existing keyring backends heading.
Related,
--verbosenow reports which backend keyring settled on. That decides where the credentials came from and was not reported anywhere, which makes it the first thing worth knowing when the credentials are not the ones expected.Command arguments keep their word boundaries
Found while measuring, and fixed here. Every shell but
cmdtakes its command as one string after-c, andrun_cmdbuilt that string by joining the arguments with spaces. The shell then split it back into words, so whatever the first split had held together came apart:The two ways of invoking keycmd want opposite things here, and argv is what tells them apart:
keycmd 'echo $SECRET'is the form the README recommends and every example in it uses, and the shell is there precisely to interpret it, so it is handed over as typed. Variables, pipes and redirections keep working.Quoting is per dialect:
shlexfor posix shells, doubled quotes for powershell, which also reads a quoted command name as a string to print and so needs the call operator once its first word ends up quoted.cmdalready kept its arguments separate after/Cand is untouched.This is a behavior change for the multi-argument form, in that shell syntax in an argument is no longer interpreted a second time:
keycmd echo '$SECRET'now prints the name rather than the value. The single argument form the README documents is unaffected. Both forms are now spelled out under Usage. Two existing tests reached for the multi-argument form to check that the environment arrives and were relying on that second round of expansion; they now use the documented single argument form to test the same thing.DRY and reorganization
conf.py—load_confwalked the file system twice, once for.keycmdand once forpyproject.toml, over exactly the same ground. It now walks once, over awalk_upgenerator that replacesfind_filealong with its two@overloads and thefirst_onlyflag that made them necessary.creds.py—get_envspelled out the b64 and format handling, and the log line reporting it, once for keys and again for aliases, and carried both options through a five fieldKeyDatatuple of which the alias path unpacked two as_, _. A key and an alias differ in where the credential comes from, not in what happens to it on the way into the environment, which is now whatexpose_confsays over a three field tuple.logs.py—vlogre-implementedlog's prefix instead of calling it.Subprocess layers: investigated, not changed
run_cmdbuilds[shell, "-c", "command line"], so a posix run is keycmd, replaced by the shell, replaced by the command. Skipping the shell when no argument carries shell syntax is feasible, but the layer measures at ~1.1ms, and it carries real hazards: shell builtins that shadow binaries with different flags (echo -ediffers between bash's builtin and macOS/bin/echo), functions sourced from.zshenv, and on windows thecmdonly builtins and.batfiles. Not worth ~1ms against the ~67ms the import work already returns, so the shell stays.Testing
ruff check,ruff formatandty check --error-on-warningall clean. The suite is 92 passed / 3 skipped with the keyring paths exercised (PYTHON_KEYRING_BACKEND=keyrings.alt.file.PlaintextKeyring), up from 68 / 14 on master.New tests: the import guard;
test_run_cmd_quoting, which asserts the command line built for bash, powershell and cmd alike, so the dialects that are not installed locally are covered the way the rest of the suite covers them; andtest_run_cmd_preserves_argv, which runs a real command through every installed shell and checks the argv that arrives. Thefind_filetests becomewalk_uptests over the same three stopping conditions.The README's example of verbose output had drifted from what keycmd actually prints (
belonging to userrather thanwith user, and noformat:field), and is refreshed to match.🤖 Generated with Claude Code
https://claude.ai/code/session_016numkAkNoMbJRUgVWy63g7