Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ The README is a landing page and nothing more: what it says about behaviour it s

## Architecture

`cli.main` wires the three halves together: `load_conf` produces the configuration, `get_env` turns it into an environment, and `run_cmd`/`run_shell` hand that environment to a shell. `--detect-backend` and `--reset-backend` return before any of it, since neither has a use for a configuration or a command. Errors reach the user through `logs.error`, which exits with status 1 and takes the hint lines that go under the error with it; `logs.vlog` output only appears under `--verbose` and is the first thing to reach for when debugging a configuration.
`cli.main` wires the three halves together: `load_conf` produces the configuration, `get_env` turns it into an environment, and `run_cmd`/`run_shell` hand that environment to a shell. `--detect-backend` and `--reset-backend` return before any of it, since neither has a use for a configuration or a command. The command is an argparse `REMAINDER`, so everything from its first word onwards reaches it verbatim — including options keycmd has of its own — which leaves `end_of_options` to strip the `--` that argparse keeps in place, since only the first one is keycmd's to remove. Errors reach the user through `logs.error`, which exits with status 1 and takes the hint lines that go under the error with it; `logs.vlog` output only appears under `--verbose` and is the first thing to reach for when debugging a configuration.

**`conf.py` — where the configuration comes from.** Later sources win, merged deeply by `merge_conf`: defaults, then `~/.keycmd`, then every `.keycmd` found walking up from the working directory (outermost first), then the first `pyproject.toml` found walking up, whose `[tool.keycmd]` table is used. Both searches cover the same ground, so `load_conf` collects them in a single pass over `walk_up`, which stops at a `.git` directory, at the home folder, and at the root of the file system, so the walk never escapes a repository. `USERPROFILE` is a module attribute so tests can point the user config elsewhere. The merged result is `cast` to `Conf` rather than validated: it is user authored, and `get_env` reports violations as user errors.

Expand All @@ -54,7 +54,7 @@ The README is a landing page and nothing more: what it says about behaviour it s

**`wsl.py` — the boundary between WSL and Windows.** WSL users install keycmd on Windows, which leaves it a Windows process with a Windows idea of a shell. `from_wsl` decides whether it was called from a distro — a `wsl.exe`/`wslhost.exe` ancestor decides it, a Windows shell found first decides against it, and a UNC working directory settles the rest — after which `run_shell`/`run_cmd` hand the work to `wsl.exe` rather than to a Windows shell. A UNC working directory also names the distro, which `wsl_argv` passes as `--distribution` so that a second distro does not send the command to the default one. `KEYCMD_WSL` overrides that decision in either direction. Neither side of the boundary inherits the other's environment, so `share_env` lists the exposed variables in `WSLENV`, which is the only thing that crosses.

**`shell.py` — where the platform differences live.** `get_shell` asks shellingham which shell invoked the process and falls back to `$SHELL` or `%COMSPEC%`. `cmd` takes `/C` and keeps the command's arguments separate; every other shell takes `-c` and the single string `join_cmd` builds. One argument is already a command line — the form the README recommends — and is handed over as typed, so the shell interprets it; several arguments are an argv vector, and `quote` protects each so that the shell does not split them into words a second time. `quote` reaches for `shlex` for posix shells and doubles the quote for powershell, which also needs the call operator once its command name ends up quoted. `exec` replaces the process with `execvpe` on posix, but runs a subprocess on Windows, which has no equivalent; `USE_SUBPROCESS` and the `IS_WINDOWS`/`IS_POSIX` flags are module attributes so tests can drive both paths on either platform.
**`shell.py` — where the platform differences live.** `get_shell` asks shellingham which shell invoked the process and falls back to `$SHELL` or `%COMSPEC%`. `cmd` takes `/C` and keeps the command's arguments separate; every other shell takes `-c` and the single string `join_cmd` builds. Several arguments are an argv vector — the form the docs lead with, `keycmd npm install` — and `quote` protects each so that the shell does not split them into words a second time; one argument is already a command line, and is handed over as typed, so the shell interprets it. `quote` reaches for `shlex` for posix shells and doubles the quote for powershell, which also needs the call operator once its command name ends up quoted. `exec` replaces the process with `execvpe` on posix, but runs a subprocess on Windows, which has no equivalent; `USE_SUBPROCESS` and the `IS_WINDOWS`/`IS_POSIX` flags are module attributes so tests can drive both paths on either platform.

## Conventions

Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,10 @@ Store a credential in your OS keyring, name it in a `.keycmd` file:
OPENAI_API_KEY = { credential = "my-openai-token", username = "your-username" }
```

...and run anything that needs it:
...and run anything that needs it, by putting `keycmd` in front of the command you were going to run anyway:

```bash
keycmd 'python my_openai_script.py'
keycmd python my_openai_script.py
```

The variable exists inside that command, and nowhere else — no `.env` file, no secret pasted into your terminal, nothing left behind afterwards. 😱 → 😌
Expand All @@ -45,7 +45,7 @@ Continue with the [Quick Start tutorial](https://korijn.github.io/keycmd/getting
Everything lives at **[korijn.github.io/keycmd](https://korijn.github.io/keycmd)**:

* [Installation](https://korijn.github.io/keycmd/getting-started/installation/) — globally, under pyenv, or from WSL
* [Running commands](https://korijn.github.io/keycmd/guide/running-commands/) — the two invocation forms, quoting, subshells
* [Running commands](https://korijn.github.io/keycmd/guide/running-commands/) — prefixing a command, quoting one, subshells
* [Configuration](https://korijn.github.io/keycmd/guide/configuration/) — where it lives, keys, format strings, aliases
* [Keyring backends](https://korijn.github.io/keycmd/guide/keyring-backends/) — third party backends, and keycmd's startup time
* [WSL](https://korijn.github.io/keycmd/guide/wsl/) — reaching the Windows Credential Manager from a distribution
Expand Down
4 changes: 2 additions & 2 deletions docs/examples/azure-artifacts.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ always-auth=true
Now I can set up my `node_modules` just by calling:

```bash
keycmd 'npm install'
keycmd npm install
```

🚀
Expand All @@ -116,4 +116,4 @@ secrets:
environment: PAT_B64
```

When I call `keycmd 'docker compose build'` these two variables are exposed by keycmd and subsequently they are available as [docker compose build secrets](https://docs.docker.com/compose/use-secrets/). 👌
When I call `keycmd docker compose build` these two variables are exposed by keycmd and subsequently they are available as [docker compose build secrets](https://docs.docker.com/compose/use-secrets/). 👌
4 changes: 2 additions & 2 deletions docs/examples/openai.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,13 @@ OPENAI_API_KEY = { credential = "my-openai-token", username = "your-username" }
Now you can run any OpenAI script by just prefixing your command with `keycmd`:

```bash
keycmd 'python my_openai_script.py'
keycmd python my_openai_script.py
```

Or a Jupyter notebook:

```bash
keycmd 'jupyter notebook'
keycmd jupyter notebook
```

That's all! 🤘 Now you can rest easily, knowing your tokens are safe. 🛌💤
Expand Down
16 changes: 13 additions & 3 deletions docs/getting-started/quick-start.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,17 @@ This says: look up the credential `my-secret` for user `my-username`, and expose

## 3. Run a command

Open a terminal and run a command that prints the secret. That looks different depending on the shell you use:
Open a terminal and put `keycmd` in front of the command you want the secret to reach. That is all there is to it, and it is how you will use keycmd for real:

```bash
keycmd npm install
keycmd docker compose up
keycmd pytest
```

The tools you run read their credentials from the environment themselves, and keycmd is what puts them there.

To check your setup right now, print the secret instead. That is the one thing that *does* need quotes, since a variable written in your command line is expanded by your own shell — before keycmd has set it:

=== "bash / zsh"

Expand Down Expand Up @@ -78,11 +88,11 @@ You've successfully set up keycmd! 👏

keycmd read your configuration, looked `my-secret` up in your OS keyring, put the password in the environment as `SECRET`, and handed that environment to your shell along with your command. When the command finished, the variable went with it: your own shell never had it.

Note the quotes in the bash and PowerShell examples. Quoting the whole command as one argument is what lets *your command's shell* expand `$SECRET`, rather than your own shell expanding it before keycmd ever sees it. See [Running commands](../guide/running-commands.md) for the details.
Note the quotes in the bash and PowerShell examples. Quoting the whole command as one argument is what lets *your command's shell* expand `$SECRET`, rather than your own shell expanding it into nothing beforehand. (`cmd` is the exception: it leaves an undefined `%SECRET%` alone, so it survives the trip unquoted.) Commands that read the environment themselves — which is nearly all of them — need none of this. See [Running commands](../guide/running-commands.md) for the details.

## Where to go next

* [Running commands](../guide/running-commands.md) — the two ways to invoke keycmd, quoting, and subshells.
* [Running commands](../guide/running-commands.md) — prefixing a command, quoting one, and subshells.
* [Configuration](../guide/configuration.md) — where configuration lives, and everything you can put in it.
* [Examples](../examples/openai.md) — an OpenAI API key, and a real world setup where poetry, npm and docker compose share a single Azure DevOps token.
* [Troubleshooting](../guide/troubleshooting.md) — if any of the above did not go as planned, `keycmd --verbose` will tell you why.
2 changes: 1 addition & 1 deletion docs/guide/keyring-backends.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ keycmd: PYTHON_KEYRING_BACKEND=keyring.backends.SecretService.Keyring already na
If keyring finds no backend it can use, there is nowhere for keycmd to read credentials from, and it says so rather than failing on the first lookup:

```
❯ keycmd 'npm install'
❯ keycmd npm install
keycmd: error: keyring has no backend to read credentials from
keycmd: hint: install one for this platform, or name one you have with PYTHON_KEYRING_BACKEND
keycmd: hint: see https://github.com/jaraco/keyring#third-party-backends
Expand Down
60 changes: 46 additions & 14 deletions docs/guide/running-commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,42 +3,74 @@
There are two ways to use keycmd:

```bash
keycmd 'your command' # run one command with the credentials exposed
keycmd your command # run one command with the credentials exposed
keycmd --shell # open a subshell with the credentials exposed
```

The first is the preferred one, since your secrets are only exposed as environment variables for the duration of a single command. The second is less preferable, but can be convenient when you are debugging a process that depends on the credentials you are exposing.

## One command
## Prefix your command

In its most common form, keycmd takes the command to run as a single quoted argument:
Write the command the way you would have written it anyway, and put `keycmd` in front of it:

```bash
keycmd 'npm install'
keycmd npm install
keycmd docker compose up -d
keycmd pytest -k auth --maxfail 1
```

Quoting the whole command as one argument is what lets you use your shell's syntax inside it:
That is the whole idea, and it is all most commands need: the tool you are running reads its credential from the environment itself, and keycmd is what puts it there.

Everything after `keycmd`'s own options belongs to your command, dashes and all. `keycmd pytest --verbose` runs pytest verbosely; it is `keycmd --verbose pytest` that makes *keycmd* verbose.

Each argument is passed on as the word it was, so nothing you typed is split or expanded a second time:

```bash
keycmd 'echo $SECRET | tr a-z A-Z'
# arrives as a single argument, spaces and all
keycmd mytool --message 'hello world'
```

keycmd hands that line to your shell exactly as you typed it, and your shell does the rest — pipes, redirects, variable expansion and all. This matters especially for the credentials themselves: `$SECRET` has to be expanded by the shell keycmd starts, because that is the only shell the variable exists in.
The quotes there are your own shell's, doing their usual job. keycmd re-quotes each argument for the shell it hands the command to, so what `mytool` receives is the argv your shell built.

!!! note "Two limits on Windows"

## Separate arguments
Both are the platform's rather than keycmd's, and no amount of quoting lifts either. `cmd` reaches a command through the Windows command line, which cannot hold a newline and which expands `%VAR%` inside an argument. Windows PowerShell drops an embedded `"` and an empty argument when it calls a native command; `pwsh` (PowerShell 7.3 and up) does not.

You can also write the command out as separate arguments, and then keycmd keeps them separate:
## Ending keycmd's options with `--`

A command whose *first* word starts with a dash would be read as an option of keycmd's. Put `--` in front of it to say that keycmd's own options have ended:

```bash
# arrives as a single argument, spaces and all
keycmd mytool --message 'hello world'
keycmd -- --my-oddly-named-tool
```

Since each argument is passed on as the word it was, your shell's syntax is *not* interpreted a second time in this form. If you want `$SECRET` expanded, either let your own shell expand it, or use the single argument form above.
`--` is also simply a habit worth keeping, since every tool that goes on to run another one takes it:

```bash
keycmd -- npm install
```

Only the `--` that ends keycmd's options is removed; any further `--` is your command's own and is passed along untouched.

## A quoted command line

Written as a single quoted argument, the command is handed to your shell as typed, and the shell interprets it — pipes, redirects, globs, `&&` and variable expansion included:

```bash
keycmd 'echo $SECRET | tr a-z A-Z'
keycmd 'npm ci && npm run build'
```

This matters especially for the credentials themselves. `$SECRET` has to be expanded by the shell keycmd starts, because that is the only shell the variable exists in:

```bash
keycmd echo $SECRET # your own shell expands it, before keycmd sets it — empty
keycmd 'echo $SECRET' # the shell keycmd starts expands it — correct
```

!!! tip "Which form should I use?"

Use the quoted form for anything that needs a shell: pipes, `&&`, redirects, globs, and above all the credentials you came here for. Use separate arguments when you are passing along text that must survive untouched, such as an argument that itself contains `$` or quotes.
Prefix your command as you normally write it, and reach for quotes when you need a shell: pipes, `&&`, redirects, globs, and above all a credential you want expanded into the command line rather than read from the environment.

## A subshell

Expand All @@ -52,7 +84,7 @@ Every command you run in it has the credentials available, until you exit it. Ke

## Which shell keycmd uses

keycmd asks [shellingham](https://github.com/sarugaku/shellingham) which shell invoked it, and falls back on `$SHELL` on posix or `%COMSPEC%` on Windows if that fails. In other words, it runs your command in the shell you were already using.
keycmd asks [shellingham](https://github.com/sarugaku/shellingham) which shell invoked it, and falls back on `$SHELL` on posix or `%COMSPEC%` on Windows if that fails. In other words, it runs your command in the shell you were already using — in both forms, which is what lets `keycmd npm install` find the `npm.cmd` on your Windows `PATH`.

On posix, keycmd replaces its own process with the shell (`execvpe`), so it does not sit in the process tree waiting around. Windows has no equivalent, so there keycmd runs the shell as a subprocess and passes its exit code along.

Expand Down
12 changes: 10 additions & 2 deletions docs/guide/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ That output answers, in order, the four questions a misbehaving run usually come
* **Which backend answered?** See [keyring backends](keyring-backends.md).
* **What was actually run?** Including the shell, and the exact argument vector handed to it.

The example above uses `cmd.exe`; in bash or PowerShell the command would be quoted as one argument, as in `keycmd -v 'echo $ARTIFACTS_TOKEN_B64'`.
The example above echoes a variable, which is why the command is written out for `cmd.exe` rather than prefixed the usual way; in bash or PowerShell it would be quoted as one argument, as in `keycmd -v 'echo $ARTIFACTS_TOKEN_B64'`. See [running commands](running-commands.md).

## Common problems

Expand All @@ -48,7 +48,15 @@ keycmd echo $SECRET # your shell expands $SECRET — before keycmd sets it
keycmd 'echo $SECRET' # the shell keycmd starts expands it — correct
```

Quote the whole command, so that the shell keycmd starts is the one interpreting it. See [running commands](running-commands.md).
Quote the whole command, so that the shell keycmd starts is the one interpreting it. This only comes up when you write the credential into the command line yourself; a tool that reads it from its own environment needs nothing but `keycmd` in front of it. See [running commands](running-commands.md).

### `sh: --: invalid option`, or `--: command not found`

Your keycmd is old enough to pass a leading `--` on to the shell as the first word of the command. Upgrade, or leave the `--` out — `keycmd npm install` works on every version.

### keycmd took my command's `--verbose` (or `--version`, or `-v`)

Only the options *before* your command are keycmd's; everything from the first word of the command onwards is passed on untouched. `keycmd --verbose pytest` makes keycmd verbose, `keycmd pytest --verbose` makes pytest verbose. If the command's own name starts with a dash, put `--` in front of it.

### `keycmd: error: keyring has no backend to read credentials from`

Expand Down
8 changes: 6 additions & 2 deletions docs/guide/wsl.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,10 @@ keycmd works out where it was called from by looking at its own process tree and

If you have more than one distribution installed, and you are working somewhere on the distribution's own file system, that working directory also names the distribution, and keycmd passes it to `wsl.exe` as `--distribution`, so your command goes to the distribution you are in rather than the default one.

!!! warning "Quotes do not survive the crossing"

`wsl.exe` strips the quotes from its own command line before the distribution's shell ever sees it, so an argument containing spaces arrives as several words no matter how it is written. `keycmd npm install` is unaffected, and so is anything else without spaces inside an argument; `keycmd mytool --message 'hello world'` is not, and there is nothing keycmd can do about it from the Windows side.

## Your credentials have to be told to cross

Your credentials do not come along by themselves, since neither side of the WSL boundary inherits the other's environment. Only the variables listed in [`WSLENV`](https://devblogs.microsoft.com/commandline/share-environment-vars-between-wsl-and-windows/) make the trip, so keycmd adds the variables from your configuration to it. Anything you had already listed in `WSLENV` yourself is kept.
Expand All @@ -39,8 +43,8 @@ Your credentials do not come along by themselves, since neither side of the WSL
Set the `KEYCMD_WSL` environment variable to override the decision in either direction:

```bash
KEYCMD_WSL=0 keycmd 'echo $SECRET' # stay on the windows side
KEYCMD_WSL=1 keycmd 'echo $SECRET' # go through wsl.exe regardless
KEYCMD_WSL=0 keycmd npm install # stay on the windows side
KEYCMD_WSL=1 keycmd npm install # go through wsl.exe regardless
```

`keycmd --verbose` reports which way it went, and what it based that on:
Expand Down
Loading