Default Git Bash on Windows and load embeddings without Hugging Face

[TeleporterGuy]

Summary

• Windows run_command now prefers Git Bash when bash.exe is on disk, and falls back to cmd.exe if it is not (still switchable in Settings).
• The default embedder is shipped in-repo as gzip parts under vendor/embed-models/ (under GitHub's 50 MB warning). Stem unpacks them into the local cache on first launch and does not call Hugging Face.
• npm run vendor:embed-models rebuilds those packs from a machine that already has a Stem cache. The default reranker is still too large for git and downloads from the Hub when the network allows it.
• collectBlocks in the HTTP transport tests stops at the requested frame count, so a single TCP read that carries pushTo and a broadcast no longer fails CI.

Test plan

• Fresh Windows profile with Git for Windows installed: Settings shows Git Bash, ls auto-runs, cat 'a & whoami' does not run whoami.
• Windows without Git: commands run under cmd.exe (dir / type) with no spawn error.
• Explicitly pick Command Prompt in Settings and confirm it sticks across restart.
• Clone with no Hugging Face access: first launch unpacks vendor/embed-models, Memory tab shows the embedder ready.
• Unit: npx vitest run tests/unit/embed-files.test.ts tests/unit/embed-manager.test.ts tests/unit/exec-git-bash.test.ts tests/unit/settings.test.ts tests/unit/transport-http.test.ts

[join3r]

Thank you for this — the Windows shell work is a real contribution, and it's on main now. Splitting it up rather than merging the branch, so here's exactly what happened to each part.

Landed

The Git Bash host shell — c196bfa, committed with your authorship. Making the shell a value of its own rather than a fact derived from process.platform, and threading it through the parser, the allowlist, the protected-roots scan and the judge prompt, is the right shape. The comment in executor.ts about windowsVerbatimArguments being per-shell instead of per-platform is exactly the kind of thing that saves the next person an afternoon.

The SSE test helper fix — same commit. That flake was real and it was ours, not yours.

Two fixes on top — 7e086d8, before any of it could ship:

1. WSL could be selected as "Git Bash". bashFromPathEnv accepted the first bash.exe on PATH, and isUsableGitBashPath only checked the filename. On a machine with WSL enabled that file is C:\Windows\System32\bash.exe, and System32 precedes everything else on PATH — so a box with Git installed somewhere unusual would have auto-selected the Linux VM launcher. Commands there address the disk as /mnt/c/..., which msysToWindows doesn't translate, so the read-only connected-folder guard would have quietly stopped matching: cat /mnt/c/Users/me/vault/secrets.txt would have gone straight through. Detection now requires the layout Git for Windows actually has — bin\bash.exe with cmd\git.exe or libexec\git-core beside it — which also rules out MSYS2 and Cygwin, whose PATH shape and path translation aren't the ones gitBashPathEnv and msysToWindows implement.

2. The shell was resolved twice. handleExecRequest resolved it to parse and judge; run() resolved it again to spawn, with an approval card in between that can sit for minutes. resolveHostShell touches the filesystem, so a Git upgrade replacing bin\bash.exe in that window would have handed a command parsed under bash quoting to cmd.exe, where ' is not a quote — the exact smuggle your new tier-1 tests guard against. There's now one resolveHostShellTarget answer, carrying the executable it chose, travelling all the way to spawn.

Also in that commit: the run_command tool description said cmd.exe was the Windows default when it's Git Bash; the Windows shell setting was gated on window.stem.platform, which is the client's OS while the setting governs the server's machine (a new exec:hostShellInfo answers with the server's, so a Windows server is manageable from a Mac); a pending path-field debounce could overwrite a shell you'd just clicked; and Browse accepted a git.exe that nothing would ever spawn.

The unit suite now also runs on windows-latest — the WSL distinction only means anything against a real filesystem, and those runners have both bashes.

Not landed: the vendored weights

vendor/embed-models/ isn't something we can take. Not a quality judgement on the code — the packing and unpacking both worked, I checked that the split parts gunzip to a valid 118 MB ONNX — but:

• Stem isn't a mirror for Hugging Face, and that holds even for the models we pick as defaults. Redistributing someone else's weights is a licensing question we have no reason to take on.
• Git history is permanent. 81 MB of blobs would be in every clone and every CI checkout from now until forever, and undoing it means a history rewrite.
• The electron-builder.yml entry put them in every platform's installer — macOS and Linux too, which can reach the Hub — plus the AppImage self-update payload.

Two things in the packing script would also have bitten: it walks the whole embed-models cache with no model filter, so anyone who had warmed the default reranker and ran the documented npm run vendor:embed-models would have gzipped ~570 MB into git; and a re-pack producing fewer parts leaves the old .gz.NN files behind, which packedOnnxGroups would then concatenate into a corrupt stream for every clone.

What replaced it

Your actual problem — a work laptop that can't reach huggingface.co, so memory search never starts — was worth solving, and it's solved:

• e31f986 keeps the good idea from your embed-files.ts: once a complete model is on disk, transformers.js is told not to reach for the Hub at all, so a copied cache doesn't hang on a DNS lookup. It judges "complete" by every file a load actually reads rather than the .onnx alone — with the Hub off, a missing tokenizer.json is a dead end, because a failed load poisons transformers.js for the rest of the worker's life.
• 6e1ac67 adds Import model files under Memory → Facts → Relevance ranking: point Stem at a folder that already holds the model — a copy of Stem's own model folder from another machine, a huggingface-cli download, a USB stick — and it validates, copies and loads. All three folder layouts are accepted, it names the missing file rather than failing later, and it never overwrites what's already there. With Stem on a server the picker walks the server's disk.
• 6c33f83 + 60763e4 go further than the catalog: a model Stem has never heard of can be imported too, answering only what the folder can't say (prompt prefixes for an embedder; scoring mode and score floors for a reranker), and it then sits in the model list beside the built-in ones.

So the offline case is covered, and it's covered for models we don't ship rather than only for the one that was in the branch.

Closing this, since the parts that could land have. If you get a chance to run the Windows smoke checklist in docs/windows-dev.md against main — especially on a machine with WSL installed — that'd be genuinely useful; you have the only Windows install there is. Thanks again.