[Hackathon] fix(input): plan, steps, code and password files saved by Windows PowerShell 5.1 are UTF-16 and get rejected or garbled #463

Verified against (commit SHA or released version)

commit 988c035 (main, v0.14.0), built and run on Windows 11, Windows PowerShell 5.1.26100, Node 24.16.

Discord Username / User ID

gradzzz

What does this improvement do?

The docs create plan files with shell redirection: testsprite test scaffold > first-test.plan.json (DOCUMENTATION.md L348) and testsprite test create --plan-template > plan.json (L384). In Windows PowerShell 5.1 (powershell.exe, the default shell on Windows 10 and 11), > writes UTF-16LE with a BOM. The CLI decodes every input file as UTF-8, so its own template fails its own linter:

PS> testsprite test create --plan-template > plan.json
PS> Format-Hex plan.json | Select-Object -First 1
00000000   FF FE 7B 00 0D 00 0A 00 20 00 20 00 22 00 24 00  .þ{..... . .".$.
PS> testsprite test lint --plan-from plan.json
plan.json: plan-from: not valid JSON: Unexpected token '�', "��{
   "... is not valid JSON
0/1 valid, 1 problem(s)
Error: lint: 1 problem(s) across 1 file(s)
PS> $LASTEXITCODE
5

test scaffold > first-test.plan.json fails the same way. The same happens to any file a user writes on PS 5.1 with >, Out-File or echo ... >:

• --password-file: no error, but the secret is garbled. The CLI's own reader returns "��h\u0000u\u0000n\u0000t\u0000e\u0000r\u00002\u0000\r\u0000\n\u0000" for a file made with "hunter2" > pw.txt (it returns "hunter2" for the UTF-8 version), and that value is what the project's login gets.
• --code-file: the code is uploaded with a NUL byte after every character.

Only a UTF-8 BOM is handled today (stripBom, from #34, for Set-Content -Encoding utf8).

Readers involved, all at 988c035:

• readCappedFileGuarded, test.ts#L2151: --plan-from, --steps, create-batch --plans, and test lint --plan-from/--steps
• readCodeFile, test.ts#L1781: --code-file
• the plan-directory reader, test.ts#L5103: create-batch --plan-from-dir
• test lint --plans, test.ts#L6010: this one reads the file with its own readFileSync(..., 'utf8') instead of the shared reader, so a UTF-16LE JSONL file reports every line as invalid JSON
• readSecretFileGuarded, secret-file.ts#L101: --password-file on project create/update, project env, project sign-in

Details / implementation notes

Proposal: one small decoder for user-supplied text files, used by the five readers above so the encoding rule lives in one place:

• UTF-8 BOM (EF BB BF): strip it and decode as UTF-8 (today's behaviour)
• UTF-16LE BOM (FF FE): decode as UTF-16LE
• UTF-16BE BOM (FE FF): decode as UTF-16BE
• anything else: UTF-8, exactly as today (no guessing for BOM-less input, no new dependency)

Size caps keep counting raw bytes, as now. UTF-8 input behaves exactly as before.

Tests: Buffer fixtures for each BOM in the decoder's unit test; test lint --plan-from and test create --plan-from --dry-run with a UTF-16LE fixture; --password-file with a UTF-16LE fixture asserting the decoded value; the existing UTF-8 BOM tests unchanged.

Out of scope: the raw credential, client-secret and refresh-token file reads in project.ts, which #282 / #283 move onto readSecretFileGuarded. They get this fix once that lands.

Confirmations

• I have searched existing issues and this is not a duplicate.
• I have provided my Discord identity above for reward coordination.