Skip to content

Double-click end-user installer (~/.ndi) and MATLAB-safe viewer console scripts - #332

Merged
stevevanhooser merged 5 commits into
mainfrom
claude/ndi-end-user-installer
Oct 17, 2026
Merged

stevevanhooser merged 5 commits into
mainfrom
claude/ndi-end-user-installer

Conversation

@stevevanhooser

@stevevanhooser stevevanhooser commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

An installer for end users on Linux, macOS (Apple Silicon and Intel) and Windows that NDI-matlab can also run, and console scripts that are safe to start from MATLAB without shell wrappers. The ndi_install.py file is untouched. A fresh ndi_install.py install behaves as before, except the two viewers now clean up after MATLAB themselves (Part 1).

⚠️ Where this deviates from the shared contract (read this first)

The NDI-matlab-facing contract is implemented as specified. That means ~/.ndi/{bin,python,venvs} and nothing else, ~/.ndi/bin/{ndi,napariViewGEF,napariViewLightsheet} (.exe on Windows) plus uv, the script URLs, the flags --yes/--no-path/--uninstall and -Yes/-NoPath/-Uninstall, exit 0 only when the check passes, a last line of NDI installed: <abs path of ~/.ndi/bin>, and NDI_FROM_MATLAB=1. Three internal details differ from the brief:

  1. The escape from MATLAB's environment runs in a stdlib-only top-level module, src/_ndi_launch.py, not ndi/common/launch.py. The console scripts now point at _ndi_launch:napariViewGEF and _ndi_launch:napariViewLightsheet.
    • I measured that importing ndi.gui.app.genepyramid.cli runs ndi/__init__.py, which loads numpy, pydantic_core, DID and about 1190 modules before main() is called.
    • A helper inside ndi.*, called from main(), would therefore be weaker than the shell wrappers it replaces: those cleaned up before Python started.
    • Each main() still calls escapeMatlabEnvironment() first, as requested. That covers python -m ndi napari, and it does nothing after the console script has already re-exec'd.
    • The wheel is otherwise unchanged. I diffed it against main's: the only difference is _ndi_launch.py.
  2. uv is installed with UV_UNMANAGED_INSTALL as well as UV_INSTALL_DIR and UV_NO_MODIFY_PATH. I checked the cargo-dist template that generates Astral's installer. Without UV_UNMANAGED_INSTALL it writes ~/.config/uv/uv-receipt.json (or %LOCALAPPDATA%\uv\…). That receipt would redirect a user's own uv self update to our pinned copy. CI asserts that no receipt is written.
  3. On Windows the User PATH is read and written as the raw registry value (REG_EXPAND_SZ). SetEnvironmentVariable is used only to broadcast the change, which is also what Astral's installer does.
    • A plain [Environment]::SetEnvironmentVariable('Path', …, 'User') would rewrite the value as REG_SZ.
    • That flattens the stock %USERPROFILE%\AppData\Local\Microsoft\WindowsApps entry.

Additions beyond the brief:

  • On both platforms, ~/.ndi/bin is appended to the PATH, so the uv we ship never shadows a user's own uv.
  • Testing overrides: NDI_INSTALL_SOURCE, NDI_PYRAVIEW_FIND_LINKS, NDI_PYRAVIEW_CONSTRAINTS, and NDI_INSTALL_PYTHON. The last one exists for the Rosetta leg, see CI below.

Part 1: MATLAB-safe console scripts

  • When it acts: NDI_FROM_MATLAB is set, or LD_*/DYLD_* names a MATLAB architecture directory (glnxa64, maci64, maca64), and the marker NDI_MATLAB_ENV_CLEANED is not set.
  • What it removes: LD_LIBRARY_PATH, LD_PRELOAD, DYLD_LIBRARY_PATH, DYLD_FALLBACK_LIBRARY_PATH, DYLD_FRAMEWORK_PATH, DYLD_INSERT_LIBRARIES, PYTHONHOME and PYTHONPATH. Each is justified in the code. It then calls os.execve(sys.executable, [sys.executable, *sys.orig_argv[1:]], env). It only execs when something was actually removed.
  • When it never execs: an interpreter embedded in another program, such as MATLAB's in-process pyenv. Signs of that are no __main__ file, or on Linux /proc/self/exe not being a Python. Without this guard, an exec would replace MATLAB itself.
  • Windows: no exec. PYTHONHOME/PYTHONPATH and MATLAB's bin\win64-style directories are removed from os.environ in place, and only when NDI_FROM_MATLAB is set. Windows behaviour under MATLAB has not been measured, and the docstring says so.
  • Debug hook: with NDI_LAUNCH_DEBUG=1 the final process prints one JSON line to stderr, reporting cleaned: reexec | in-place | null and the remaining variables.
  • Tests: tests/test_ndi_launch.py has 31 tests. Environment, platform, argv and execve are injected so nothing really execs, plus one real exec in a child process.
  • Deleted tools/napariViewGEF and scripts/napariViewLightsheet, and updated every reference: pyproject, both cli.py, the lightsheet README, four bridge YAMLs, the CHANGELOG, README and getting-started.

Existing machines with a hand-installed wrapper. ndi_install.py never installed these wrappers. Their own comments told people to put them in /usr/local/bin by hand.

  • A copy (sudo install -m 0755 scripts/napariViewLightsheet /usr/local/bin/, or a cp of tools/napariViewGEF) keeps working after this merges. Both call entry points that still exist (python -m ndi napari and python -m ndi.gui.app.lightsheetZarr.cli).
  • A symlink into the clone (e.g. ln -s ~/NDI-python/tools/napariViewGEF /usr/local/bin/) breaks as soon as that clone pulls this change, because the target file is gone.
  • Fix: point the link at the environment's own console script instead (e.g. ln -sf /path/to/venv/bin/napariViewGEF /usr/local/bin/napariViewGEF), or set GUI.GEFManager.ViewerLauncher to that path. The console script now does the wrapper's job itself.

To check a machine: ls -l /usr/local/bin/napariView*.

INTERFACE UPDATE: I have modified the bridge contract for preferences (src/ndi/ndi_matlab_python_bridge.yaml) to reflect the MATLAB source. GUI.GEFManager.ViewerLauncher now defaults to "", meaning "find napariViewGEF automatically": ~/.ndi/bin first, then the PATH. Its description is updated, and the decision_log records that NDI-matlab makes the same change in a parallel PR. The hash is unchanged until that lands. In four other bridge entries only the decision_log changed, to note that the wrappers are gone; no hash moved.

Part 2: the installer (install/)

  • install-ndi.sh (POSIX sh; shellcheck-clean):
    • pins uv in one place (UV_VERSION="0.11.32") and fetches https://astral.sh/uv/0.11.32/install.sh only if that version is not already in ~/.ndi/bin;
    • runs uv tool install --python 3.12 --reinstall --find-links <raw pyraview-wheels.html> --constraints <raw pyraview-constraints.txt> --no-build-package pyraview "ndi[gui,napari,pyraview] @ <source>" with UV_PYTHON_INSTALL_DIR, UV_TOOL_DIR and UV_TOOL_BIN_DIR. It also sets UV_PYTHON_PREFERENCE=only-managed and UV_NO_CONFIG=1;
    • I confirmed with uv 0.11.32 that uv tool install accepts every flag, including a remote -c.
  • Post-install check: all three commands exist, both viewers answer --help, and ndi check passes. napari[all] brings pandas and matplotlib, so the full check applies.
  • Before running anything, the installer unsets the same loader/Python variables, because MATLAB's LD_LIBRARY_PATH can break the system curl.
  • PATH: an idempotent marked block in .zshrc/.zprofile or .bashrc/.profile, plus .bash_profile if it exists. --uninstall removes only bin, python, venvs and that block, and restores each profile file byte for byte: the marker records whether the installer added a missing final newline or created the file.
  • install-ndi.ps1: the same steps, for Windows PowerShell 5.1+. Empty entries in the User PATH (e.g. a trailing ;) are kept, so add-then-remove gives back the identical string.
  • Double-click wrappers:
    • Install NDI.command is committed with mode 100755.
    • Install NDI.cmd keeps CRLF via .gitattributes.
  • install/README.md: one page for end users, including the Gatekeeper right-click > Open step and the one-line curl … | sh and irm … | iex commands.

CI: .github/workflows/installer.yml

  • Triggers: filtered to install/**, pyproject, the launcher and the two cli.py; plus workflow_dispatch and a weekly run.
  • Legs: ubuntu-latest, ubuntu-24.04-arm, windows-latest (PowerShell 5.1), macos-latest, macos-15-intel, and macos-latest with Rosetta x86_64. All six pass.
  • What it installs: this PR's checkout, via NDI_INSTALL_SOURCE, with git, cmake and the compilers hidden (the Install every runtime dependency from archive URLs, not git #328/Install Pyraview 0.4.2 from its pre-built release wheels, never from source #330 technique).
  • Rosetta leg: uses Install Pyraview 0.4.2 from its pre-built release wheels, never from source #330's approach, an x86_64 interpreter via NDI_INSTALL_PYTHON=cpython-3.12-macos-x86_64. Running the installer under arch -x86_64 would not work: Astral's installer detects Rosetta with sysctl and installs the arm64 uv anyway.
  • Assertions:
    • the last line of output;
    • the three commands plus uv, and each --help;
    • the venv's machine type, that it uses uv's managed Python, and that pyraview is a release wheel (tools/check_pyraview_wheel.py);
    • with NDI_FROM_MATLAB=1 and junk loader/Python variables, the child reports "cleaned": "reexec" (Windows: "in-place") with every variable null;
    • an ordinary launch is left untouched;
    • exactly one PATH block, and a new login shell finds the command;
    • no uv receipt;
    • uninstall removes bin, python and venvs, and diff -r shows the rest of ~/.ndi (a dummy NDI_Preferences.json and tools/), the profile files and the Windows User PATH are byte-identical to before.

What was and wasn't verified locally

This sandbox gets 403 from github.com archive/release downloads and from astral.sh, so a real end-to-end install can only be proven in CI. Locally:

  • The full sh installer ran under dash in a fake HOME, with three stand-ins:
    • the pinned uv was pre-seeded (the sandbox's uv is exactly 0.11.32);
    • the GitHub-archive dependencies were redirected to local clones via UV_OVERRIDE;
    • Pyraview was replaced by a dummy wheel.
  • With those stand-ins it installed 160 packages and ndi check passed. A re-run was idempotent, and uninstall left the prefs and tools/ untouched.
  • The installed napariViewGEF re-exec'd correctly.
  • pip install -e . (what ndi_install.py runs) still gives an editable install from src/, with the new console-script targets.
  • The PowerShell script was not run locally; there is no PowerShell here. It is proven only by the Windows CI leg.
  • black, ruff, the full pytest suite (5865 passed), actionlint and shellcheck are all clean.

🤖 Generated with Claude Code

https://claude.ai/code/session_015KyLF4xAq4Sm4aT5gDyMTS

install/: install-ndi.sh (Linux, macOS), install-ndi.ps1 (Windows), the
double-clickable "Install NDI.command" / "Install NDI.cmd", and a README for
end users. A pinned uv (0.11.32) installs Python 3.12 and
ndi[gui,napari,pyraview] as a uv tool under ~/.ndi/{bin,python,venvs} only,
with Pyraview from its release wheels (--no-build-package pyraview).
--uninstall removes exactly those folders and the marked PATH block.
NDI_INSTALL_SOURCE / NDI_PYRAVIEW_* / NDI_INSTALL_PYTHON let CI install the
code under test.

napariViewGEF and napariViewLightsheet now go through src/_ndi_launch.py, a
stdlib-only module beside the package: started from MATLAB it drops
LD_LIBRARY_PATH, LD_PRELOAD, DYLD_*, PYTHONHOME and PYTHONPATH and re-execs
BEFORE `import ndi` loads numpy (cleans PATH in place on Windows). Replaces
the shell wrappers tools/napariViewGEF and scripts/napariViewLightsheet.
GUI.GEFManager.ViewerLauncher now defaults to "" (find it automatically).

.github/workflows/installer.yml runs the real installer with git and
compilers hidden on Linux x86_64/aarch64, Windows, macOS arm64, macOS Intel
and x86_64-under-Rosetta, then checks the commands, the MATLAB re-exec, the
PATH block and a clean uninstall.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015KyLF4xAq4Sm4aT5gDyMTS
The macOS runners' ~/.bash_profile ends without a newline. Adding the PATH
block added one, and --uninstall left it behind. The block's first line now
records when a newline was added (and when the file was created), and
--uninstall takes exactly that away again. A pre-existing empty profile file
is no longer deleted; only one this script created.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015KyLF4xAq4Sm4aT5gDyMTS
…ninstall

The Windows runner's user Path ends in ';'. Dropping empty entries on the
way in meant uninstall wrote back the string without it. Entries are now
kept as they are, ours goes before a trailing empty one, and removal takes
only ours, so add-then-remove returns the identical string.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015KyLF4xAq4Sm4aT5gDyMTS
Resolves the CHANGELOG conflict with #333 (both sides added an entry under
Unreleased/Added). Also puts this PR's entries in the right sections: the
first commit had opened a second 'Changed' heading above existing 'Added'
entries.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015KyLF4xAq4Sm4aT5gDyMTS

Copy link
Copy Markdown
Contributor Author

MATLAB/Python bridge completeness is red on 14f404a, and the cause is not this PR.

test_no_entry_has_drifted flags 10 entries: 9 in src/ndi/cloud/ and src/ndi/cloud/sync/, plus preferences.

  • All 10 come from one NDI-matlab change: Make cloud upload/download batch sizes configurable via preferences VH-Lab/NDI-matlab#1033 (018518f4f, "Wire cloud batch-size preferences into upload and download", merged as 7180cc8).
  • It reached NDI-matlab main after main's last green bridge run (12:48 UTC today). This job checks against NDI-matlab's current main, so every NDI-python PR fails it now.
  • The preferences drift is #1033's Max_File_Batch_Size default (500 MB → 250 MB), not the ViewerLauncher change here.

The fix is #334, which ports #1033 and bumps exactly these 10 hashes; its bridge job is green.

  • I have not copied it into this PR. It is an ~800-line cloud-upload change.
  • Bumping only the hashes here would assert a port that did not happen, which section 7 of docs/developer_notes/ndi_matlab_python_bridge.yaml forbids.

Plan: merge #334 first. I will then merge main into this branch and resolve the expected overlap: both PRs touch src/ndi/preferences.py and the preferences bridge entry, on different items. CI then re-runs on that push. All other checks on this head are green or still running.


Generated by Claude Code

CHANGELOG conflict only: both sides added Unreleased entries; kept all of
main's and this PR's two. preferences.py and the preferences bridge entry
merged cleanly (#334's batch-size default and 7180cc8 hash, this PR's
ViewerLauncher default and decision_log note).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015KyLF4xAq4Sm4aT5gDyMTS
@stevevanhooser
stevevanhooser merged commit 5723c87 into main Oct 17, 2026
21 checks passed
@stevevanhooser
stevevanhooser deleted the claude/ndi-end-user-installer branch October 17, 2026 17:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants