Skip to content

Navigator: DAQ system browser, cloud dataset id, Assign to workspace; faster daqsystem_load - #344

Merged
stevevanhooser merged 4 commits into
mainfrom
claude/loving-einstein-ppjymo
Oct 24, 2026
Merged

stevevanhooser merged 4 commits into
mainfrom
claude/loving-einstein-ppjymo

Conversation

@stevevanhooser

Copy link
Copy Markdown
Contributor

Summary

The Python counterpart of VH-Lab/NDI-matlab#1058. Merge that PR first. The bridge records here point at commits on its branch.

Navigator (ndi.gui.nav)

  • DaqSystemBrowser (daq_system_browser.py): opened from Session → DAQ Systems....
    • Left: a list of the session's DAQ systems.
    • Right: a detail panel showing the reader, file navigator and file patterns, the epoch probe map class and patterns, and the metadata readers.
    • Load epochs: nothing is read from disk until clicked. It then shows an epoch's actual files, which pattern each matches, and which file is the epoch probe map.
    • Class names use MATLAB form (ndi.daq.reader.mfdaq.intan, via ndi_matlab_classname), so both languages show the same names. File patterns are matched the same way find_file_groups matches them.
    • The logic is in static methods that need no display: describe, epoch_file_rows, matching_pattern, detail_items. load_epochs and choose_epoch work without widgets too.
  • CloudStatusDialog (cloud_status_dialog.py): Check Cloud status now opens this window instead of an alert. It shows the NDI Cloud dataset ID with a Copy button, greyed out when the dataset is not in the cloud. datasets_cloud.check_cloud_status now returns the ID, in the new CloudActionResult.cloud_dataset_id field.
  • Assign to workspace… on dataset and session nodes:
    • It binds the object to a variable in __main__, the namespace the tree already scans for variable names.
    • It checks the name with datasets_text.workspace_name_error and confirms before replacing an existing variable.
    • It updates labels in place (relabel_nodes), using each node's base label and id stored on the tree item.
    • Datasets are now listed once per id (datasets_model.unique_by_id).

Session

  • Faster daqsystem_load: a name that matches everything ('(.*)', '.*', optionally anchored) is no longer sent as a regexp search. It is detected by the new ndi_session.isMatchAllRegexp, alias is_match_all_regexp. On a dataset of about 100,000 documents the regexp roughly doubled the search time, and probe epoch tables, getprobes and the sync graph all load DAQ systems that way. SessionInfo and the browser call daqsystem_load() directly.
  • Fix: daqsystem_add called dev.setsession, but the method on ndi_daq_system is named set_session. Adding any DAQ system therefore raised AttributeError.

Bridge

  • Updated records: datasetsPane (d2b5170), sessionInfo (b1da61b), session (10bad94, adds isMatchAllRegexp), and ndi.daq.system (e066e8d). For ndi.daq.system, NDI-matlab added an experiment and then reverted it, so there is no net change; the decision log names both commits.
  • New records: daqSystemBrowser and cloudStatusDialog. Like the other ndi.gui.nav records, they name no python_class, because the attributes are snake_case (figure, epoch_table, …). The name mapping is written out in each decision log.

Testing

New tests:

  • tests/test_gui_nav_daq_system_browser.py: helper functions, plus a real session with an mfdaq/intan DAQ system, with and without Qt.
  • tests/test_gui_nav_cloud_status_dialog.py
  • tests/test_session_daqsystem_load_name.py: also covers daqsystem_add.
  • Assign to workspace and the cloud status window in tests/test_gui_nav_datasets_pane.py.
  • unique_by_id, workspace assignment and name checks in tests/test_gui_nav_datasets_model.py.

Results:

  • Locally, under QT_QPA_PLATFORM=offscreen: 6462 passed, 352 skipped and 1 failed (tests/symmetry excluded). The skips are mostly cloud and network tests. The failure was the snake_case-alias convention check, fixed in b017b02; that check and the related tests pass on their own afterwards, and the whole suite was not rerun.
  • The bridge checks (test_matlab_bridge_*, test_matlab_branch_pending) pass under NDI_BRIDGE_CHECK_STRICT=1 against the NDI-matlab branch.
  • black --check and ruff check are clean.

Generated by Claude Code

Mirrors VH-Lab/NDI-matlab#1058.

- "Session > DAQ Systems..." opens ndi.gui.nav.DaqSystemBrowser: a
  read-only master-detail view of a session's DAQ systems (reader, file
  navigator and file patterns, epoch probe map class and patterns,
  metadata readers) with an on-request example epoch showing the actual
  files, the pattern each matches and which is the epoch probe map.
- "Check Cloud status" opens ndi.gui.nav.CloudStatusDialog, showing the
  NDI Cloud dataset id with a Copy button (disabled when not in the
  cloud); check_cloud_status now returns the id.
- "Assign to workspace..." on dataset and session nodes binds the object
  to a variable in __main__ and relabels the tree in place; datasets are
  listed once per id.
- daqsystem_load drops a match-everything name ('(.*)', '.*') instead of
  sending a slow regexp search (new ndi_session.isMatchAllRegexp);
  SessionInfo and the browser load DAQ systems with no name filter.
- Fix session.daqsystem_add, which called dev.setsession (the method is
  set_session) and so failed for every DAQ system.

Bridge records synchronized with the NDI-matlab branch commits.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PqqXTGv4TzdR86CnB2MBLv
Every MATLAB-named method on ndi_session carries a snake_case alias
(tests/test_matlab_name_aliases.py).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PqqXTGv4TzdR86CnB2MBLv
session._find_all_dependencies (the database_rm cascade) searched
depends_on("", id), relying on an empty name meaning "any name". DID-python
main now runs depends_on in SQL (DID-python#101) and, like its brute-force
path, treats only '*' as the wildcard, so the cascade found nothing: with
current DID, deleting an element left its neuron_extracellular and epoch
documents in place, failing
test_gui_app_spike_sorter_importer::TestDeleting on main and on this PR.
'*' is what MATLAB's ndi.database.fun.findalldependencies asks for, and it
passes with both the old and the new DID.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PqqXTGv4TzdR86CnB2MBLv

Copy link
Copy Markdown
Contributor Author

The test (3.10/3.11/3.12) failure on b017b02 did not come from this PR's changes: test_gui_app_spike_sorter_importer.py::TestDeleting::test_deleting_every_neuron_clears_the_provenance_marker fails on main too with current DID-python main (8c3ac14).

Cause: DID-python#101 runs depends_on searches in SQL and treats only '*' as "any dependency name".

  • NDI's delete cascade, session._find_all_dependencies, searched depends_on("", id), so it found nothing.
  • Deleting an element therefore left its neuron_extracellular and epoch documents behind.

Fix: 859a4ac changes the search to depends_on("*", id), as MATLAB's findalldependencies does.

  • It passes with both the old (b224054) and the new (8c3ac14) DID-python.
  • The full local suite passes against DID main: 6463 passed.
  • Once this merges, main needs nothing more.

Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants