Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions src/did/+did2/+convert/+migrators/Contents.m
Original file line number Diff line number Diff line change
Expand Up @@ -42,8 +42,9 @@
%
% The position_metadata, distance_metadata and stimulus_bath base migrators
% are deleted too: each class has a +migrators_j migrator, which reads raw v1
% (distance_metadata's reshapes the v1 spelling itself, where it used to hand
% the renamed body to the base migrator).
% (distance_metadata's reshaped the v1 spelling itself, where it used to hand
% the renamed body to the base migrator; since [reshaped retire classes
% carried], Jess Haley, 2026-10-29, it carries every document whole as v1).
%
% The element, ontology_image and ontology_label base migrators are deleted
% (2026-10-27): each class has a +migrators_j migrator that reads raw v1, and
Expand Down
5 changes: 2 additions & 3 deletions src/did/+did2/+convert/+migrators_j/Contents.m
Original file line number Diff line number Diff line change
Expand Up @@ -695,8 +695,7 @@
% deleted 2026-10-21; the class is now `schemas/V_eta/stable/demo.json`); the
% fold itself is in private/jDemoFold.
% distance_metadata
% Brainstorm-J migrator: reshape a did_v1 distance_metadata into the validated
% V_eta distance_metadata. 1 -> 1.
% Brainstorm-J migrator: carry a v1 distance_metadata whole as v1. 1 -> 1.
% electrode_offset_voltage
% Brainstorm-J migrator: did_v1 electrode_offset_voltage -> an `observation`
% with value kind `voltage` about the probe's entity, qualified by the
Expand Down Expand Up @@ -820,7 +819,7 @@
% sorted unit's own entity (+ the shared session anchor, the software and its
% run); no entity of its own.
% oneepoch
% Brainstorm-J migrator: keep the class, fold its inherited block.
% Brainstorm-J migrator: carry a v1 oneepoch whole as v1. 1 -> 1.
% ontology_image
% Brainstorm-J migrator: did_v1 ontology_image, dispatched ON SHAPE. The
% current NDI shape (`ontology_nodes` + an `ontologyTableRow_id` edge + the
Expand Down
150 changes: 29 additions & 121 deletions src/did/+did2/+convert/+migrators_j/distance_metadata.m
Original file line number Diff line number Diff line change
@@ -1,6 +1,22 @@
function bodies = distance_metadata(preBody)
%DISTANCE_METADATA Brainstorm-J migrator: reshape a did_v1 distance_metadata into
% the validated V_eta distance_metadata. 1 -> 1.
%DISTANCE_METADATA Brainstorm-J migrator: carry a v1 distance_metadata whole as
% v1. 1 -> 1.
%
% ================ [reshaped retire classes carried] =====================
% Decided by Jess Haley, 2026-10-29 (did-schema
% V_eta_go_forward_class_audit.md, "B"): v1 `distance_metadata` and
% `oneepoch` are carried whole as v1 for now, not reshaped into their retire
% classes; the reshape waits for their V2 homes (`distance_metadata`: a
% decision still to make, docs/V2_PLAN.md G50). So EVERY distance_metadata
% is carried whole as v1 (the untouched v1 source, with a `carried_as_v1`
% note): one holding a field outside its NDI template under carry-100, as
% before, and every other one under register row carry-147, awaiting_home.
% The reshape into the V_eta `distance_metadata` (a class whose disposition
% is `retire`) described below is no longer done; the description is the
% record of what was replaced. did2.convert.roundtrip.distance_metadata, its
% inverse, is reached by no migrator output any more; a carried document's
% round trip is the carry itself (did2.convert.roundtrip.check).
% =======================================================================
%
% Routed from did2.convert.v1_to_v2 only when TargetVersion == 'V_eta'. The v1
% doc is FLAT and records the two ENDPOINTS of a distance measurement, not the
Expand Down Expand Up @@ -96,34 +112,17 @@
'distance_metadata body is missing the distance_metadata property block.');
end

% the endpoint labels: the v1 suffixes, and on the renamed path (v1_to_v2
% 'RawV1Classes', {}, until universalRenames is deleted) the suffixes of the
% spelling that pass gives them (ontology_node_a, integer_i_ds_a, ...)
labels = upper(suffixes);
renamed = renamedSuffixes(blk);
labels = [labels, upper(renamed(~ismember(upper(renamed), labels)))];
[labels, order] = sort(labels);
raw = [suffixes, repmat({''}, 1, numel(labels) - numel(suffixes))];
raw = raw(order);
endpoints = struct('label', {}, 'measurement', {}, ...
'integer_ids', {}, 'string_ids', {}, 'numeric_values', {});
for k = 1:numel(labels)
endpoints(end+1) = buildEndpoint(blk, labels{k}, raw{k}); %#ok<AGROW>
end

unitsNode = '';
if isfield(blk, 'units')
unitsNode = char(blk.units);
end

v2Body = preBody;
if isfield(v2Body, 'depends_on') && ~isempty(v2Body.depends_on)
% the {name, document_id} entries the renamed path gave the document
deps = did2.convert.v1read.readDependsOn(src);
v2Body.depends_on = struct('name', {deps.name}, 'document_id', {deps.document_id});
end
v2Body.distance_metadata = struct('endpoints', endpoints, 'units', ontologyTerm(unitsNode));
bodies = {v2Body};
% [reshaped retire classes carried] (Jess Haley, 2026-10-29, did-schema
% V_eta_go_forward_class_audit.md): no longer reshaped into the retire class
% `distance_metadata`; carried whole as v1 until its V2 home is decided.
b = did2.convert.v1read.readBase(src);
id = b.id;
if ~ischar(id); id = ''; end
% carry-register: carry-147
did2.convert.reportNote('carried_as_v1', 'distance_metadata', id, ...
['[carry-147] awaiting_home: distance_metadata, until its V2 home is ' ...
'decided ([reshaped retire classes carried])']);
bodies = {src};
end

function suffixes = endpointSuffixes(block)
Expand All @@ -138,94 +137,3 @@
end
end
end

function suffixes = renamedSuffixes(block)
% Every x with an `ontology_node_x` field (universalRenames' spelling).
suffixes = {};
fns = fieldnames(block);
for k = 1:numel(fns)
tokens = regexp(fns{k}, '^ontology_node_(.+)$', 'tokens', 'once');
if ~isempty(tokens) && ~any(strcmpi(suffixes, tokens{1}))
suffixes{end+1} = tokens{1}; %#ok<AGROW>
end
end
end

function [v, found] = endpointField(block, rawName, x, renamedName, label)
% The v1 field <rawName>_<x>, else universalRenames' <renamedName>_<label>.
v = [];
found = false;
names = {};
if ~isempty(x)
names{end+1} = [rawName '_' x];
end
names{end+1} = [renamedName '_' lower(label)];
for k = 1:numel(names)
if isfield(block, names{k})
v = block.(names{k});
found = true;
return;
end
end
end

function ep = buildEndpoint(block, label, x)
% One V2 endpoint record from v1's per-suffix fields (X, as the document
% spells it, or '' when only the renamed spelling holds the endpoint). A field
% the document does not have is an empty value of the right type.
nodeVal = '';
[v, found] = endpointField(block, 'ontologyNode', x, 'ontology_node', label);
if found
nodeVal = char(v);
end
intIds = [];
[v, found] = endpointField(block, 'integerIDs', x, 'integer_i_ds', label);
if found
intIds = double(v);
intIds = intIds(:)';
end
stringIds = string.empty(1, 0);
[raw, found] = endpointField(block, 'ontologyStringValues', x, 'ontology_string_values', label);
if found
if ischar(raw)
parts = strsplit(raw, ',');
parts = cellfun(@strtrim, parts, 'UniformOutput', false);
parts = parts(~cellfun('isempty', parts));
stringIds = string(parts);
elseif isstring(raw)
stringIds = raw(:)';
end
end
nums = [];
[v, found] = endpointField(block, 'ontologyNumericValues', x, 'ontology_numeric_values', label);
if found
if isnumeric(v)
nums = double(v(:)');
end
end
ep = struct( ...
'label', label, ...
'measurement', ontologyTerm(nodeVal), ...
'integer_ids', intIds, ...
'string_ids', stringIds, ...
'numeric_values', nums);
end

function term = ontologyTerm(curie)
% An ontology_term composite. Empty input yields a node-only, name-empty
% struct. ndi.ontology.lookup failures leave name ''.
term = struct('node', char(curie), 'name', '');
if isempty(curie)
return;
end
try
[~, resolved] = ndi.ontology.lookup(char(curie));
if isstring(resolved) && isscalar(resolved)
term.name = char(resolved);
elseif ischar(resolved)
term.name = resolved;
end
catch
% ndi.ontology not on path or curie unknown; leave name ''.
end
end
4 changes: 2 additions & 2 deletions src/did/+did2/+convert/+migrators_j/element_epoch.m
Original file line number Diff line number Diff line change
Expand Up @@ -102,8 +102,8 @@
% referent "the element_epoch doc and its binary"). When this dissolves the
% edge retargets to whatever absorbs the payload.
% * `oneepoch` inherits its whole shape from `element_epoch` and is handled by
% its own migrator as a fold-into-tombstone; fork A1 for it is chosen but
% NOT SIGNED, and is gated on #30 besides.
% its own migrator, which carries it whole as v1 (carry-126 / carry-148,
% [reshaped retire classes carried], Jess Haley, 2026-10-29).
%
% ALSO STILL TRUE, and repairs rather than decisions:
% `acquisition_epoch.axes` / `.channels` / `.storage` appear in NO NDI template
Expand Down
90 changes: 47 additions & 43 deletions src/did/+did2/+convert/+migrators_j/oneepoch.m
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
function v2Body = oneepoch(preBody)
%ONEEPOCH Brainstorm-J migrator: keep the class, fold its inherited block.
%ONEEPOCH Brainstorm-J migrator: carry a v1 oneepoch whole as v1. 1 -> 1.
%
% EVERY oneepoch IS CARRIED WHOLE AS v1 (the untouched v1 source, with a
% `carried_as_v1` note): one stitched on device-local time under carry-126,
% one on a global clock under carry-148, see THE 2026-10-27 / 2026-10-29
% DECISIONS below. The class is no longer kept and its inherited block is no
% longer folded; the sections up to READS RAW v1 are the record of what that
% fold did, kept until the global-clock calculation replaces it.
%
% `oneepoch` is NOT an epoch class. It is the record of a CONCATENATION:
% `ndi.element.oneepoch` glues an element's N epochs into a single epoch, and
Expand Down Expand Up @@ -87,21 +94,32 @@
% +element/oneepoch.m: "as though the recordings occurred one
% immediately after the other") is carried whole as v1
% `needs_mapping`.
% BUILT HERE: the device-local carry. ndi.element.oneepoch picks the first
% GLOBAL clock of the input's epoch table (ndi.time.clocktype.isGlobal:
% utc, approx_utc, exp_global_time, approx_exp_global_time,
% dev_global_time, approx_dev_global_time) and stitches on
% dev_local_time only when there is none; it writes EVERY clock of the
% element, comma-joined, as element_epoch.epoch_clock. So a oneepoch none
% of whose clocks is global was stitched on device-local time, and is
% carried whole as v1 (isStitchedOnLocalTime), citing register row
% carry-126 (did-schema 64ad58b8).
% NOT BUILT YET: the global-clock calculation. It still keeps the class
% `oneepoch` as below, until the questions in the PR that builds the
% device-local carry are answered (which statements are "the per-epoch
% recordings" when the input's per-epoch observations are not in the
% batch; the calculation's variable and anchor; a sampled_body from a
% .vhsb the DID pass cannot read).
% [reshaped retire classes carried] (Jess Haley, 2026-10-29, "B")
% v1 `distance_metadata` and `oneepoch` are carried whole as v1 for
% now, not reshaped into their retire classes; the reshape waits for
% their V2 homes (`oneepoch`: docs/V2_PLAN.md G44).
% BUILT HERE: both carries. ndi.element.oneepoch picks the first GLOBAL
% clock of the input's epoch table (ndi.time.clocktype.isGlobal: utc,
% approx_utc, exp_global_time, approx_exp_global_time, dev_global_time,
% approx_dev_global_time) and stitches on dev_local_time only when there is
% none; it writes EVERY clock of the element, comma-joined, as
% element_epoch.epoch_clock. So a oneepoch none of whose clocks is global
% was stitched on device-local time, and is carried whole as v1,
% needs_mapping (isStitchedOnLocalTime), citing register row carry-126
% (did-schema 64ad58b8). One with a global clock, which until 2026-10-29
% kept the class `oneepoch` with its element_epoch block folded onto it, is
% carried whole as v1, awaiting_home, citing register row carry-148
% ([reshaped retire classes carried]).
% NOT BUILT YET: the global-clock calculation and the batch pass for its
% per-epoch `input_id` links (docs/V2_PLAN.md G44); when it is built it
% replaces the carry-148 branch. Open questions: which statements are "the
% per-epoch recordings" when the input's per-epoch observations are not in
% the batch; the calculation's variable and anchor; a sampled_body from a
% .vhsb the DID pass cannot read.
%
% did2.convert.roundtrip.oneepoch, the inverse of the kept class, is reached
% by no migrator output while every oneepoch is carried; a carry's round
% trip is the carry itself (did2.convert.roundtrip.check).

arguments
preBody (1,1) struct
Expand Down Expand Up @@ -149,11 +167,11 @@
% V_eta_go_forward_class_audit.md: a oneepoch NDI stitched on device-local time
% (none of its clocks global) is carried whole as v1, needs_mapping. Its
% times exist in no input.
id = '';
if isfield(src, 'base') && isstruct(src.base) && isfield(src.base, 'id')
id = char(src.base.id);
end
if isStitchedOnLocalTime(v2Body.element_epoch.clocks)
id = '';
if isfield(src, 'base') && isstruct(src.base) && isfield(src.base, 'id')
id = char(src.base.id);
end
% carry-register: carry-126
did2.convert.reportNote('carried_as_v1', 'oneepoch', id, ...
['[carry-126] needs_mapping: a oneepoch stitched on ' ...
Expand All @@ -163,29 +181,15 @@
return;
end

% The concrete class is unchanged -- `oneepoch` persists as a source tombstone
% until the global-clock calculation ([oneepoch is a redundant calculation]) is
% built; see THE 2026-10-27 / 2026-10-29 DECISIONS above.
v2Body.document_class.class_name = 'oneepoch';

if ~isfield(v2Body, 'oneepoch') || ~isstruct(v2Body.oneepoch)
v2Body.oneepoch = struct();
end

% Fold the inherited block onto the concrete one. Field-by-field rather than
% wholesale, so a v1 field this migrator has never seen lands somewhere the
% strict-fields check will REPORT it (`undeclaredField` on `oneepoch`) instead of
% being carried in a block nothing validates.
if isfield(v2Body, 'element_epoch')
inherited = v2Body.element_epoch;
if isstruct(inherited)
fns = fieldnames(inherited);
for k = 1:numel(fns)
v2Body.oneepoch.(fns{k}) = inherited.(fns{k});
end
end
v2Body = rmfield(v2Body, 'element_epoch');
end
% [reshaped retire classes carried] (Jess Haley, 2026-10-29, did-schema
% V_eta_go_forward_class_audit.md): a oneepoch on a global clock, which kept
% the class `oneepoch` until then, is carried whole as v1 until the
% global-clock calculation ([oneepoch is a redundant calculation]) is built.
% carry-register: carry-148
did2.convert.reportNote('carried_as_v1', 'oneepoch', id, ...
['[carry-148] awaiting_home: a oneepoch on a global clock, until its ' ...
'data-combination calculation is built ([reshaped retire classes carried])']);
v2Body = {src};
end

function tf = isStitchedOnLocalTime(clocks)
Expand Down
7 changes: 7 additions & 0 deletions src/did/+did2/+convert/+roundtrip/distance_metadata.m
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,13 @@
%
% Any other emitted document is unplaced.
%
% NO MIGRATOR OUTPUT REACHES THIS INVERSE since [reshaped retire classes
% carried] (Jess Haley, 2026-10-29, did-schema
% V_eta_go_forward_class_audit.md): migrators_j.distance_metadata carries every document
% whole as v1 (carry-147), and a carry's round trip is the carry itself
% (did2.convert.roundtrip.check). It is kept until the class's V2 home is
% built, whose inverse replaces it.
%
% See also did2.convert.roundtrip.check,
% did2.convert.migrators_j.distance_metadata.

Expand Down
7 changes: 7 additions & 0 deletions src/did/+did2/+convert/+roundtrip/oneepoch.m
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,13 @@
% element_epoch.epoch_clock the clocks' names, comma-joined
% element_epoch.t0_t1 the clocks' [t0; t1], one column each
%
% NO MIGRATOR OUTPUT REACHES THIS INVERSE since [reshaped retire classes
% carried] (Jess Haley, 2026-10-29, did-schema
% V_eta_go_forward_class_audit.md): migrators_j.oneepoch carries every document
% whole as v1 (carry-126 / carry-148), and a carry's round trip is the carry itself
% (did2.convert.roundtrip.check). It is kept until the class's V2 home is
% built, whose inverse replaces it.
%
% See also did2.convert.roundtrip.check,
% did2.convert.migrators_j.oneepoch,
% did2.convert.roundtrip.element_epoch.
Expand Down
9 changes: 5 additions & 4 deletions tests/+did2/+unittest/testFixtureCorpus.m
Original file line number Diff line number Diff line change
Expand Up @@ -1251,10 +1251,11 @@ function testACarriedCalculatorWithNoTombstoneIsCountedNotQuarantined(testCase)
% ---- distance_metadata (element_id) -- REAL FLAT v1 shape ------------------
% The writer emits FLAT per-endpoint fields (ontologyNode_A/_B, integerIDs_A/_B,
% ontologyStringValues_A/_B, ontologyNumericValues_A/_B empty by design, units),
% NOT a nested `endpoints` block. The migrator reshapes flat -> the nested
% `endpoints` array the schema requires; this fixture is the real shape so the
% reshape is exercised under validation (element_id -> the minted subject; the
% endpoint node ids are field values, not orphan-checked depends_on edges).
% NOT a nested `endpoints` block. Until 2026-10-29 the migrator reshaped flat ->
% the nested `endpoints` array of the retire class; since [reshaped retire
% classes carried] (Jess Haley, 2026-10-29) it carries the document whole as v1
% (carry-147), and this fixture checks the carry does not quarantine under
% validation.
function batch = fx_distance_metadata()
sub = subjDoc('dm_sub', 'elemDM');
d = struct();
Expand Down
3 changes: 2 additions & 1 deletion tests/+did2/+unittest/testMigrators.m
Original file line number Diff line number Diff line change
Expand Up @@ -317,6 +317,7 @@ function testTheUnreachedBaseMigratorsAreGone(testCase)
% --- position_metadata, distance_metadata, stimulus_bath ---
% Their +migrators base migrators are deleted ([early migration pieces fixed]
% (1)): never reached (each class has a +migrators_j migrator), or reached only
% through distance_metadata's own, which now reshapes the raw v1 spelling. The
% through distance_metadata's own, which reshaped the raw v1 spelling itself
% (and since [reshaped retire classes carried], 2026-10-29, carries it). The
% classes read raw v1 (did2.convert.rawV1Classes); their round trips are
% testRawV1RoundTripBath and testRawV1RoundTripPosition.
Loading
Loading