Skip to content

Use the declaring assembly to attribute a versioning finding - #22

Open
sakanni wants to merge 7 commits into
developfrom
feat/runner-read-asm
Open

sakanni wants to merge 7 commits into
developfrom
feat/runner-read-asm

Conversation

@sakanni

@sakanni sakanni commented Sep 14, 2026 •

Copy link
Copy Markdown
Collaborator

Issues addressed by this PR

The versioning check decides which repository a finding belongs to. For an object record it has only a type name, so it infers ownership from the namespace, and namespaces are shared. BHoM/BHoM run 35986238826 reported 145 findings, every one attributed by guess and none by evidence.

An object record's declaring assembly is in the dataset, in the record's own _asm field. The runner now reads it from the tree the build stages on the same machine, which an earlier step already validates, and a record without the field keeps today's behaviour.

Run 36512880648 re-ran that population against a dataset carrying the field. The 145 findings become 47, all attributed by declaring assembly and none by namespace guess, over the same subject set, 30 assemblies and 1,192 types on both. 98 dropped as another repository's, 46 stayed unverified, and one became a real failure: BH.oM.Structure.Loads.LoadCombination, declared by Structure_oM. The run is red for that one finding and is meant to be.

Test files

201 tests pass, 135 of them pre-existing and unmodified. Sixteen mutation checks ran, across the map, the collector, the reclassification guard, the drop counter and each candidate source; each failed tests and reverted clean.

Changelog

ci-versioning attributes an object-record finding by the declaring assembly recorded in the dataset, rather than inferring ownership from the namespace.

Additional comments

Inert until a dataset carries the field. No version does today, so the map is empty on a current pull request and nothing changes.

A finding is joined to its record on the leaf's description, which for an object record is the _t value. That join is what holds attribution together, and it rests on Versioning_Toolkit continuing to describe a failing object record by its type name; if that changes, attribution returns to the guess with no test in either repository going red. Against that: the namespace fallback already reads the same description as a type name, so nothing newly depends on it, and the description is the finding's label in every artefact, so a change shows rather than hides.

The map unions every staged version directory rather than the newest, because which version a pull request reads is not visible to the runner. Two versions naming different assembly families for one type leave it unattributed and reported. Only one version carries the field today, so the run above had nothing to disagree with and that path is covered by unit tests alone; comparing the two backfilled versions offline gives 1,711 shared type names and 0 disagreements.

AttributeToSubject, IsFromSubjectAssembly and StripConfigSuffix are unchanged, as is every dataset. The action's inputs are unchanged: it emits the dataset path its own validation step already resolves, passes that to the runner, and adds two summary rows.

The ambiguity metric did change, because this gave it two new sources. Candidates previously came only from the method signature probe, where more than one match means enumeration order decided the answer. An object record adds a closure scan, which reports how many assemblies also declare a type the dataset had already settled, and a dataset dispute, where nothing was attributed at all. Reporting all three as order-dependent would have told a reader load order explained a finding it had no part in, so the counter splits by source and the source is recorded on the diagnostic rather than inferred from two other fields.

Nothing counted findings dropped at attribution, so a run reporting none because nothing failed and one reporting none because attribution discarded everything were the same output. Both counts are now carried, printed in every state, and a run whose whole population was dropped says so.

Object records carry a type name and nothing else, so attribution has always
had to guess ownership from the namespace prefix. Namespaces are shared, so
that guess cannot be made correct at any precision. The dataset's new `_asm`
field closes the gap; this is the runner half that consumes it.

Three changes:

- `ParseObjectEventAssembly` reads the declaring assembly and type from a new
  event shape, consulted only when the Method event yielded nothing so the
  method path is untouched.
- `ProbeTypeCandidates` collects every loaded assembly declaring a given type,
  so an object record whose type is declared more than once is counted by the
  existing ambiguity metric instead of being normalised away silently.
- No change to year handling. `StripConfigSuffix` already collapses the Revit
  `_20NN` suffix and `IsFromSubjectAssembly` already compares on it, so this
  adds tests rather than code.

Inert until Versioning_Toolkit emits the event: the runner's only view of the
dataset is the TestResult tree, so with nothing emitting, every path is
unchanged. The existing 135 tests pass unmodified.
The dataset-provenance tests and one comment in DatasetProvenance.cs named the
oM assemblies of three private Revit tools. Each of those names is a literal
project directory in a private repository, and a path search on it returns that
repository alone, so carrying it in a public repository identifies it.

Substituted for names that identify nothing, reusing two schemes already in this
repository rather than inventing any: Revit_X_oM_20NN, which SerialiserRunner's
HostDependencyTests already uses as the placeholder for a year-suffixed Revit oM
assembly, and the public Revit_Core_Engine_2022 method-event fixture from
ClosureReclassificationTests. Year suffixes, family distinctness and ordinal
ordering are all preserved, so every assertion tests what it tested before.

DatasetProvenanceTests' header comment claimed the file named only public
assemblies while using one of these names five times. Rewritten to a claim the
file meets.

193 tests pass, unchanged from before the substitution. Eleven mutation checks
ran, five on the map and six on the collector, and each one failed tests and
reverted clean.
… was dropped

Two reporting defects, both of which let a run say something that is not true.

The ambiguity warning describes one population and now covers three. Candidates used
to come only from the method signature probe, where more than one match genuinely
means enumeration order decided the answer. Reading an object record's declaring
assembly from the dataset added two more sources: a closure scan that reports how
many assemblies also declare a type the dataset had already settled, and a dataset
dispute where nothing was attributed at all. The single message asserted enumeration
order for all three, so it could tell someone load order explained a finding it had
no part in. The counter now splits by source and each message says what actually
happened.

The source is recorded on the diagnostic rather than derived. It is inferable today
from Path plus a null check on DeclaringAssembly, which is exactly the coupling worth
avoiding: that inference breaks silently the first time either field moves for an
unrelated reason.

Nothing counted findings dropped at attribution. A dropped leaf returns before it can
produce a diagnostic row, so a run reporting zero findings because nothing failed and
one reporting zero because attribution discarded everything were the same output in
every surface. Both counts are now carried on the coverage record, printed in every
state including zero, and shown as a summary row. A run whose entire population was
dropped now says so, because that result rests on attribution being right rather than
on nothing having failed.

The counter counts leaves dropped at attribution. It is not a diff against a previous
run and the comment says so in both places, because the two coincide on the validation
run and would be easy to conflate.

Also removed: six citations of an issue number in a repository whose issues are
disabled, so the reference resolved to nothing for any reader. One of the sentences
carrying it also named a private repository's assembly, and that went with the rewrite
rather than being left in a line being edited anyway.

201 tests pass, 193 of them pre-existing and unmodified. Five mutation checks ran
against the new behaviour, covering the drop count, its split by basis, and each of
the three candidate sources. Each failed tests and reverted clean.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant