Skip to content

research: hemanth1986's ten held-out Revit 2022 and 2024 to 2027 pairs join the held-out models, counts only (#408) - #1253

Merged
DrunkOnJava merged 2 commits into
mainfrom
research/408-hemanth-holdout-models
Nov 2, 2026
Merged

DrunkOnJava merged 2 commits into
mainfrom
research/408-hemanth-holdout-models

Conversation

@DrunkOnJava

Copy link
Copy Markdown
Owner

Summary

hemanth1986's ten held-out pairs (#408) join research/holdout-models.tsv: a two-storey house and a one-storey office, each saved by Revit 2022, 2024, 2025, 2026 and 2027 and exported by the same release ("IFC4 Reference View [Architecture]", base quantities on). They are pinned at commit b9499a43 of hemanth1986/rvt-holdout-models, each file by sha256 and size from the repository's SHA256SUMS.

Rights and handling

  • The modelling is CC BY 4.0 (credit "hemanth1986"). The README says the door, window and furniture families are Autodesk's template content, which that licence does not cover.
  • So they are handled exactly like the other held-out models: fetched from the contributor's repository for a run, never committed, and only summary counts leave the runner (tools/ci/scrub-holdout-results.sh).
  • The repository was created for Held-out validation: measure on licensed models not used to develop the decoder #408 in one commit, and its README states the models were built by a Revit add-in and never used for rvt-rs work.

First measurement, before any fix

Measure 41542881928, main at c8620bd, -f holdout=true -f models="holdout-rvt-holdout-*", counts only:

model Revit's classes: written as Revit's / absent GlobalId = Revit's properties equal / different / missing / written that Revit omits quantities equal / different / missing / written that Revit omits
house 2022 41 of 42 / 1 (the roof) 37 109 / 0 / 104 / 0 16 / 0 / 210 / 0
house 2024 90 of 90 / 0 86 298 / 1 / 76 / 0 87 / 1 / 234 / 0
house 2025 90 of 90 / 0 86 299 / 0 / 77 / 0 89 / 1 / 232 / 0
house 2026 90 of 90 / 0 86 209 / 0 / 143 / 0 21 / 1 / 300 / 0
house 2027 90 of 90 / 0 86 209 / 0 / 145 / 0 17 / 1 / 349 / 0
office 2022 37 of 38 / 1 (the roof) 33 105 / 0 / 59 / 0 10 / 2 / 173 / 0
office 2024 77 of 77 / 0 73 239 / 0 / 51 / 0 74 / 3 / 186 / 0
office 2025 77 of 77 / 0 73 239 / 0 / 52 / 0 74 / 3 / 186 / 0
office 2026 77 of 77 / 0 73 186 / 0 / 96 / 0 14 / 3 / 246 / 0
office 2027 77 of 77 / 0 73 186 / 0 / 96 / 0 10 / 3 / 292 / 0

These stay held out: no fix is to be developed against them, and a rule that writes new values runs -f holdout=true before it lands, as with the others.

Testing

  • N/A: research data only. The fetch script checks each file against its pinned sha256 and size, and run 41542881928 fetched and verified all twenty.

Audit-honesty checklist

  • README / docs reflect the current state after this PR
  • N/A: changelog fragment (no user-visible behaviour)
  • Commit messages describe what changed and why
  • No emojis in code, docs, or commit messages
  • No private paths, personal email addresses, or internal jargon in committed files

Legal and contribution hygiene

  • Apache-2.0
  • No Autodesk proprietary content committed (the files are fetched, never stored)
  • No fixtures

Breaking changes

None.

@DrunkOnJava DrunkOnJava added area:corpus Work in the corpus area. type:research Reverse engineering, discovery, or design investigation. labels Nov 2, 2026
@DrunkOnJava
DrunkOnJava enabled auto-merge (squash) November 2, 2026 23:15
@DrunkOnJava
DrunkOnJava merged commit a2cec9d into main Nov 2, 2026
27 checks passed
@DrunkOnJava
DrunkOnJava deleted the research/408-hemanth-holdout-models branch November 2, 2026 23:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:corpus Work in the corpus area. type:research Reverse engineering, discovery, or design investigation.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant