diff --git a/README.md b/README.md index 4318d965..372aeb1f 100644 --- a/README.md +++ b/README.md @@ -89,7 +89,7 @@ Every deploy checks the viewer's privacy posture in two ways. `viewer/scripts/au The full list, with each gap's status and evidence, is in [`docs/evidence.md`](docs/evidence.md#what-does-not-work-yet). In brief: - Files from before Revit 2008 are not read, and 2008 to 2014 files are experimental, measured on local files only and outside the 1.0 range. -- Revit 2015 to 2022 and 2027 projects are experimental: measured against the same models saved by a neighbouring release, not against Revit's own export (one 2015 project aside). Revit 2026 is experimental too, measured on one project. +- Revit 2015 to 2022 and 2027 projects are experimental: measured against the same models saved by a neighbouring release, not against Revit's own export (one 2015 project, RE-198, and one 2018 project, RE-315, aside). Revit 2026 is experimental too, measured on one project. - On Revit 2008 to 2023, 2026 and 2027 projects an element's body is its bounding box, cut back at its joins where they are read. On 2024 and 2025 it is the box only where the bytes are not decoded: roofs by extrusion outside Revit 2025, hip roofs over a T or U footprint, monolithic spiral stair flights and straight riserless ones with a nosing, edited wall profiles, ducts and pipe fittings. From 2023, doors and windows are drawn from Revit's saved meshes where those decode whole; furniture and other loadable families stay boxes unless `--saved-meshes` is given. - `rvt-elements` and Python list each element's saved parameters on Revit 2023 to 2027 files, experimentally on 2015 to 2022 ones and not on 2008 to 2014, but built-in parameters have no name, a wall's or floor's type parameters are not joined to it, and only `rvt-ifc --revit-property-sets` writes the rest into IFC. - Family files give metadata, previews and types but no family geometry, and template files (`.rte`, `.rft`) are not measured, because no template is in any corpus. diff --git a/ROADMAP.md b/ROADMAP.md index 86cf5566..0dbe7d8e 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -28,8 +28,9 @@ zero-upload browser viewer. Revit 2023 projects get a narrower version of the same, and Revit 2015 to 2022, 2026 and 2027 projects are experimental. Every recovery claim is measured against the oracle its report names: Revit's own IFC export of the same model wherever one is measured, and for 2015 to 2018, -2019 to 2022 and 2027, which have no measured same-release export (except one -2015 project, RE-198), the same +2019 to 2022 and 2027, which have no measured same-release export (except two +projects, BIMcollab's 2015 example, RE-198, and Vitoria's 2018 `ALD26 Before`, +RE-315), the same sample projects saved by Revit 2019, 2023 and 2026 (but 2015's `rst_basic`, another model than its 2019 copy). What is not decoded diff --git a/changelog.d/fixed/1261-32bit-second-prologue.md b/changelog.d/fixed/1261-32bit-second-prologue.md new file mode 100644 index 00000000..1703b87c --- /dev/null +++ b/changelog.d/fixed/1261-32bit-second-prologue.md @@ -0,0 +1,34 @@ +- **A Revit 2008 to 2023 project whose element frames carry no ElementId of + their own now exports those elements, on their storeys (measured on one + Revit 2018 project).** Revit 2018's + `ALD26 Before` exported none of the 1,394 elements and spaces in Revit's own + export (#1261): most of its frames in `Partitions/0` do not open with a + declared `u32` ElementId, but sit inside their element's partition record, as + 2024 and later frames do (RE-35). Those records are `u32` ElementId, `u32` + size, the tag of `ElementHeader`, a count, the body and a 12-byte trailer that + echoes the size; the 2024 walk's flag test stops at the first record with + another flag word, after 115 of this file's 12,626. A placed-instance frame + with no declared id now takes the id of the declared record around it when + its own reference list names that id, and so does a Level frame that is a + `Level` with a name block in its stream, which gives the file its 7 storeys. + The file now writes 1,136 of Revit's 1,321 elements by ElementId, 1,127 of + them in Revit's class or IFC4's for its IFC2X3 one (9 curtain walls come out + as walls), none that Revit's export lacks, 1,073 on Revit's storey. A curtain + panel frame that opens with a count rather than `ff ff ff ff` is left out, + since 96 of the 102 on that file are the shells of panels Revit does not + write. The 185 elements still missing are 178 conduits, conduit fittings and + gutters, whose categories are not yet recovered, 4 panel walls and 3 + assemblies (RE-315). Files whose frames all open with their id export as + before. +- **Revit 2016 to 2023 exports leave out the elements Revit's own export + leaves out by phase and by design option.** Only 2024 and later applied the + export phase (RE-184) and the primary design option (RE-40). A Revit 2018 + project wrote 198 pipes, fittings and other elements that Revit's export does + not hold, and writes none now; the counts cannot say whether the phase or the + design options removed them, and the held-out file is not opened to find + out. The phase is read as RE-281 reads it. Design options are read as on 2024 + with `u32` ids: on the `rst_basic_sample_project` of each of 2016 to 2023 the + one `Mass` in a non-primary option is found and counted, as the 2024 and 2025 + copies of the model count it, and leaves the production counts of the + diagnostics without changing the exported IFC. Revit 2008 to 2015 keep every + element (RE-315). diff --git a/docs/evidence.md b/docs/evidence.md index cdeec1b7..52206873 100644 --- a/docs/evidence.md +++ b/docs/evidence.md @@ -15,9 +15,9 @@ The recovery figures in this README compare rvt-rs's export with Revit's own IFC | Revit 2024 and 2025 projects | yes | typed IFC entities from each element's own record; by ElementId the set equals Revit's export on Core Interior, RE1 Architecture, Electrical and Mechanical, the 22 of puzzbobb's Revit 2025 models that have an export, and the BIM4LCA office as built; RE1 Plumbing and the office at building permit each write one element Revit's export leaves out, and 23 of Snowdon Towers' 5,968 are not in it (local only; details below) | measured per category: exact joins, profiles and cuts where decoded, the element's bounding box where not | names, family and type names, storey, materials and layers, Revit's GlobalId, stair dimensions, IFC export overrides; most other parameters not read | | Revit 2026 projects | yes | **experimental (since 0.5.0):** typed IFC entities from each element's record (RE-124), on flowbim.ee's single-dwelling house ([reference models](reference-models.md)) 113 of the 115 elements of Revit's export, none outside it | the element's bounding box, walls cut back at their joins (on that house wall thickness Revit's on 59 of 59, length on 35 of 59) | Revit's GlobalIds (113 of 113), storeys with Revit's names and elevations (4 of 4), rooms with Revit's number and name (13 of 13) and, since RE-314, their stored solid's outline as on 2025, doors and windows in their host walls, family and type names, wall and floor layer sets as thick as Revit's, walls joined by their join lists; roof slopes, stairs and IFC export overrides not read | | Revit 2027 projects | yes; 325 fields of the 2027 schema carry a flag whose meaning is not measured (RE-264) | **experimental (since 0.6.0):** typed IFC entities from each element's record, with the constants the file's schema gives (RE-179); on Autodesk's five 2027 sample projects the same ElementIds and classes as their 2026 copies (20,818 of 20,818), with **no Revit 2027 export measured** | as on 2026 | as on 2026: storeys, rooms (their outline as on 2025, RE-314), doors and windows in their hosts, family and type names, layer sets, wall joins and mullion materials read the same as the 2026 copies; 79 corner mullions are not put in a curtain wall | -| Revit 2023 projects | yes | typed IFC entities (since 0.4.0) from each element's record (RE-81): on two 2023 projects every element of Revit's export (modelo_bim, 37 of 37) or all but the 8 window trims nested in windows (Exemplo_data, 37 of 45), none outside it | the element's bounding box, walls cut back by the walls they join (Exemplo_data 17 of 17 and modelo_bim 4 of 4 as Revit's, RE-120), beams stopped at the faces of the columns they frame into (modelo_bim 8 of 8, RE-122); rooms their outline (RE-102) | ElementId, category, each door's and window's host wall (30 of 30 as Revit's, RE-85), and the storeys of Revit's own Levels with each element on the one its record names, a beam on the one at its top (Exemplo_data 37 of 37 and modelo_bim 37 of 37 in Revit's storey, RE-107, RE-119), each family instance's family and type name (50 of 50 as Revit names them, RE-109), each wall's, floor's and roof's type and system family (24 of 24 with Revit's type, RE-111, RE-112), family instances' materials (44 of 45 as Revit's, RE-113), and layer sets, named by their own or their category's material (RE-114, RE-115): Exemplo_data 36 of 37 and modelo_bim 31 of 37 elements with Revit's materials, and materials with Revit's shading colours (20 of 21, none different, RE-116); no layered or angled joins, design options or IFC export overrides, and the diagnostics say so | +| Revit 2023 projects | yes | typed IFC entities (since 0.4.0) from each element's record (RE-81): on two 2023 projects every element of Revit's export (modelo_bim, 37 of 37) or all but the 8 window trims nested in windows (Exemplo_data, 37 of 45), none outside it | the element's bounding box, walls cut back by the walls they join (Exemplo_data 17 of 17 and modelo_bim 4 of 4 as Revit's, RE-120), beams stopped at the faces of the columns they frame into (modelo_bim 8 of 8, RE-122); rooms their outline (RE-102) | ElementId, category, each door's and window's host wall (30 of 30 as Revit's, RE-85), and the storeys of Revit's own Levels with each element on the one its record names, a beam on the one at its top (Exemplo_data 37 of 37 and modelo_bim 37 of 37 in Revit's storey, RE-107, RE-119), each family instance's family and type name (50 of 50 as Revit names them, RE-109), each wall's, floor's and roof's type and system family (24 of 24 with Revit's type, RE-111, RE-112), family instances' materials (44 of 45 as Revit's, RE-113), and layer sets, named by their own or their category's material (RE-114, RE-115): Exemplo_data 36 of 37 and modelo_bim 31 of 37 elements with Revit's materials, and materials with Revit's shading colours (20 of 21, none different, RE-116); no layered or angled joins or IFC export overrides, and the diagnostics say so; on 2016 to 2023, elements in a non-primary design option or outside the export phase are left out since RE-315 | | Revit 2019 to 2022 projects | yes | **experimental (since 0.6.0):** typed IFC entities from each element's record, which 2019 to 2022 frame as 2023 does, read with the tags each release's schema gives (RE-180); on Autodesk's 2019 to 2022 sample projects every ElementId shared with the 2023 copy has the class it has there (`rac_basic`, `rst_basic` and `rac_advanced` of each release), with **no Revit export of a 2019 to 2022 file measured** | as on 2023 | as on 2023: storeys, doors and windows in their hosts, family and type names, layer sets and materials read as the 2023 copies do, with a few type names and type materials different (RE-180 §6) | -| Revit 2015 to 2018 projects | yes on 2016 to 2018; on 2015, partial: they open on Autodesk's sample projects, and previews and PartAtom are not measured | **experimental (since 0.6.0):** typed IFC entities from each element's record, which 2015 to 2018 frame as 2019 does, read with the tags each release's schema gives (RE-181); on Autodesk's 2016, 2017 and 2018 sample projects every ElementId shared with the 2019 copy has the class it has there (`rac_basic`, `rst_basic` and `rac_advanced` of each release), and on five of its six 2015 sample projects (the sixth is another model), with **Revit's own export measured on one 2015 project and on none of 2016 to 2018**: BIMcollab's structural example (RE-198, a public download, not redistributable) has all 241 of Revit's elements by ElementId, each in Revit's class or IFC4's for its IFC2X3 one, 3 air terminals Revit's export leaves out (#879), 16 footings on no storey and none of its elements' GlobalIds Revit's (#878); 2014's element records are read as 2015's are, measured on one local 2014 file only (RE-259, the row below) | as on 2019 | as on 2019: storeys, doors and windows in their hosts, family and type names, layer sets and materials read as the 2019 copies do, but where the files' own bytes differ (RE-181 §6) | +| Revit 2015 to 2018 projects | yes on 2016 to 2018; on 2015, partial: they open on Autodesk's sample projects, and previews and PartAtom are not measured | **experimental (since 0.6.0):** typed IFC entities from each element's record, which 2015 to 2018 frame as 2019 does, read with the tags each release's schema gives (RE-181); on Autodesk's 2016, 2017 and 2018 sample projects every ElementId shared with the 2019 copy has the class it has there (`rac_basic`, `rst_basic` and `rac_advanced` of each release), and on five of its six 2015 sample projects (the sixth is another model), with **Revit's own export measured on one 2015 project and one 2018 project**: BIMcollab's structural example (RE-198, a public download, not redistributable) has all 241 of Revit's elements by ElementId, each in Revit's class or IFC4's for its IFC2X3 one, 3 air terminals Revit's export leaves out (#879), 16 footings on no storey and none of its elements' GlobalIds Revit's (#878); BIM-Speed's Vitoria `ALD26 Before` (2018, CC BY 4.0), most of whose frames carry no ElementId of their own and sit in their element's record (RE-315), has 1,136 of Revit's 1,321 elements by ElementId, 1,127 of them in Revit's class or IFC4's for its IFC2X3 one (9 curtain walls are written as walls), none written that Revit's export leaves out, 1,073 on Revit's storey, and 7 storeys, as many as Revit's; the 185 missing are 178 conduits, conduit fittings and gutters, whose categories are not recovered, 4 panel walls and 3 assemblies, and its held-out sibling writes 1,336 elements and spaces against Revit's 1,520, counts only, none of its elements one that Revit's export lacks (198 were, until the export phase and design options were applied, which the counts cannot tell apart); 2014's element records are read as 2015's are, measured on one local 2014 file only (RE-259, the row below) | as on 2019 | as on 2019: storeys, doors and windows in their hosts, family and type names, layer sets and materials read as the 2019 copies do, but where the files' own bytes differ (RE-181 §6) | | Revit 2011 to 2014 projects | yes: 2011 to 2013 open since RE-259, whose schemas have no GUID lists; measured on local closed-source files only, outside the 1.0 range (2015 to 2027) | **experimental:** typed IFC entities from each element's record, which 2011 to 2014 frame as 2015 does, read with the tags each release's schema gives (RE-259); on a local copy of Autodesk's 2013 `rac_advanced` sample every ElementId shared with its 2015 copy has the class it has there and 62,071 of 62,153 fields agree, with **no Revit export of a file before 2015 measured** and no same-model pair for 2011, 2012 or 2014 | as on 2015 | as on 2015: storeys, doors and windows in their hosts, family and type names, layer sets and materials read as the 2015 copy does, but where the files' own bytes differ (RE-259 §7) | | Revit 2008 to 2010 projects | yes: they open and report their release, which 2008 and 2009 take from their build date (RE-259); measured on local closed-source files only, outside the 1.0 range (2015 to 2027) | **experimental:** typed IFC entities from each element's record, which 2009 and 2010 frame as 2011 does and 2008 4 bytes shorter (RE-260); on one local 2009 project 8 doors and their wall, on one 2008 project one generic model, with **no Revit export and no same-model copy of any 2008 to 2010 file measured** | as on 2015 | storeys with their names and elevations (RE-260, also on 2011 and 2012), materials named and coloured; type names, layer sets and hosts as on 2015 but not measured | | Families (`.rfa`) | yes | family metadata, OmniClass, previews; no family geometry | none | a family's types and type parameters as its `PartAtom` stream lists them (`rvt-info`, `PartAtom::family_types`) | @@ -109,7 +109,7 @@ All of the above were used to develop the decoder. Three models that were not, A | Gap | Status | Evidence | |---|---|---| -| Element extraction from real `.rvt` project files | **partial** | Revit 2024 and 2025 projects decode their element records into typed IFC elements; see [What rvt-rs reads from real projects](#what-rvt-rs-reads-from-real-projects). Revit 2023 projects decode their element records into typed IFC elements with the joins, storeys, names, types, materials and layer sets listed [by input](#what-you-can-rely-on-by-input) (RE-81 onward), and Revit 2026 projects their element records experimentally (since 0.5.0), measured on one project (RE-124). Revit 2015 to 2022 and 2027 projects decode their element records experimentally (since 0.6.0), measured against the same models saved by a neighbouring release (RE-179 to RE-181), and against Revit's own export on one 2015 project only (RE-198). Revit 2008 to 2014 projects decode their element records experimentally, outside the 1.0 range, on local files only (RE-259, RE-260). Files from before Revit 2008 are not read. The per-finding history, RE-21 to RE-43, is in [`docs/status.md`](status.md) and `reports/element-framing/`. | +| Element extraction from real `.rvt` project files | **partial** | Revit 2024 and 2025 projects decode their element records into typed IFC elements; see [What rvt-rs reads from real projects](#what-rvt-rs-reads-from-real-projects). Revit 2023 projects decode their element records into typed IFC elements with the joins, storeys, names, types, materials and layer sets listed [by input](#what-you-can-rely-on-by-input) (RE-81 onward), and Revit 2026 projects their element records experimentally (since 0.5.0), measured on one project (RE-124). Revit 2015 to 2022 and 2027 projects decode their element records experimentally (since 0.6.0), measured against the same models saved by a neighbouring release (RE-179 to RE-181), and against Revit's own export on one 2015 project (RE-198) and one 2018 project (RE-315) only. Revit 2008 to 2014 projects decode their element records experimentally, outside the 1.0 range, on local files only (RE-259, RE-260). Files from before Revit 2008 are not read. The per-finding history, RE-21 to RE-43, is in [`docs/status.md`](status.md) and `reports/element-framing/`. | | 81 registered classes wired into walker | partial | `MVP_TYPED_CLASSES` are preferred via their registered decoder in `iter_elements` (fail closed). The remaining registry entries still use generic `decode_instance`. ArcWall uses a separate partition decoder. | | Element geometry beyond what is listed above | partial | Where the bytes are not decoded, a body is the element record's bounding box, and the export diagnostics say so. Doors and windows are drawn by default from the mesh Revit saved for display, on Revit 2023 to 2027, where the whole mesh decodes (every vertex within 0.001 ft of Revit's on all 132 of Core Interior's doors and 6 windows and all 269 of the BIM4LCA office's windows, Measure run 40932136360; `--no-saved-meshes` keeps their box). Other loadable families stay boxes by default, turned with the instance where it is turned (RE-87); `--saved-meshes` draws them from their saved mesh too (RE-78: within 0.01 ft of Revit's export for 158 of 158 on Core Interior, 207 of 214 on RE1 (Revit 2025) and 3,332 of 3,736 on Snowdon Towers; RE1 Plumbing's 30 elbows and 8 transitions come out 1.08 to 1.32 of Revit's volume with every Revit vertex on the mesh, because Revit's export of them is a polyhedron of 6 to 10 sides round the pipe; its 10 tees and 4 closet bends are 1.11 to 1.17, with some of Revit's vertices at their junctions 0.006 to 0.02 ft off the mesh, RE-311); since 0.9.0 a saved mesh any part of which does not decode is not drawn, and its element keeps its box (RE-224), which `rvt-ifc` counts as "kept their box, their saved graphics not decoding whole"); ducts and pipe fittings, though pipes are cylinders where their ends are proven (RE-131); openings across and in height, which are the door's or window's box cut to the wall's depth (#227). Not decoded: roofs by extrusion outside Revit 2025 (RE-211) and hip roofs over a T or U footprint (#884), monolithic spiral stair flights and straight riserless ones with a nosing (#357), edited wall profiles and some wall ends (#358: both ends match Revit on 724 of Snowdon's 1,078 walls), the height of 78 of the 579 BIM4LCA office walls Revit's export gives a `Height`, which Revit's export draws cut at one end, mostly at a floor the wall's top is joined to, and rvt-rs as tall as their record box (#1067, RE-257, RE-258), and the boundaries of 7 of Snowdon's 54 rooms, whose stored solids do not close at the floor (#90). A slab sketched as separate pieces exports one element per piece, as Revit's export does (#331). | | Rooms in an earlier phase | not applied | Revit's exporter leaves out a room whose own phase parameter (`ROOM_PHASE`) is not the export phase, and exports an area or an MEP space whatever its phase (revit-ifc `ElementFilteringUtil.IsRoomInInvalidPhase`, called from `Exporter.ExportSpatialElements`). No model here shows that parameter to be the room's created phase, so rvt-rs checks no room against the phase and writes a room created in an earlier phase than the last, which Revit's export may leave out (RE-184) | diff --git a/docs/export-diagnostics.md b/docs/export-diagnostics.md index d16ceaad..cf80eec6 100644 --- a/docs/export-diagnostics.md +++ b/docs/export-diagnostics.md @@ -125,7 +125,10 @@ reads the primary from the set's name entry and does the same. These are omitted by design too, and do not count toward `unexported_element_records`. A set whose primary cannot be read keeps all its options. On Snowdon Towers the count is 27 (9 floors, 9 slab edges, 8 walls, one generic model), none of -which Revit's export holds (RE-40). +which Revit's export holds (RE-40). On Revit 2016 to 2023 the item is +reported too (RE-315): Autodesk's `rst_basic_sample_project` of each of those +releases counts one `Mass`, which Revit does not export, so its `Mass` leaves +the production class counts and the exported IFC does not change. `element_record_not_in_export_phase` counts placed instances of a recovered category that do not exist in the export phase: created after it, or @@ -135,7 +138,10 @@ and they do not count toward `unexported_element_records`. Rooms are never counted or left out, because Revit's exporter skips a room by its own phase parameter, which is not measured to be its created phase. On Snowdon Towers Architectural (2024) the count is 6 (Floor), the slabs of #328, which are -created and demolished in the Legends phase (RE-184). A read of the phases +created and demolished in the Legends phase (RE-184). On Revit 2016 to 2023 +the same filter applies (RE-315); no Autodesk sample has an element outside +its last phase, and the held-out Vitoria Project's counts are the only +evidence that it is right on a 32-bit file. A read of the phases that fails on a Revit 2015 to 2027 file is reported in `warnings` ("The project's phases could not be read"), and the export then writes every element. diff --git a/docs/format.md b/docs/format.md index 59d235c4..16811be6 100644 --- a/docs/format.md +++ b/docs/format.md @@ -14,7 +14,7 @@ If you are building BIM or AEC tooling and want an Apache-2.0 Revit reader to bu - **Revit 2024 and 2025 projects** — typed elements with Revit's ElementIds, names, types, storeys, materials, layers and GlobalIds, as IFC4, glTF 2.0, plan SVG and CSV, each export with a diagnostics sidecar naming what was approximated or left out. - **Revit 2023 projects** — typed elements with Revit's ElementIds and categories, each drawn as its bounding box (RE-81), walls cut back at their joins (RE-120), beams at their columns (RE-122), rooms as their outline (RE-102), on the storeys of Revit's own Levels (RE-107), family instances named as Revit names them (RE-109), walls, floors and roofs with their type, read from its layers (RE-111, RE-112). - **Revit 2026 projects, experimentally (since 0.5.0)**: typed elements, storeys, rooms, doors and windows, type names and layer sets, measured on one project (RE-124). -- **Revit 2015 to 2022 and 2027 projects, experimentally (since 0.6.0)**: typed elements, measured against the same sample projects saved by Revit 2019 (for 2015 to 2018), 2023 (for 2019 to 2022) and 2026 (for 2027); Revit's own export is measured on one 2015 project (RE-198) and on none of the others (RE-179 to RE-181). +- **Revit 2015 to 2022 and 2027 projects, experimentally (since 0.6.0)**: typed elements, measured against the same sample projects saved by Revit 2019 (for 2015 to 2018), 2023 (for 2019 to 2022) and 2026 (for 2027); Revit's own export is measured on one 2015 project (RE-198) and one 2018 project (RE-315) and on none of the others (RE-179 to RE-181). - **Revit 2008 to 2014 projects, experimentally and outside the 1.0 range (2015 to 2027)**: typed elements, measured on a local copy of Autodesk's 2013 `rac_advanced` sample against its 2015 copy (RE-259); 2008 to 2010 only on their own files' consistency, with storeys and named materials (RE-260). The pre-2015 files are closed source and measured locally only. - **Not yet**: element records of Revit 2007 and earlier, most parameters, family geometry, and semantic editing of a Revit file. diff --git a/docs/reference-models.md b/docs/reference-models.md index 9db294e5..71a45432 100644 --- a/docs/reference-models.md +++ b/docs/reference-models.md @@ -34,8 +34,8 @@ lists the local-only files with their source and hash, and never fetches them. | 255ribeiro Projeto1, `teste_export_2025` | 2025 | none declared, local only | IFC4X3 CoordinationView; IFC4 DesignTransferView 1.0, Revit 25.4 (PTB) | metric projects; IFC export overrides; files saved in `PTB` (RE-123) | | 255ribeiro `Exemplo_data`, `modelo_bim` | 2023 | none declared, local only | IFC4 ReferenceView 1.2, Revit 23.1 (`Exemplo_data`); IFC4 DesignTransferView 1.0, Revit 26.2 PTB (`modelo_bim`, exported from the 2023 file by a later Revit) | Revit 2023 element records (RE-81 to RE-122): 37 of 37 and 37 of 45 elements, storeys, types, layers, materials, wall joins, beams; files saved in `PTB` | | BIMcollab example, structural model (`02_BIMcollab_Example_STR`) | 2015 | none stated, local only; BIMcollab's public example project (`research/bimcollab-models.tsv`, fetched by Measure with `-f bimcollab=true`, STE1200's pointer) | IFC2X3 CoordinationView 2.0, Revit Architecture 2015, 2014-10-08, rewritten by Solibri's IFC Optimizer; the model was renamed after it | held out when measured (RE-198); RE-208 and RE-220 have since used it, so it no longer counts as held out: all 241 exported elements by ElementId, in Revit's class or IFC4's for its IFC2X3 one, 3 more written, GlobalIds not in the file (#878, RE-208), 16 footings on no storey (#879); wall faces and layer sets as Revit's | -| BIM-Speed Vitoria (`ALD26 Before`, `ALD26 Project`), Frigento, Gdynia, Warmond | 2018, 2018, 2020, 2021, 2021 | CC BY 4.0 (TU Berlin DepositOnce, doi:10.14279/depositonce-16441); fetched by Measure with `-f holdout=true` (`research/holdout-archive-models.tsv`) | each release's own export: IFC2X3 CoordinationView 2.0 (Vitoria, Gdynia), IFC4 DesignTransferView (Frigento), IFC4 passed through BIMServer.center's preprocessor (Warmond); the IFC2X3 exports and Warmond's are counted apart and never remove a value | **held out** (Griffin, 2026-11-03): measured before any fix, counts only; the first same-release pairs for 2018, 2020 and 2021 (#408, #1261). Before any fix (run 41658225355): both Vitoria saves export no element against Revit's 1,394 and 1,520. Every Autodesk 2018 sample exports its elements, so `ALD26 Before` is spent to find why (measured in full as `spent-bimspeed-vitoria-before-2018`) and `ALD26 Project` stays held out as the check; Frigento 151 of 151 elements paired by Tag, Gdynia 199 of 203 (18 of 19 spaces), Warmond 710 of 710 (179 of 179 spaces); an element is counted as found by its Tag, whatever its class | -| BIM Whale `AdvancedProject` | 2021 | the IFC from andrewisen/bim-whale-ifc-samples (MIT); the `.rvt` from the Dropbox link its README gives, under the README's "feel free to use these files however you like"; fetched by Measure with `-f holdout=true` | IFC2X3 CoordinationView 2.0, exporter 21.1.10.26, counted apart | **held out** (Griffin, 2026-11-03): before any fix it had 1,698 of 1,714 elements paired by Tag (456 of them in another class than Revit's: 407 `IfcFurniture` for `IfcFurnishingElement`, 49 `IfcSanitaryTerminal` for `IfcFlowTerminal`) and none of its 8 spaces. Those spaces are Revit Areas of a 'Rentable' scheme, not rooms; rvt-rs reads Areas on Revit 2024 only. It was briefly spent to look for the cause and restored: measured in full once (run 41661841411, its artifact deleted), with only its room probes, diagnostics and the spaces' category read (#1261) | +| BIM-Speed Vitoria (`ALD26 Before`, `ALD26 Project`), Frigento, Gdynia, Warmond | 2018, 2018, 2020, 2021, 2021 | CC BY 4.0 (TU Berlin DepositOnce, doi:10.14279/depositonce-16441); fetched by Measure with `-f holdout=true` (`research/holdout-archive-models.tsv`) | each release's own export: IFC2X3 CoordinationView 2.0 (Vitoria, Gdynia), IFC4 DesignTransferView (Frigento), IFC4 passed through BIMServer.center's preprocessor (Warmond); the IFC2X3 exports and Warmond's are counted apart and never remove a value | **held out** (Griffin, 2026-11-03): measured before any fix, counts only; the first same-release pairs for 2018, 2020 and 2021 (#408, #1261). Before any fix (run 41658225355): both Vitoria saves export no element against Revit's 1,394 and 1,520. Every Autodesk 2018 sample exports its elements, so `ALD26 Before` is spent to find why (measured in full as `spent-bimspeed-vitoria-before-2018`) and `ALD26 Project` stays held out as the check. After RE-315 (frames with no ElementId of their own take their record's, run 41707934595): `ALD26 Before` writes 1,136 of Revit's 1,321 elements by Tag, 1,127 of them in Revit's class or IFC4's, with none that Revit's export lacks, on 7 storeys; `ALD26 Project` first wrote 1,534 elements and spaces against Revit's 1,520, counts only, 198 of them elements Revit's export does not hold, and writes 1,336 (1,263 elements and 73 spaces) with none, since the export phase and design options are applied to Revit 2016 to 2023 (the counts cannot tell which of the two removed them); Frigento, Gdynia and Warmond export byte for byte as before. Before any fix, Frigento 151 of 151 elements paired by Tag, Gdynia 199 of 203 (18 of 19 spaces), Warmond 710 of 710 (179 of 179 spaces); an element is counted as found by its Tag, whatever its class | +| BIM Whale `AdvancedProject` | 2021 | the IFC from andrewisen/bim-whale-ifc-samples (MIT); the `.rvt` from the Dropbox link its README gives, under the README's "feel free to use these files however you like"; fetched by Measure with `-f holdout=true` | IFC2X3 CoordinationView 2.0, exporter 21.1.10.26, counted apart | **held out** (Griffin, 2026-11-03): before any fix it had 1,698 of 1,714 elements paired by Tag (456 of them in another class than Revit's: 407 `IfcFurniture` for `IfcFurnishingElement`, 49 `IfcSanitaryTerminal` for `IfcFlowTerminal`) and none of its 8 spaces. Those spaces are Revit Areas of a 'Rentable' scheme, not rooms; rvt-rs reads Areas on Revit 2024 only. It was briefly spent on 2026-11-03 to look for the cause and restored. While spent it was measured in full in four runs (41661841411, 41668542833, 41671242397 and 41672953553, their artifacts deleted), and the RE-315 work read its record chain and frame counts beside Vitoria's; no rule or constant came from it (#1261, #1286) | ## Candidates, not yet measured here diff --git a/docs/release-1.0-criteria.md b/docs/release-1.0-criteria.md index 50e2b8a8..fe3f010b 100644 --- a/docs/release-1.0-criteria.md +++ b/docs/release-1.0-criteria.md @@ -37,7 +37,7 @@ only in a pull request that links the new evidence. | | Requirement | Bar | Measured now | Status | |---|---|---|---|---| | W1 | A new user installs it | Every channel (crates.io, the PyPI wheels, the CLI archives, the container image, the viewer) is checked from what was published, as [Post-publish install verification](../RELEASE.md#post-publish-install-verification) describes | All five checked from what was published for v0.12.0 ([`install.md`](install.md)); `cargo install` from crates.io itself was not run, only on the tagged source in the publish workflow | **passed** for 0.12.0, but for that install; repeated at the 1.0 tag | -| W2 | Every release from 2015 to 2027 is supported | No release in that range is experimental. Each has at least one project with Revit's own IFC export of that release that meets W4, W6 and W7 | 2024 has exact element sets against a same-release export (Core Interior, RE-21). 2025's same-release evidence is puzzbobb's Revit 2025 models (22 with a Revit export, all exact by ElementId, Measure only) and `teste_export_2025` (a Revit 25.4 export, local only); RE1's exports are by Revit 26.2, a cross-release pair (RE-32). 2023 has 37 of 37 and 37 of 45, the other 8 being window trims nested in families (RE-81). 2026 has 113 of 115 on one project (RE-124). 2015 has one: BIMcollab's structural example against Revit 2015's own export, all 241 of its elements by ElementId, in Revit's class or IFC4's for its IFC2X3 one, plus 3 it leaves out ([RE-198](../reports/element-framing/RE-198-revit-2015-bimcollab-export.md), [#879](https://github.com/DrunkOnJava/rvt-rs/issues/879)). 2016 to 2022 and 2027 have no same-release export, and are measured only against the same models saved by a neighbouring release (RE-179 to RE-181, [#421](https://github.com/DrunkOnJava/rvt-rs/issues/421)); the native readers on 2015 to 2022 are [#1158](https://github.com/DrunkOnJava/rvt-rs/issues/1158), and geometry and properties beyond each element's box on 2015 to 2023, 2026 and 2027 are [#1190](https://github.com/DrunkOnJava/rvt-rs/issues/1190) | **failed** | +| W2 | Every release from 2015 to 2027 is supported | No release in that range is experimental. Each has at least one project with Revit's own IFC export of that release that meets W4, W6 and W7 | 2024 has exact element sets against a same-release export (Core Interior, RE-21). 2025's same-release evidence is puzzbobb's Revit 2025 models (22 with a Revit export, all exact by ElementId, Measure only) and `teste_export_2025` (a Revit 25.4 export, local only); RE1's exports are by Revit 26.2, a cross-release pair (RE-32). 2023 has 37 of 37 and 37 of 45, the other 8 being window trims nested in families (RE-81). 2026 has 113 of 115 on one project (RE-124). 2015 has one: BIMcollab's structural example against Revit 2015's own export, all 241 of its elements by ElementId, in Revit's class or IFC4's for its IFC2X3 one, plus 3 it leaves out ([RE-198](../reports/element-framing/RE-198-revit-2015-bimcollab-export.md), [#879](https://github.com/DrunkOnJava/rvt-rs/issues/879)). 2018 has one, used up for W5: Vitoria's `ALD26 Before` against Revit's own IFC2X3 export, 1,136 of its 1,321 elements by ElementId, 1,127 of them in Revit's class or IFC4's (9 curtain walls come out as walls), none written that the export leaves out, 185 missing, so it does not meet W4 (RE-315). 2016, 2017, 2019 to 2022 and 2027 have no same-release export, and are measured only against the same models saved by a neighbouring release (RE-179 to RE-181, [#421](https://github.com/DrunkOnJava/rvt-rs/issues/421)); the native readers on 2015 to 2022 are [#1158](https://github.com/DrunkOnJava/rvt-rs/issues/1158), and geometry and properties beyond each element's box on 2015 to 2023, 2026 and 2027 are [#1190](https://github.com/DrunkOnJava/rvt-rs/issues/1190) | **failed** | | W3 | Family and template files are read like projects | A family (`.rfa`) or template (`.rte`, `.rft`) of every supported release yields its types, parameters and geometry, measured against Revit's export of the family placed in a project | Metadata, previews, schema and OmniClass, and a family's types and type parameters as its `PartAtom` stream lists them, Revit's text at save rather than the family's records, and not compared with an export of the family placed in a project ([supported profile](supported-profile.md#by-input)) ([#1189](https://github.com/DrunkOnJava/rvt-rs/issues/1189)) | **failed** | | W4 | Correct elements on the models the decoder was developed on | On every reference model, the elements written and their IFC classes equal Revit's export of that model in every category: none missing, none extra | Core Interior (2024): exact in its six instance categories (RE-21, run in CI). RE1 (2025): Architecture 84, Electrical 49 and Mechanical 74 exact by ElementId; Plumbing writes one element Revit's export leaves out, a water closet tank (442378, pinned in `tests/element_records_2025.rs`). puzzbobb's Revit 2025 models: all 22 with an export exact. BIM4LCA office (2024): all 12,974 as built; at building permit one column Revit's export leaves out (1927742), and on both files one fixture in another class (3095244) ([#792](https://github.com/DrunkOnJava/rvt-rs/issues/792)) (Measure run 40927543524, 2026-10-29). Snowdon Towers (2024, local only): 5,945 of 5,968 in Revit's export; the other 23 are curtain panels with no resolved type ([#309](https://github.com/DrunkOnJava/rvt-rs/issues/309)), and the 6 slabs in the Legends phase of [#328](https://github.com/DrunkOnJava/rvt-rs/issues/328) are no longer written (RE-184; Measure run 40930512289: `IfcSlab` 224 to 218, the only change on any CI model; [evidence page](evidence.md#what-rvt-rs-reads-from-real-projects)) | **failed** | | W5 | Correct on models it was not developed on ([#408](https://github.com/DrunkOnJava/rvt-rs/issues/408)) | At least two projects of every supported release, each with Revit's own IFC export of that release, whose first measurement, made before any fix is made for them, meets W4, W6 and W7. A pair whose first measurement does not is used up for W5: it stays in Measure as a held-out regression check, counts only and never developed on, and its counts may be quoted in reports (the maintainer, 2026-11-03). Since no model meets W6 and W7 yet, any pair measured now would be used up, so a fresh pair is pinned sealed: no Measure run fetches it until the final 1.0 check, made once when W4, W6 and W7 pass on the reference models. The results are published as measured | Held-out measurements so far: Snowdon Towers Plumbing, HVAC and Electrical (2024). It scored against a VIM of the 2027 edition rather than Revit's IFC export of these files: all 10,385 exported elements in the expected category, storeys and GlobalIds equal, every body a box ([report](../reports/validation/held-out-snowdon-mep-2026-09-27.md)). The first held-out project with Revit's own export of its release was BIMcollab's Revit 2015 structural example (RE-198), measured before any fix; RE-208 and RE-220 have since used it, so it no longer counts toward this bar, and neither do the holdout pack's models (`research/holdout-models.tsv`), whose first measurements were not recorded against W4, W6 and W7. Since 2026-11-03 Measure also holds out six projects with their release's own export (`research/holdout-archive-models.tsv`, [#1261](https://github.com/DrunkOnJava/rvt-rs/issues/1261)), measured once before any fix, counts only: Vitoria's two Revit 2018 saves export no element against Revit's 1,394 and 1,520, so `ALD26 Before` is spent to find why and `ALD26 Project` stays held out; Frigento (2020) has 151 of 151 elements paired by Tag, Gdynia (2021) 199 of 203, Warmond (2021) 710 of 710, and BIM Whale's `AdvancedProject` (2021) 1,698 of 1,714 with none of its 8 spaces, which are Revit Areas (held out, after being measured in full once). All six fail W4 at that first measurement, so they are used up for W5 and stay as regression checks: both Vitoria saves miss every element, Gdynia misses 4 and BIM Whale 16, its 8 Areas among them, and Frigento writes 22 elements and Warmond 6 spaces that Revit's export leaves out (run 41658225355). No model meets this bar today, and every release needs fresh pairs. BIMcollab's example fails W4: 3 elements written that Revit's export leaves out, and 16 footings on no storey, whose saved Level is unset ([#879](https://github.com/DrunkOnJava/rvt-rs/issues/879), RE-281). It also fails on identity: the project's, site's, building's and five storeys' GlobalIds are Revit's, from the values the file stores ([RE-220](../reports/element-framing/RE-220-stored-global-ids.md)), but none of its 241 elements' ([#878](https://github.com/DrunkOnJava/rvt-rs/issues/878)) | **failed** | diff --git a/docs/rooms.md b/docs/rooms.md index 5cb85cbc..bcf4fb41 100644 --- a/docs/rooms.md +++ b/docs/rooms.md @@ -36,7 +36,7 @@ The export also counts the spaces drawn as their record's box (rooms and family | Release | Outline | Measured against Revit's export | |---|---|---| -| 2008 to 2021 | read from the stored solid where the reader finds one | held-out models (below): two 2021 projects; a Revit 2018 project exports no element at all until PR #1286 lands (#1261) | +| 2008 to 2021 | read from the stored solid where the reader finds one | held-out models (below): two 2021 projects; Vitoria's Revit 2018 `ALD26 Before` (spent, below), whose frames sit in their records (RE-315, #1261) | | 2022 | stored solid | 8 held-out rooms | | 2023 | stored solid | 9 of 9 rooms of one project, local only (RE-102) | | 2024 | stored solid; Areas as spaces without a body | Core Interior, the BIM4LCA office, held-out models (below) | @@ -67,8 +67,9 @@ Measure runs the scorer on every reference model (`-f scorers=space_profiles_vs_ | One held-out house (2026) | 12 of 13 | 10 | 0 | 0 | 12 rooms, stored solid: 10, 0 | | Held-out Gdynia (2021, Revit's IFC2X3 export) | 18 of 19 | 18 | 0 | 0 | 18 rooms, stored solid: 18, 0 | | Held-out Warmond (2021, export passed through another tool) | 179 | 174 | 174 | 174 | 171 rooms, stored solid: 171, 171; 8 rooms, record box: 3, 3 | +| Vitoria `ALD26 Before` (2018, Revit's IFC2X3 export; spent) | 73 | 72 | 72 | 71 | 73 rooms, stored solid: 73, 72 | -A model with no rooms in Revit's export (the other R2025 models, RE1 Electrical, Mechanical and Plumbing) scores 0 of 0. Two held-out models write none of Revit's spaces: Vitoria's 2018 project (73), which exports no element until PR #1286 (#1261), and BIM Whale's 2021 project (8), whose spaces are Areas, which rvt-rs reads on Revit 2024 only. +A model with no rooms in Revit's export (the other R2025 models, RE1 Electrical, Mechanical and Plumbing) scores 0 of 0. One held-out model writes none of Revit's spaces: BIM Whale's 2021 project (8), whose spaces are Areas, which rvt-rs reads on Revit 2024 only. Vitoria's 2018 project exported no element, and none of its 73 spaces, before PR #1286 (#1261): its spent `ALD26 Before` now writes all 73 (run 41707934595, full output, in the row above), and its held-out sibling `ALD26 Project` writes 73 `IfcSpace` against Revit's 73 (counts only, not scored here). Held-out figures are Measure run 41680717786 (`main` at 21cb2efd), counts only. On the 2026 house and on Gdynia, every room's area is right where it is scored, but none of their outlines is within 0.01 ft of Revit's; the files are held out, so why is not examined. diff --git a/docs/status.md b/docs/status.md index 3006c4bc..0634fa98 100644 --- a/docs/status.md +++ b/docs/status.md @@ -79,9 +79,9 @@ measured on one project against Revit's own export (RE-124). On projects saved by Revit 2015 to 2022 and by Revit 2027, experimentally since 0.6.0, it exports their elements, measured against the same sample projects saved by Revit 2019 (for 2015 to 2018, RE-181), 2023 (for 2019 to 2022, RE-180) and -2026 (for 2027, RE-179) rather than against a Revit export of them, except one -2015 project, BIMcollab's structural example, against Revit 2015's own export -(RE-198); 2015's +2026 (for 2027, RE-179) rather than against a Revit export of them, except two +projects against Revit's own export of their release: BIMcollab's 2015 +structural example (RE-198) and Vitoria's 2018 `ALD26 Before` (RE-315); 2015's `rst_basic` is another model than its 2019 copy, so it is not measured. On projects saved by Revit 2011 to 2014, experimentally and outside the 1.0 range (2015 to 2027), it exports their elements too, measured on a local copy diff --git a/docs/support-matrix.json b/docs/support-matrix.json index 05b11881..06239f3c 100644 --- a/docs/support-matrix.json +++ b/docs/support-matrix.json @@ -158,7 +158,7 @@ "name": "Revit 2008 to 2010 partition element records, read in the 32-bit layout of Revit 2011 to 2023, with storeys and named materials", "status": "experimental", "evidence": "partition_element_records_2023::REVIT_32BIT_RELEASES, schema_tags and envelope_shortfall; partition_materials (name_before_properties); partition_level_records::framed_plane_marker; probes/examples/probe_re260_revit_2008_2010.rs; reports/element-framing/RE-260-revit-2008-2010-element-records.md", - "notes": "Revit 2009 and 2010 frame their element records as 2011 does (a declared u32 ElementId 52 bytes before every marker of the local files); Revit 2008's envelope has 12 bytes between the category and the placement kind where later releases have 16, all ff, so its id is 48 bytes before the marker. Each release takes its own tags of the five schema classes RE-180 lists, and 2011 to 2013's shading frame and value blocks (RE-259) hold on 2008 and 2009. A 2008 to 2011 material's name is a u32-counted string followed by three equal f64s, with no PhysicalParamSet tag after it: every material of the local 2008 to 2011 files is now named, and where its own MaterialElemGroupHelper frame names it too the two agree (43 of 43). A 2008 to 2012 Level's elevation follows ff ff ff ff and the Plane tag after its name, one reference slot and 16 bytes, written twice 24 bytes apart: every local 2008 to 2012 file reads its Levels as storeys (Revit's template 0 and 10 ft, and 0 and 4 m on one 2012 project). There is no Revit export and no copy of the same model saved by another release of any 2008 to 2010 file here, and every one is closed source and measured locally only; what is shown is that the records read without a byte left over and that the elements hang together (a 2009 wall and its 8 doors on two storeys, a 2008 causeway as one generic model). Not read, as on 2015 to 2023: design options, layered joins, IFC export overrides, parameters.", + "notes": "Revit 2009 and 2010 frame their element records as 2011 does (a declared u32 ElementId 52 bytes before every marker of the local files); Revit 2008's envelope has 12 bytes between the category and the placement kind where later releases have 16, all ff, so its id is 48 bytes before the marker. Each release takes its own tags of the five schema classes RE-180 lists, and 2011 to 2013's shading frame and value blocks (RE-259) hold on 2008 and 2009. A 2008 to 2011 material's name is a u32-counted string followed by three equal f64s, with no PhysicalParamSet tag after it: every material of the local 2008 to 2011 files is now named, and where its own MaterialElemGroupHelper frame names it too the two agree (43 of 43). A 2008 to 2012 Level's elevation follows ff ff ff ff and the Plane tag after its name, one reference slot and 16 bytes, written twice 24 bytes apart: every local 2008 to 2012 file reads its Levels as storeys (Revit's template 0 and 10 ft, and 0 and 4 m on one 2012 project). There is no Revit export and no copy of the same model saved by another release of any 2008 to 2010 file here, and every one is closed source and measured locally only; what is shown is that the records read without a byte left over and that the elements hang together (a 2009 wall and its 8 doors on two storeys, a 2008 causeway as one generic model). Not read, as on 2015 to 2023: layered joins, IFC export overrides, parameters. Design options and the export phase are not applied before Revit 2016 (RE-315 applies them on 2016 to 2023).", "user_impact": "Outside the 1.0 range (2015 to 2027), measured on the contributor's local closed-source files only. Revit 2008 to 2010 projects export typed elements under Revit's ElementIds, on their storeys, with named materials; checked only for consistency within each file, not against Revit's export. Revit 2011 and 2012 projects gain their storeys and, on 2011, their material names." }, { @@ -167,7 +167,7 @@ "name": "Revit 2011 to 2014 partition element records, read in the 32-bit layout of Revit 2015 to 2023, with storeys, hosts, type names, layer sets and materials", "status": "experimental", "evidence": "basic_file_info (release years from 2008, the build-date release); formats::schema_classes (schemas without GUID lists); partition_element_records_2023::REVIT_32BIT_RELEASES and schema_tags; partition_materials::shading_ids_32; partition_type_materials (2011 to 2013 blocks that hold no map); probes/examples/probe_re259_revit_2011_2014.rs; reports/element-framing/RE-259-revit-2011-2014-element-records.md", - "notes": "Files written before Revit 2014 did not open: BasicFileInfo was read only for a year from 2014 on, and a schema written before 2014 has no GUID list after a class's fields. With both read, every local 2008 to 2013 file opens and its schema reads to the end; 2008 and 2009, which state no year, take the build year plus one. Revit 2011 to 2014 frame their element records as 2015 does: on eight local 2011 to 2014 files every record marker has a declared u32 ElementId 52 bytes before it. Each release takes its own tags of the five schema classes RE-180 lists. Two layouts differ from 2015's on 2011 to 2013: a material's shading frame holds two u32 ElementIds between the 0.5 and the pattern slots, and an ElementId and the value-block mark also open blocks that hold no type's map. 2011 to 2013 Levels hold the instance placement kind; 2014 Levels hold 2015's. There is no Revit export of a file before 2015, and every one measured is closed source and measured locally only: the oracle is a local copy of Autodesk's 2013 rac_advanced sample against the pinned 2015 copy. Every ElementId both hold has its class, and 62,071 of 62,153 fields agree; the rest are in the files' own records (a re-divided curtain wall, a shorter wall, two beams' Level references). 2011, 2012 and 2014 have no same-model pair. Revit 2008 to 2010 files open with their metadata and schema, and their element records are not read. Not read, as on 2015 to 2023: design options, layered joins, IFC export overrides, parameters.", + "notes": "Files written before Revit 2014 did not open: BasicFileInfo was read only for a year from 2014 on, and a schema written before 2014 has no GUID list after a class's fields. With both read, every local 2008 to 2013 file opens and its schema reads to the end; 2008 and 2009, which state no year, take the build year plus one. Revit 2011 to 2014 frame their element records as 2015 does: on eight local 2011 to 2014 files every record marker has a declared u32 ElementId 52 bytes before it. Each release takes its own tags of the five schema classes RE-180 lists. Two layouts differ from 2015's on 2011 to 2013: a material's shading frame holds two u32 ElementIds between the 0.5 and the pattern slots, and an ElementId and the value-block mark also open blocks that hold no type's map. 2011 to 2013 Levels hold the instance placement kind; 2014 Levels hold 2015's. There is no Revit export of a file before 2015, and every one measured is closed source and measured locally only: the oracle is a local copy of Autodesk's 2013 rac_advanced sample against the pinned 2015 copy. Every ElementId both hold has its class, and 62,071 of 62,153 fields agree; the rest are in the files' own records (a re-divided curtain wall, a shorter wall, two beams' Level references). 2011, 2012 and 2014 have no same-model pair. Revit 2008 to 2010 files open with their metadata and schema, and their element records are not read. Not read, as on 2015 to 2023: layered joins, IFC export overrides, parameters. Design options and the export phase are not applied before Revit 2016 (RE-315 applies them on 2016 to 2023).", "user_impact": "Outside the 1.0 range (2015 to 2027), measured on the contributor's local closed-source files only. Revit 2008 to 2013 files open and report their release, metadata and schema. Revit 2011 to 2014 projects export typed elements under Revit's ElementIds, on Revit's storeys, with doors and windows in their walls and named types and layer sets; checked against one 2013 sample saved by Revit 2015, not against Revit's export." }, { @@ -175,9 +175,9 @@ "area": "elements", "name": "Revit 2015 to 2018 partition element records, read in the 32-bit layout of Revit 2019 to 2023, with storeys, hosts, type names, layer sets and materials", "status": "experimental", - "evidence": "partition_element_records_2023::REVIT_32BIT_RELEASES and schema_tags; partition_level_records::LEVEL_PLACEMENT_KIND_2015; partition_compound_structure::wall_orientation_has_word; partition_materials::pattern_slots_32; partition_names::find_element_data_names_32; probes/examples/probe_re181_revit_2015_2018.rs; reports/element-framing/RE-181-revit-2015-2018-element-records.md; reports/element-framing/RE-198-revit-2015-bimcollab-export.md", - "notes": "Revit 2015 to 2018 frame their element records as 2019 does (RE-180): on Autodesk's 2016 to 2018 sample projects every record marker but 4 to 109 per file (as many as on the 2019 copy, but 15 on 2016 rst_basic) has a declared u32 ElementId 52 bytes before it, and so do all but 4 to 6 per file of Autodesk's six 2015 sample projects (pinned by #801) and 0 to 68 per file of three of its 2015 templates; 2014 is not read, since the one 2014 file measured is not documented. Each release takes its own tags of the five schema classes RE-180 lists. Three layouts differ from 2019's: a Level record's placement kind is 0xffffef6f, a wall's flip follows its location line with no word between (as on 2019 and 2020), and a material's shading frame holds two pattern slots, not four. On 2015 to 2018 a type's element-data name is looked for in its own data only and a one-unit entry is passed over. Revit's own export is measured on one 2015 project, BIMcollab's structural example (RE-198, a public download): all 241 of its elements by ElementId, in Revit's class or IFC4's for its IFC2X3 one, 3 written that the export leaves out (#879), and none of its elements' GlobalIds Revit's (#878). For 2016 to 2018 there is no Revit export: the oracle is the same model saved by Revit 2019. On rac_basic, rst_basic and rac_advanced of 2016 to 2018 every ElementId shared with the 2019 copy has its class; 2018 rac_basic and rac_advanced agree on every field, and the other differences are in the files' own bytes (boxes, reference lists, 2016's shading colours, 2016 rst_basic's renamed families). On five of the six 2015 sample projects every ElementId shared with the 2019 copy has its class; the sixth, rst_basic, shares no ElementId with it (another model), and the differing fields (type materials from RE-82's scan, renamed families, curtain-wall member boxes) are counted in RE-181 §6, not explained. Not read, as on 2019 to 2023: design options, layered joins, IFC export overrides, parameters.", - "user_impact": "Revit 2015 to 2018 projects export typed elements under Revit's ElementIds, on Revit's storeys, with doors and windows in their walls and named types and layer sets; checked against the same models saved by Revit 2019, and for one 2015 project against Revit 2015's own export (RE-198)." + "evidence": "partition_element_records_2023::REVIT_32BIT_RELEASES and schema_tags; partition_level_records::LEVEL_PLACEMENT_KIND_2015; partition_compound_structure::wall_orientation_has_word; partition_materials::pattern_slots_32; partition_names::find_element_data_names_32; probes/examples/probe_re181_revit_2015_2018.rs; reports/element-framing/RE-181-revit-2015-2018-element-records.md; reports/element-framing/RE-198-revit-2015-bimcollab-export.md; probes/examples/probe_re315_32bit_record_chain.rs; reports/element-framing/RE-315-32bit-second-prologue-records.md", + "notes": "Revit 2015 to 2018 frame their element records as 2019 does (RE-180): on Autodesk's 2016 to 2018 sample projects every record marker but 4 to 109 per file (as many as on the 2019 copy, but 15 on 2016 rst_basic) has a declared u32 ElementId 52 bytes before it, and so do all but 4 to 6 per file of Autodesk's six 2015 sample projects (pinned by #801) and 0 to 68 per file of three of its 2015 templates; 2014 is not read, since the one 2014 file measured is not documented. Each release takes its own tags of the five schema classes RE-180 lists. Three layouts differ from 2019's: a Level record's placement kind is 0xffffef6f, a wall's flip follows its location line with no word between (as on 2019 and 2020), and a material's shading frame holds two pattern slots, not four. On 2015 to 2018 a type's element-data name is looked for in its own data only and a one-unit entry is passed over. Revit's own export is measured on one 2015 project, BIMcollab's structural example (RE-198, a public download): all 241 of its elements by ElementId, in Revit's class or IFC4's for its IFC2X3 one, 3 written that the export leaves out (#879), and none of its elements' GlobalIds Revit's (#878). Revit's own export is measured on one 2018 project, Vitoria's ALD26 Before (RE-315, CC BY 4.0, spent to find why it exported nothing): 1,136 of Revit's 1,321 elements by ElementId, 1,127 of them in Revit's class or IFC4's (9 curtain walls are written as walls), none written that the export leaves out, 185 missing; for 2016, 2017 and the other 2018 projects there is no Revit export and the oracle is the same model saved by Revit 2019. On rac_basic, rst_basic and rac_advanced of 2016 to 2018 every ElementId shared with the 2019 copy has its class; 2018 rac_basic and rac_advanced agree on every field, and the other differences are in the files' own bytes (boxes, reference lists, 2016's shading colours, 2016 rst_basic's renamed families). On five of the six 2015 sample projects every ElementId shared with the 2019 copy has its class; the sixth, rst_basic, shares no ElementId with it (another model), and the differing fields (type materials from RE-82's scan, renamed families, curtain-wall member boxes) are counted in RE-181 §6, not explained. Not read, as on 2019 to 2023: layered joins, IFC export overrides, parameters. Design options and the export phase are applied on 2016 to 2018 and not on 2015 (RE-315). Design options are read as on 2024 with u32 ids, on the eight saves of Autodesk's rst_basic_sample_project of 2016 to 2023 only: 1 set, 4 frames in a non-primary option, a Mass that Revit does not export, so the exported IFC does not change and no Revit export of these files is measured. The phase is read as RE-281 reads it for 2015 to 2023; no Autodesk sample has an element outside it, so the only evidence that applying it is right on a 32-bit file is the held-out Vitoria Project's counts (198 elements Revit's export lacks, then none), which cannot tell the phase from the design options. A frame with no declared ElementId of its own (the second prologue) takes the id of the declared record around it when it is a placed instance and its reference list names that id (RE-315): on Vitoria's Revit 2018 `ALD26 Before`, which wrote no element before, 1,136 of Revit's 1,321 elements by ElementId, 1,127 of them in Revit's class or IFC4's, none written that its IFC2X3 export leaves out, 1,073 on Revit's storey and 7 storeys, as many as Revit's; the 185 missing are conduits, conduit fittings and gutters in categories not yet recovered (178), 4 panel walls and 3 assemblies.", + "user_impact": "Revit 2015 to 2018 projects export typed elements under Revit's ElementIds, on Revit's storeys, with doors and windows in their walls and named types and layer sets; checked against the same models saved by Revit 2019, and for one 2015 project (RE-198) and one 2018 project (RE-315) against Revit's own export of their release." }, { "id": "element-records-revit-2019-2022", @@ -185,7 +185,7 @@ "name": "Revit 2019 to 2022 partition element records, read in Revit 2023's 32-bit layout, with storeys, hosts, type names, layer sets and materials", "status": "experimental", "evidence": "partition_element_records_2023::REVIT_32BIT_RELEASES and schema_tags; partition_names::element_data_header_32, find_element_data_names_32 and lone_non_ascii; partition_compound_structure::wall_orientation_has_word; partition_materials::scan_materials_32; partition_type_materials::type_material_names_32; probes/examples/probe_re180_revit_2019_2022.rs; reports/element-framing/RE-180-revit-2019-2022-element-records.md", - "notes": "Revit 2019 to 2022 frame their element records as 2023 does (RE-81): on Autodesk's 2019 to 2022 sample projects every record marker but 4 to 11 per file (the same count as the 2023 copy) has a declared u32 ElementId 52 bytes before it, and none at any other distance. Every constant of the 2023 readers is 2023's tag of a schema class (CellList, Material, PhysicalParamSet, PatternHelper, VerticalRegionsStructure), and each release takes its own tag of the same class. Revit 2019 and 2020 write a wall's flip right after its location line, with no word between them. In 32-bit data a one-id list reads as a one-unit name; such names are not taken. There is no Revit export of a 2019 to 2022 file: the oracle is the same model saved by Revit 2023. On rac_basic, rst_basic and rac_advanced of 2019 to 2022 every ElementId shared with the 2023 copy has its class; the field differences are in model edits between releases, RE-113's type-material scan, a faucet and a sink without a type, and type names where the 2024 or 2026 copy sides with 2019 to 2022. Not read, as on 2023: design options, layered joins, IFC export overrides, parameters.", + "notes": "Revit 2019 to 2022 frame their element records as 2023 does (RE-81): on Autodesk's 2019 to 2022 sample projects every record marker but 4 to 11 per file (the same count as the 2023 copy) has a declared u32 ElementId 52 bytes before it, and none at any other distance. Every constant of the 2023 readers is 2023's tag of a schema class (CellList, Material, PhysicalParamSet, PatternHelper, VerticalRegionsStructure), and each release takes its own tag of the same class. Revit 2019 and 2020 write a wall's flip right after its location line, with no word between them. In 32-bit data a one-id list reads as a one-unit name; such names are not taken. There is no Revit export of a 2019 to 2022 file: the oracle is the same model saved by Revit 2023. On rac_basic, rst_basic and rac_advanced of 2019 to 2022 every ElementId shared with the 2023 copy has its class; the field differences are in model edits between releases, RE-113's type-material scan, a faucet and a sink without a type, and type names where the 2024 or 2026 copy sides with 2019 to 2022. Not read, as on 2023: layered joins, IFC export overrides, parameters. Design options and the export phase are applied on 2016 to 2023 (RE-315), with the evidence stated for 2015 to 2018.", "user_impact": "Revit 2019 to 2022 projects export typed elements under Revit's ElementIds, on Revit's storeys, with doors and windows in their walls and named types and layer sets; checked against the same models saved by Revit 2023, not against Revit's export." }, { @@ -203,7 +203,7 @@ "name": "ElementIds for second-prologue partition element records, read from the enclosing partition record", "status": "partial", "evidence": "partition_element_records::partition_record_chain / assign_second_prologue_ids / RevitFile::second_prologue_ids; tests/partition_record_chain.rs (every first-prologue frame starts a record of its own id on Core Interior and the RE1 models); tests/element_records_2025.rs; probes/examples/probe_re35_record_wrapper.rs; reports/element-framing/RE-35-partition-record-wrapper.md; reports/element-framing/RE-43-invalid-element-id.md", - "notes": "Every Partitions/* stream opens with a chain of records, one per element: u64 ElementId, u32 size, u16 prologue constant (release constant + 28), u16 count, the body, and a 16-byte trailer that repeats the size (RE-35). A first-prologue frame starts its record; a second-prologue frame (RE-30) sits inside its element's record, so its ElementId is the record's, read rather than inferred. All 30,432 first-prologue frames across nine 2024 and 2025 files start a record of their own id, and the chains hold every declared id on Core Interior and the RE1 models. Records after the chain belong to loaded families' own documents and never export. Against Revit's own exports: RE1 Architecture, Mechanical and Plumbing reach 100% of every recovered entity; Snowdon Towers Architectural exports 6,045 elements, 5,945 in Revit's export (a frame holding u32::MAX, the invalid ElementId, also takes its record's id, RE-43), with every wall, column, door, window, curtain wall and panel, railing, ceiling, furniture item, fixture and generic model of Revit's export attributed once RE-41 declares the ElemTable's second id. The rest are real elements Revit's export leaves out (#309); the 18 in non-primary design options are left out as Revit leaves them out (RE-40). Floors, building pads and ceilings get their ids too. It replaces RE-34's reference-order inference, which gave a door its type's id on a Revit 2025 file outside its hold-outs. Placed instances with no 3D volume are left out, as Revit leaves them out. Revit 2024 and 2025 only.", + "notes": "Every Partitions/* stream opens with a chain of records, one per element: u64 ElementId, u32 size, u16 prologue constant (release constant + 28), u16 count, the body, and a 16-byte trailer that repeats the size (RE-35). A first-prologue frame starts its record; a second-prologue frame (RE-30) sits inside its element's record, so its ElementId is the record's, read rather than inferred. All 30,432 first-prologue frames across nine 2024 and 2025 files start a record of their own id, and the chains hold every declared id on Core Interior and the RE1 models. Records after the chain belong to loaded families' own documents and never export. Against Revit's own exports: RE1 Architecture, Mechanical and Plumbing reach 100% of every recovered entity; Snowdon Towers Architectural exports 6,045 elements, 5,945 in Revit's export (a frame holding u32::MAX, the invalid ElementId, also takes its record's id, RE-43), with every wall, column, door, window, curtain wall and panel, railing, ceiling, furniture item, fixture and generic model of Revit's export attributed once RE-41 declares the ElemTable's second id. The rest are real elements Revit's export leaves out (#309); the 18 in non-primary design options are left out as Revit leaves them out (RE-40). Floors, building pads and ceilings get their ids too. It replaces RE-34's reference-order inference, which gave a door its type's id on a Revit 2025 file outside its hold-outs. Placed instances with no 3D volume are left out, as Revit leaves them out. Revit 2024 and 2025; on Revit 2008 to 2023, whose ElementId is a u32 in a 12-byte record header, the same idea is RE-315 (RE-35's chain walk stops at the first record whose trailer flag word is neither 0 nor 0x01000000, which these files have), measured against Revit's own export on one Revit 2018 project only, ALD26 Before: 1,136 of Revit's 1,321 elements by ElementId, 1,127 of them in Revit's class or IFC4's, and all 7 of its Levels; no other release of that layout is measured for it.", "user_impact": "Files whose element records mostly lack an ElementId at their start (Autodesk's Snowdon Towers samples, the RE1 models, projects saved incrementally) export their walls, doors, windows, floors, ceilings, columns, curtain-wall parts, railings, furniture, fixtures, ducts and pipes under the ElementIds Revit itself uses; the diagnostics count what is still left out." }, { @@ -248,7 +248,7 @@ "name": "Revit 2023 partition element records, with wall joins, beam cuts, room outlines, storeys, hosts, family and type names, materials and layer sets (RE-81)", "status": "partial", "evidence": "src/research/partition_element_records_2023.rs; partition_schema_mvp::recover_2023_records and nested_in_openings; probes/examples/probe_re81_2023_element_records.rs; reports/element-framing/RE-81-2023-element-records.md; RE-107: partition_level_records::find_level_records_2023 / schema_elevation_marker / OWNER_SENTINEL_RUN_LEN_2023, partition_schema_mvp::select_newest_instance_records, tools/re/storeys_vs_ifc.py, reports/element-framing/RE-107-levels-2023.md; RE-111: partition_names::find_element_data_names_2023, partition_schema_mvp::system_type_names_2023, probes/examples/probe_re111_system_types_2023.rs, reports/element-framing/RE-111-system-types-2023.md; RE-112: partition_compound_structure::layer_layout, probes/examples/probe_re112_layers_2023.rs, reports/element-framing/RE-112-layers-2023.md; RE-114: partition_names::ElementDataLayout / element_data_layout, probes/examples/probe_re114_wall_lines_2023.rs, reports/element-framing/RE-114-wall-layers-2023.md; RE-115: partition_materials::CATEGORY_MATERIAL_FRAME_2023 / scan_category_material, probes/examples/probe_re115_category_materials_2023.rs, reports/element-framing/RE-115-category-materials-2023.md; RE-116: partition_materials::frame_at_2023 / scan_material_appearances, partition_schema_mvp::materials_from_records, tools/re/material_colours_vs_ifc.py, reports/element-framing/RE-116-material-colours-2023.md; RE-117: partition_room_parameters::scan_room_parameter_entries, probes/examples/probe_re117_room_names.rs, reports/element-framing/RE-117-room-names.md; RE-119: partition_schema_mvp::resolve_framing_top_levels, tools/re/storeys_vs_ifc.py, reports/element-framing/RE-119-framing-storeys-2023.md; RE-120: partition_schema_mvp::recover_2023_records, element_record_wall_joins::join_trims, reports/element-framing/RE-120-wall-joins-2023.md; RE-122: element_record_beam_cuts::beam_column_trims, probes/examples/probe_re122_beam_column_refs.rs, probes/examples/probe_re122_beam_supports.rs, reports/element-framing/RE-122-beam-column-cuts.md", - "notes": "A 2023 record sits behind the schema-derived marker (RE-80) with a u32 ElementId 52 bytes and its BuiltInCategory 38 bytes before it, the rest where 2024 has it, and u32 reference lists. Exported with rvt-ifc against Revit's own IFC4 exports of two local 2023 projects: modelo_bim 37 of 37 elements, none outside the export; Exemplo_data 37 of 45, none outside it, the 8 missing being window trims nested in windows, left out with the mullion patterns Revit also leaves out because nothing decoded tells them apart. Every body is the record's bounding box, walls cut back at their joins (RE-120) and beams at their columns (RE-122). Doors and windows bind to their host wall by RE-85's rule on the u32 reference lists: all 19 on Exemplo_data and 11 on modelo_bim take the wall Revit's filled opening voids, and 15 of the 30 openings equal Revit's within 0.01 ft. Not read on 2023: profiles, layered or angled joins, design options, IFC export overrides (4 spread footings Revit writes as IfcSlab come out as IfcFooting); names, types, storeys and materials are read since RE-107 to RE-116 (user_impact). Einhoven (2023, no Revit export) gains 2 walls, 9 curtain panels and 2 windows, unverified.", + "notes": "A 2023 record sits behind the schema-derived marker (RE-80) with a u32 ElementId 52 bytes and its BuiltInCategory 38 bytes before it, the rest where 2024 has it, and u32 reference lists. Exported with rvt-ifc against Revit's own IFC4 exports of two local 2023 projects: modelo_bim 37 of 37 elements, none outside the export; Exemplo_data 37 of 45, none outside it, the 8 missing being window trims nested in windows, left out with the mullion patterns Revit also leaves out because nothing decoded tells them apart. Every body is the record's bounding box, walls cut back at their joins (RE-120) and beams at their columns (RE-122). Doors and windows bind to their host wall by RE-85's rule on the u32 reference lists: all 19 on Exemplo_data and 11 on modelo_bim take the wall Revit's filled opening voids, and 15 of the 30 openings equal Revit's within 0.01 ft. Not read on 2023: profiles, layered or angled joins, IFC export overrides (4 spread footings Revit writes as IfcSlab come out as IfcFooting); names, types, storeys and materials are read since RE-107 to RE-116 (user_impact). Einhoven (2023, no Revit export) gains 2 walls, 9 curtain panels and 2 windows, unverified. Design options and the export phase are applied since RE-315, with the evidence stated for 2015 to 2018.", "user_impact": "Revit 2023 projects export their walls, doors, windows, floors, roofs, columns, beams, foundations and rooms as typed IFC elements in the right place and size, under Revit's own ElementIds, instead of an empty scaffold. Since RE-107 they also carry Revit's own Levels as storeys, each element on the storey of the Level its record names (since RE-119 a beam on the Level at its top): Exemplo_data 37 of 37 and modelo_bim 37 of 37 in Revit's storey, all 6 storeys Revit's name and elevation. Since RE-109 each family instance carries its family and type name: all 50 doors, windows, columns, beams and footings of the two projects as Revit names them. Since RE-111 each wall, floor and roof carries its type's name (24 of 24 as Revit's), and since RE-112 its type's layers name its system family, so every element Revit's export types is typed with Revit's type Tag: 37 of 37 on each project. Since RE-113 family instances carry their type's materials (Revit's set on 44 of 45), and since RE-114 walls and floors whose layer materials are named carry a material layer set (modelo_bim's 4 walls and floor); since RE-120 walls are cut back by the walls they join, by 2024's join solver (RE-26, RE-29) over the 2023 records, whose reference lists name the joined walls: Exemplo_data's 17 and modelo_bim's 4 walls exact against Revit's export (2 of 17 and 4 of 4 before). Since RE-115 a layer that takes its category's material takes the one 2023's object styles give it (i32 category · ff ff ff ff · u32 1|2 · u32 1 · ff ff ff ff 3f 01 · 8 bytes · i32 -3000010 · u32 material): Exemplo_data's 17 walls and floor, so its elements with Revit's material set are 35 of 37, and modelo_bim's 31 of 37. Since RE-116 2023 materials carry their shading colour, transparency and shininess (a 2024 frame whose four pattern slots are eight bytes, u32 id and u32 COLORREF) and are named as RE-58 names a 2024 material: by its own name field, else the string ending before ff ff ff ff eb 0b (ff ff ff ff 17 0c on 2024); the material list is the file's own materials (Exemplo_data 114, modelo_bim 108, Einhoven 36), Revit's colour and transparency on 20 of the 21 materials the two exports style, none different, and Exemplo_data's elements with Revit's material set are 36 of 37. Since RE-117 2023 rooms carry their number and name, the parameter entries i32 -1006901 (ROOM_NUMBER) and i32 -1006900 (ROOM_NAME) that follow the room's object: 9 of 9 on Exemplo_data as Revit's (see element-record-room-recovery). Since RE-122 a beam stops at the face of each column its record names and holds its end, as Revit's export stops it: modelo_bim's 8 beams exact (their boxes ran to the column centres before)." }, { @@ -265,8 +265,8 @@ "area": "elements", "name": "Design options: leave out non-primary options, and slab edges", "status": "experimental", - "evidence": "partition_design_options::compute_design_options; PartitionElementRecord::design_option; probes/examples/probe_re40_design_options.rs; tests/design_options.rs (the licensed files read no design option); reports/element-framing/RE-40-design-option-primary.md", - "notes": "An element record holds its design option's ElementId at +0x2a. A design option set's partition record lists its options at +0x56, and the set's name entry repeats that list and closes with the primary option's ElementId. The exporter drops records in a set's other options, as Revit's IFC export does by default, and reports them as the skipped item element_record_in_non_primary_design_option. A set whose name entry is missing or ambiguous stays unresolved and its options are kept. Measured locally on Snowdon Towers Architectural, which has no licence and is not in CI: of the placed instances of recovered categories, the 29 in the primary option 2059970 are all in Revit's IFC4 export and the 27 in non-primary options are all missing from it; removing them takes the elements outside Revit's export from 118 to 100 and removes none it holds. Slab edges (OST_EdgeSlab) are added as IfcBuildingElementProxy: all 59 exported are in Revit's export. No licensed file has a design option. Revit 2024 and 2025 only.", + "evidence": "partition_design_options::compute_design_options and option_set_members_32 (Revit 2016 to 2023); PartitionElementRecord::design_option; probes/examples/probe_re315_32bit_record_chain.rs; reports/element-framing/RE-315-32bit-second-prologue-records.md; probes/examples/probe_re40_design_options.rs; tests/design_options.rs (the licensed files read no design option); reports/element-framing/RE-40-design-option-primary.md", + "notes": "An element record holds its design option's ElementId at +0x2a. A design option set's partition record lists its options at +0x56, and the set's name entry repeats that list and closes with the primary option's ElementId. The exporter drops records in a set's other options, as Revit's IFC export does by default, and reports them as the skipped item element_record_in_non_primary_design_option. A set whose name entry is missing or ambiguous stays unresolved and its options are kept. Measured locally on Snowdon Towers Architectural, which has no licence and is not in CI: of the placed instances of recovered categories, the 29 in the primary option 2059970 are all in Revit's IFC4 export and the 27 in non-primary options are all missing from it; removing them takes the elements outside Revit's export from 118 to 100 and removes none it holds. Slab edges (OST_EdgeSlab) are added as IfcBuildingElementProxy: all 59 exported are in Revit's export. No licensed file has a design option. Revit 2024 and 2025; on Revit 2016 to 2023 with u32 ids (RE-315), measured on the eight saves of Autodesk's rst_basic_sample_project only: 1 set (primary 875635), 4 frames in a non-primary option, 1 Mass left out of the production counts and reported as the skipped item, and no change to the exported IFC (Revit does not export a Mass), with no Revit export of those files measured. Revit 2008 to 2015 read no set.", "user_impact": "Projects with design options export the model Revit itself exports, the main model with each option set's primary option, instead of every option stacked on top of each other." }, { diff --git a/docs/supported-profile.md b/docs/supported-profile.md index 92ce82a1..c8981612 100644 --- a/docs/supported-profile.md +++ b/docs/supported-profile.md @@ -21,8 +21,8 @@ apart: 2024 and 2025 project files, and Revit 2023 ones with their bounding boxes only (RE-81). Experimentally, Revit 2026 ones, measured so on one project (RE-124), and Revit 2015 to 2022 and 2027 ones, which have no measured - Revit export of their own release (except one 2015 project, BIMcollab's - structural example, RE-198) and are measured against the same sample projects saved by + Revit export of their own release (except two projects, BIMcollab's 2015 + structural example, RE-198, and Vitoria's 2018 `ALD26 Before`, RE-315) and are measured against the same sample projects saved by Revit 2019 (for 2015 to 2018), 2023 (for 2019 to 2022) and 2026 (for 2027) instead: a 2027 project's ElementIds and classes are its 2026 copy's, and on 2015 to 2022 every ElementId shared with the copy keeps its class, with @@ -43,7 +43,7 @@ apart: | Revit 2023 project | yes | yes, measured on two 2023 projects against Revit's IFC4 exports (RE-81); components nested in doors and windows are left out | the element's bounding box; rooms their outline (RE-102) | ElementId, category, door and window host walls (RE-85), Levels as storeys with each element on the one its record names (RE-107), and family instances' family and type names (RE-109) | | Revit 2027 project | partial: opens on Autodesk's sample projects; previews and PartAtom not measured | experimental (since 0.6.0): the same ElementIds and classes as the 2026 copies of Autodesk's five sample projects (RE-179); no Revit 2027 export is measured | as on 2026 | as on 2026 | | Revit 2019 to 2022 project | yes | experimental (since 0.6.0): 2023's records and readers with each release's schema tags; every ElementId shared with the 2023 copies of Autodesk's sample projects keeps its class, and the `rac_basic`, `rst_basic` and 2021 and 2022 `rac_advanced` exports have the 2023 copies' building elements, class for class (RE-180); no Revit export of a 2019 to 2022 file is measured | as on 2023 | as on 2023 | -| Revit 2015 to 2018 project | yes on 2016 to 2018; 2015 partial: opens on Autodesk's sample projects; previews and PartAtom not measured | experimental (since 0.6.0): 2019's records and readers with each release's schema tags; every ElementId shared with the 2019 copies of Autodesk's 2016 to 2018 sample projects keeps its class, and their exports have the 2019 copies' building elements, class for class; on five of the six 2015 samples every shared ElementId keeps its class, and the sixth, `rst_basic`, is another model (RE-181); no Revit export of a 2015 to 2018 file is measured | as on 2019 | as on 2019 | +| Revit 2015 to 2018 project | yes on 2016 to 2018; 2015 partial: opens on Autodesk's sample projects; previews and PartAtom not measured | experimental (since 0.6.0): 2019's records and readers with each release's schema tags; every ElementId shared with the 2019 copies of Autodesk's 2016 to 2018 sample projects keeps its class, and their exports have the 2019 copies' building elements, class for class; on five of the six 2015 samples every shared ElementId keeps its class, and the sixth, `rst_basic`, is another model (RE-181); Revit's own export is measured on one 2015 project (RE-198) and one 2018 project, Vitoria's `ALD26 Before`, where 1,136 of its 1,321 elements are written by ElementId, 1,127 of them in Revit's class or IFC4's (9 curtain walls come out as walls), none that the export leaves out, 1,073 on Revit's storey (RE-315) | as on 2019 | as on 2019 | | Revit 2011 to 2014 project | yes, on local closed-source files only, outside the 1.0 range (2015 to 2027); 2011 to 2013 open since RE-259 | experimental: 2015's records and readers with each release's schema tags; on a local copy of Autodesk's 2013 `rac_advanced` sample every ElementId shared with its 2015 copy keeps its class, and its export has the 2015 copy's building elements, class for class, but for a re-divided curtain wall (RE-259); no Revit export of a file before 2015 is measured | as on 2015 | as on 2015 | | Revit 2008 to 2010 project | yes, on local closed-source files only, outside the 1.0 range (2015 to 2027) (RE-259) | experimental: 2011's records and readers with each release's schema tags, 2008's envelope 4 bytes shorter (RE-260); checked only for consistency within each file, with no Revit export and no same-model copy | as on 2015 | storeys and named materials | | Family (`.rfa`) | yes | family metadata and OmniClass only | none | a family's types and type parameters as its `PartAtom` stream lists them: Revit's text at save, not read from the family's own records | @@ -184,7 +184,7 @@ still written. ## Not supported - Converting an arbitrary Revit model to IFC with Revit-grade fidelity. -- Element records of releases other than 2008 to 2027. Of those, 2008 to 2022, 2026 and 2027 are experimental: 2008 to 2014, outside the 1.0 range, with 2008 to 2010 checked only for consistency within each local file (RE-260) and 2011 to 2014 measured on one local 2013 sample against its 2015 copy (RE-259); 2026 is measured on one project against Revit's export (RE-124), and since 0.6.0 2027 against the 2026 copies of Autodesk's sample projects (RE-179), 2019 to 2022 against their 2023 copies (RE-180), and 2015 to 2018 against their 2019 copies (RE-181), none with a Revit export of its own release except one 2015 project, BIMcollab's structural example (RE-198). On 2023 (since 0.4.0): geometry beyond each element's box, wall joins and beam cuts, layered or angled joins, design options and IFC export overrides. +- Element records of releases other than 2008 to 2027. Of those, 2008 to 2022, 2026 and 2027 are experimental: 2008 to 2014, outside the 1.0 range, with 2008 to 2010 checked only for consistency within each local file (RE-260) and 2011 to 2014 measured on one local 2013 sample against its 2015 copy (RE-259); 2026 is measured on one project against Revit's export (RE-124), and since 0.6.0 2027 against the 2026 copies of Autodesk's sample projects (RE-179), 2019 to 2022 against their 2023 copies (RE-180), and 2015 to 2018 against their 2019 copies (RE-181), none with a Revit export of its own release except one 2015 project, BIMcollab's structural example (RE-198), and one 2018 project, Vitoria's `ALD26 Before` (RE-315). On 2023 (since 0.4.0): geometry beyond each element's box, wall joins and beam cuts, layered or angled joins and IFC export overrides. On 2016 to 2023, elements in a non-primary design option (measured on Autodesk's `rst_basic` samples only, a `Mass` Revit does not export, with no Revit export of them) and outside the export phase (measured on the held-out Vitoria Project's counts only) are left out since RE-315. - Semantic editing of a Revit file. `rvt-write` patches whole streams and preserves the rest byte for byte; it does not change model data. - Door/Window typing from the opening-index rows (RE-19) and Level ElementIds diff --git a/probes/examples/probe_re315_32bit_record_chain.rs b/probes/examples/probe_re315_32bit_record_chain.rs new file mode 100644 index 00000000..75381ee2 --- /dev/null +++ b/probes/examples/probe_re315_32bit_record_chain.rs @@ -0,0 +1,434 @@ +//! RE-315 probe: the record chain of a Revit 2008 to 2023 partition, and the +//! element frames that sit in it without an ElementId of their own (#1261). +//! +//! FACT (to be measured): a 32-bit partition is runs of back-to-back records +//! (`u32` ElementId, `u32` size, the tag of `ElementHeader`, a `u16` count, +//! the body, and a 12-byte trailer of a `u32`, a flag word and the size +//! again) with stretches between the runs where no record's trailer echoes +//! its size. The 2024 walk's flag test (`0` or `0x0100_0000`) stops at the +//! first record with another flag word, and only the leading run is walked; +//! a walk that tests the tag and the echo alone, and resumes at the next +//! record that validates, reaches the later runs. A placed-instance frame +//! that does not open with its declared id sits inside one declared record, +//! and its reference list names that record's id. `rvt::research:: +//! partition_element_records_2023::scan_records` gives such a frame that id. +//! +//! For each partition it prints the records the walk finds against every +//! place the tag sits 8 bytes into a validating record, the runs they form, +//! where the 2024 flag test would stop, and the frames of the file's marker +//! in the `BuiltInCategory` band by the id they open with, whether they are +//! placed instances, and whether a declared record encloses them and their +//! list names it. Then the instances `scan_records` selects, one per +//! ElementId as the 2023 export selects them, as `category +//! ` lines that `tools/re/instances_vs_ifc_tags.py` scores +//! against Revit's own export; a class is suffixed `.enclosed` where the id +//! came from the enclosing record. +//! +//! ```bash +//! cargo run --profile ci -p rvt-probes --example probe_re315_32bit_record_chain -- model.rvt +//! ``` + +use rvt::RevitFile; +use rvt::research::partition_element_records::{ + self as per, PLACEMENT_KIND_INSTANCE, RECOVERED_CATEGORIES, +}; +use rvt::research::partition_element_records_2023 as per32; +use std::collections::{BTreeMap, BTreeSet}; + +/// Bytes from the ElementId to the marker. +const ID_BEFORE_MARKER: usize = 52; +/// Bytes from the `BuiltInCategory` to the marker. +const CATEGORY_BEFORE_MARKER: usize = 38; +/// Bytes from the container reference to the marker. +const CONTAINER_BEFORE_MARKER: usize = 22; +/// Bytes from the placement kind to the marker. +const PLACEMENT_BEFORE_MARKER: usize = 14; +/// Bytes from the marker to the counted list of `u32` references. +const LIST_AFTER_MARKER: usize = 8 + 48; + +fn u32_at(buf: &[u8], at: usize) -> Option { + let end = at.checked_add(4)?; + buf.get(at..end) + .map(|b| u32::from_le_bytes(b.try_into().expect("4 bytes"))) +} + +fn u16_at(buf: &[u8], at: usize) -> Option { + let end = at.checked_add(2)?; + buf.get(at..end) + .map(|b| u16::from_le_bytes(b.try_into().expect("2 bytes"))) +} + +fn i64_at(buf: &[u8], at: usize) -> Option { + let end = at.checked_add(8)?; + buf.get(at..end) + .map(|b| i64::from_le_bytes(b.try_into().expect("8 bytes"))) +} + +/// The record at `at` as `(element_id, end, flag word)` when its header +/// carries `header_tag` and its trailer echoes its size. +fn record_at(buf: &[u8], at: usize, header_tag: u16) -> Option<(u32, usize, u32)> { + let id = u32_at(buf, at)?; + let size = u32_at(buf, at.checked_add(4)?)? as usize; + if u16_at(buf, at.checked_add(8)?)? != header_tag || size < 12 { + return None; + } + let end = at.checked_add(size)?; + let flag = u32_at(buf, end.checked_add(4)?)?; + (u32_at(buf, end.checked_add(8)?)? as usize == size).then_some((id, end, flag)) +} + +/// The leading run as the 2024 walk reads it, with a flag word of `0` or +/// `0x0100_0000` only: its records, where it stops, and the flag word that +/// stopped it, when a record validated but for its flag. +fn flag_rule_walk(buf: &[u8], header_tag: u16) -> (usize, usize, Option) { + let (mut at, mut records) = (0usize, 0usize); + while let Some((_, end, flag)) = record_at(buf, at, header_tag) { + if !matches!(flag, 0 | 0x0100_0000) { + return (records, at, Some(flag)); + } + records += 1; + at = end + 12; + } + (records, at, None) +} + +/// Every record of the partition, declared or not, walked as `record_spans` +/// walks: `(start, end, element_id, flag word)`. +fn walk_all(buf: &[u8], header_tag: u16) -> Vec<(usize, usize, u32, u32)> { + let needle = header_tag.to_le_bytes(); + let mut records = Vec::new(); + let mut at = 0usize; + loop { + if let Some((id, end, flag)) = record_at(buf, at, header_tag) { + records.push((at, end, id, flag)); + at = end + 12; + continue; + } + let Some(rest) = buf.get(at + 9..) else { + break; + }; + let Some(hit) = memchr::memmem::find(rest, &needle) else { + break; + }; + at = at + 9 + hit - 8; + } + records +} + +fn main() -> Result<(), Box> { + let path = std::env::args() + .skip(1) + .find(|arg| !arg.starts_with("--")) + .ok_or("usage: probe_re315_32bit_record_chain ")?; + let mut rf = RevitFile::open(&path)?; + let saved = rf.basic_file_info()?.version; + let release = rf.layout_release()?; + println!("release saved {saved} layout {release}"); + if !per32::is_32bit_release(release) { + println!("not a 32-bit release: nothing to measure"); + return Ok(()); + } + let Some((marker, header_tag)) = per32::record_marker(&mut rf) else { + println!("the schema gives no record marker"); + return Ok(()); + }; + println!( + "marker {} header tag {header_tag}", + marker + .iter() + .map(|b| format!("{b:02x}")) + .collect::>() + .join(" ") + ); + let declared: BTreeSet = + rvt::elem_table::declared_ids(&rvt::elem_table::parse_records(&mut rf)?); + println!("declared ids {}", declared.len()); + let shortfall = per32::envelope_shortfall(release); + let mut with_record: BTreeSet = BTreeSet::new(); + for stream in rf.partition_stream_names() { + let Ok(inflated) = rf.inflated_partition(&stream) else { + println!("{stream}: cannot be inflated"); + continue; + }; + let buf = inflated.bytes(); + let spans = per32::record_spans(buf, header_tag, &declared); + let all = walk_all(buf, header_tag); + let hits: BTreeSet = memchr::memmem::find_iter(buf, &header_tag.to_le_bytes()) + .filter_map(|hit| hit.checked_sub(8)) + .filter(|&at| record_at(buf, at, header_tag).is_some()) + .collect(); + let walked: BTreeSet = all.iter().map(|record| record.0).collect(); + let library: BTreeSet = spans.iter().map(|span| span.start).collect(); + let declared_walked: BTreeSet = all + .iter() + .filter(|record| declared.contains(&record.2)) + .map(|record| record.0) + .collect(); + let mut runs: Vec<(usize, usize, usize)> = Vec::new(); + for record in &all { + match runs.last_mut() { + Some(run) if run.1 + 12 == record.0 => { + run.1 = record.1; + run.2 += 1; + } + _ => runs.push((record.0, record.1, 1)), + } + } + let (legacy, legacy_stop, legacy_flag) = flag_rule_walk(buf, header_tag); + let mut flags: BTreeMap = BTreeMap::new(); + for record in &all { + *flags.entry(record.3).or_default() += 1; + } + println!("{stream}: {} bytes", buf.len()); + println!( + " records walked {} ({} with a declared id), at every validating tag {}, only walked {}, only at a tag {}", + walked.len(), + declared_walked.len(), + hits.len(), + walked.difference(&hits).count(), + hits.difference(&walked).count() + ); + println!( + " record_spans equals the declared records walked: {}", + library == declared_walked + ); + let longest = runs.iter().map(|run| run.2).max().unwrap_or(0); + println!(" runs {}, longest {longest} records", runs.len()); + for run in runs.iter().take(6) { + println!(" run {}..{}: {} records", run.0, run.1, run.2); + } + println!( + " flag rule walk: {legacy} records, stops at {legacy_stop}{}", + legacy_flag + .map(|flag| format!(" on flag word 0x{flag:08x}")) + .unwrap_or_default() + ); + let mut flag_list: Vec<_> = flags.iter().collect(); + flag_list.sort_by_key(|entry| std::cmp::Reverse(*entry.1)); + println!( + " trailer flag words: {}", + flag_list + .iter() + .take(6) + .map(|(flag, count)| format!("0x{flag:08x} x{count}")) + .collect::>() + .join(", ") + ); + with_record.extend(spans.iter().map(|span| span.element_id)); + // Every frame of the marker in the category band, by how its id is + // known. + let mut frames: BTreeMap = BTreeMap::new(); + for at in memchr::memmem::find_iter(buf, &marker) { + let Some(id_at) = at.checked_sub(ID_BEFORE_MARKER - shortfall) else { + continue; + }; + let Some(category) = at + .checked_sub(CATEGORY_BEFORE_MARKER - shortfall) + .and_then(|from| i64_at(buf, from)) + else { + continue; + }; + if !(per::BUILTIN_CATEGORY_MIN..=per::BUILTIN_CATEGORY_MAX).contains(&category) { + continue; + } + let Some(leading) = u32_at(buf, id_at) else { + continue; + }; + let kind = if declared.contains(&leading) && u16_at(buf, id_at + 8) == Some(header_tag) + { + "opens with its declared id".to_string() + } else { + let instance = at + .checked_sub(CONTAINER_BEFORE_MARKER) + .and_then(|from| u32_at(buf, from)) + == Some(u32::MAX) + && at + .checked_sub(PLACEMENT_BEFORE_MARKER) + .and_then(|from| u32_at(buf, from)) + == Some(PLACEMENT_KIND_INSTANCE); + let word = if leading == u32::MAX { + "ff ff ff ff" + } else { + "a count or other word" + }; + if !instance { + format!("no id, not a placed instance ({word})") + } else { + match per32::enclosing_span(&spans, id_at) { + None => format!("no id, placed instance, in no declared record ({word})"), + Some(span) => { + let list_at = at + LIST_AFTER_MARKER; + let named = u32_at(buf, list_at).is_some_and(|count| { + count <= 1024 + && (0..count as usize).any(|slot| { + u32_at(buf, list_at + 4 + slot * 4) == Some(span.element_id) + }) + }); + if named { + format!( + "no id, placed instance, in a record whose id its list names ({word})" + ) + } else { + format!( + "no id, placed instance, in a record whose id its list does not name ({word})" + ) + } + } + } + } + }; + *frames.entry(kind).or_default() += 1; + } + for (kind, count) in &frames { + println!(" frames {kind}: {count}"); + } + } + println!( + "declared ids with a record in some partition: {} of {}", + with_record.len(), + declared.len() + ); + + // What scan_records selects, one per ElementId, newest first as the 2023 + // export picks. + let records = per32::scan_records(&mut rf, release); + let enclosed_total = records + .iter() + .filter(|record| record.id_from_enclosing_record) + .count(); + println!( + "scan_records: {} records, {} with the enclosing record's id", + records.len(), + enclosed_total + ); + let level_ids = rvt::research::partition_level_records::scan_partition_level_ids( + &mut rf, release, &declared, + )?; + let levels = + rvt::research::partition_level_records::recover_partition_levels(&mut rf, release)?; + println!( + "levels: {} standalone Level records by id, {} recovered with a name and an elevation", + level_ids.len(), + levels.len() + ); + let level_tag = rf + .schema_classes()? + .classes + .iter() + .find(|class| class.name == "Level") + .map(|class| class.tag); + println!( + "schema class Level: {}", + level_tag.map_or("none".to_string(), |tag| format!("tag {tag}")) + ); + let level_needle = (-2_000_240i64).to_le_bytes(); + let mut level_words: BTreeMap<(u16, bool), usize> = BTreeMap::new(); + for stream in rf.partition_stream_names() { + let Ok(inflated) = rf.inflated_partition(&stream) else { + continue; + }; + let buf = inflated.bytes(); + for category in memchr::memmem::find_iter(buf, &level_needle) { + let at = category + 38 - shortfall; + if buf.get(at..at + 6) != Some(&marker[2..]) { + continue; + } + let (Some(word), Some(container), Some(kind)) = ( + at.checked_sub(6).and_then(|from| u16_at(buf, from)), + u32_at(buf, category + 16 - shortfall), + u32_at(buf, category + 24 - shortfall), + ) else { + continue; + }; + let standalone = container == u32::MAX && matches!(kind, 0xffff_ef6f | 0xffff_ef7f); + *level_words.entry((word, standalone)).or_default() += 1; + } + } + println!( + "Level frames by class word, standalone placed or not: {}", + level_words + .iter() + .map(|((word, standalone), count)| format!( + "{word} {} x{count}", + if *standalone { "standalone" } else { "other" } + )) + .collect::>() + .join(", ") + ); + let phases = rf.phase_filter(); + println!( + "phases: {} in order, {} elements not in the last, read error: {}", + phases.order.len(), + phases.excluded.len(), + phases.read_error.as_deref().unwrap_or("none") + ); + let options = rf.design_options(); + println!( + "design options: {} sets with a primary, {} unresolved, {} frames name an option, {} in a non-primary one", + options.sets.len(), + options.unresolved_sets.len(), + records + .iter() + .filter(|record| record.design_option.is_some()) + .count(), + records + .iter() + .filter(|record| options.excludes(record)) + .count() + ); + let instances = || { + records + .iter() + .filter(|record| record.is_exported_instance() && record.has_volume()) + }; + println!( + "placed instances with volume left out: {} by phase, {} by design option", + instances() + .filter(|record| phases.excludes(record)) + .map(|record| record.element_id) + .collect::>() + .len(), + instances() + .filter(|record| options.excludes(record)) + .map(|record| record.element_id) + .collect::>() + .len() + ); + let partition = |stream: &str| { + stream + .rsplit('/') + .next() + .and_then(|n| n.parse::().ok()) + .unwrap_or(0) + }; + let mut selected: BTreeMap = BTreeMap::new(); + for record in instances().filter(|record| !options.excludes(record) && !phases.excludes(record)) + { + let newer = selected.get(&record.element_id).is_none_or(|held| { + (partition(&record.stream), record.offset) > (partition(&held.stream), held.offset) + }); + if newer { + selected.insert(record.element_id, record); + } + } + let mut by_class: BTreeMap> = BTreeMap::new(); + for record in selected.values() { + let class = RECOVERED_CATEGORIES + .iter() + .find(|(category, _)| *category == record.builtin_category) + .map(|(_, class)| (*class).to_string()) + .unwrap_or_else(|| format!("OST{}", -record.builtin_category)); + let name = if record.id_from_enclosing_record { + format!("{class}.enclosed") + } else { + class + }; + by_class.entry(name).or_default().push(record.element_id); + } + println!("selected instances {}", selected.len()); + for (name, ids) in &by_class { + let list: Vec = ids.iter().map(|id| id.to_string()).collect(); + println!("category 0 {} {} {}", name, ids.len(), list.join(",")); + } + Ok(()) +} diff --git a/reports/element-framing/RE-315-32bit-second-prologue-records.md b/reports/element-framing/RE-315-32bit-second-prologue-records.md new file mode 100644 index 00000000..e82d4383 --- /dev/null +++ b/reports/element-framing/RE-315-32bit-second-prologue-records.md @@ -0,0 +1,144 @@ +# RE-315: Revit 2008 to 2023 frames with no ElementId of their own sit inside a declared partition record (measured on Revit 2018), and the export phase and design options on 2016 to 2023 + +**Date:** 2026-11-03 +**Issues:** #1261 (a Revit 2018 project exports no element), #408 (W5); follows RE-30 and RE-35 (the second prologue on 2024 and later), RE-81, RE-180 and RE-181 (the 32-bit layout), RE-107 (a Level named by an element's list), RE-135 (loaded families' documents) +**Artefacts:** BIM-Speed's Vitoria `ALD26 Before.rvt` (Revit 2018, build 20190510_1515, sha256 `d3e5b04631e67265927be98352021ed6290b33ae67d39b1b6727fd76f014ff6f`, 23,699,456 bytes) and Revit's own IFC2X3 export of it (`a42c6f3b0c9b7575d4c37c675155ff53c02571ca81357a87023f9da5fbf45181`), CC BY 4.0, TU Berlin DepositOnce, doi:10.14279/depositonce-16441, pinned in `research/holdout-archive-models.tsv` and fetched by Measure with `-f holdout=true`; Autodesk's `rst_basic_sample_project` of 2016 to 2023 (`research/autodesk-sample-projects.tsv`) for the design option layout; and, by counts only, the held-out `ALD26 Project`, Frigento, Gdynia, Warmond and BIM Whale's `AdvancedProject`. +**Probe:** `probes/examples/probe_re315_32bit_record_chain.rs`, scored by `tools/re/instances_vs_ifc_tags.py`. +**Status:** positive on Revit 2018's `ALD26 Before` against Revit's own export, and on its held-out sibling by counts. Releases other than 2018 are not measured against an export for this kind of frame. Design options (§10) are measured on Autodesk's `rst_basic` samples of 2016 to 2023 as diagnostics only, with no Revit export of them, and the phase filter (§10) only by the held-out `ALD26 Project`'s counts. + +## 1. Before + +Run 41707934595, main's side (`8664b94a`). Revit's export of `ALD26 Before` holds 1,394 elements and spaces (1,321 elements with a numeric `Tag`, 73 `IfcSpace`). rvt-ifc wrote none of them: 0 building elements, 0 storeys, 84 materials, scaffold confidence 0.25. The held-out `ALD26 Project` did the same against Revit's 1,520. + +The cause is the reader, not the file. `partition_element_records_2023::scan_records` took a frame only where a declared `u32` ElementId sits 52 bytes before the record marker with the tag of `ElementHeader` 8 bytes after it. On this file 652 frames open that way; 3,992 more frames in the `BuiltInCategory` band do not, 3,351 of them placed instances. + +## 2. The records + +Every record of a partition is `u32` ElementId, `u32` size, the `u16` tag of `ElementHeader` (1259 on this file), a `u16` count, the body, and a 12-byte trailer: a `u32`, a flag word and the size again. `ALD26 Before`'s `Partitions/0` (112,541,521 bytes) opens with one run of 12,626 of them from offset 0 to 2,534,666, 12,606 with an id `Global/ElemTable` declares. Revit's loaded families' own documents follow (RE-135), as runs of records with undeclared ids from 62,303,917 on. `Partitions/3` is one run of 941 records, `Partitions/1` one of 9. All 12,629 declared ids have a record. + +What reads this run is a walk that tolerates any flag word. The 2024 walk cannot: it accepts a flag word of `0` or `0x0100_0000` only, and the trailers of this partition's 86,081 records also hold `0x010008bd` (78), `0x010011cc` (22), `0x01000c54` (8) and `0x0100089b` (7). The flag test stops after 115 records, at offset 31,871 on `0x010011cc`. Testing the tag and the echo alone (`record_spans`) walks the whole run. Where a record fails to validate the walk resumes at the next place the tag sits 8 bytes into one that does; that only matters after the run, through the loaded families' documents, and adds no declared span. Both ways find the same records as testing every place the tag sits (86,081 on `Partitions/0`, none only one way). + +A first measurement of this file stopped at offset 741,926 of 100,855,918 with a record-looking structure that repeated every few hundred bytes. It was the reader of that measurement, not the file: `Partitions/*` and the `Global` streams are stored in pages of 65,249 bytes that end in a 353-byte checksum, which `compression::strip_revit_page_checksums` removes before inflating, and a reader that does not loses the gzip members that straddle a checksum (11.7 MB of this partition). Nothing between runs in the stripped stream is a different record form. + +## 3. Frames with no id of their own + +Of the frames in the band, per partition: + +| frames | `Partitions/0` | `Partitions/3` | `Partitions/1` | +|---|---:|---:|---:| +| open with their declared id and the tag | 652 | 0 | 0 | +| no id, not a placed instance | 628 | 13 | 0 | +| no id, placed instance, in a declared record whose id the frame's reference list names | 2,567 | 781 | 2 | +| no id, placed instance, in no declared record | 1 | 0 | 0 | + +(The 2,567 split into 1,657 whose first word is `ff ff ff ff` and 910 whose first word is a count, 0 to 6; 679 and 102 in `Partitions/3`.) Every placed-instance frame a declared record encloses names that record's id in its own counted list of `u32` ElementIds, as an element's does (RE-23): 3,350 of 3,350 on the three partitions, in every category. The record that encloses it is therefore the frame's element, and `scan_records` gives the frame its id, with `id_from_enclosing_record` set, when: + +1. the frame is a placed instance (no container reference and the placed-instance kind, the RE-21 rule); +2. a declared record encloses the frame's start; +3. the frame's list names that record's id; +4. the frame is not a curtain panel whose first word is a count (§5). + +A frame that fails one gets no id, as on 2024. `OST_Levels` frames are given an id by a related rule whose guard is a different one (§6). + +## 4. The ids against Revit's export + +Measured by the probe on rvt-rs's own bytes (run 41707939060): `scan_records` gives 3,900 records of `ALD26 Before`, 3,248 of them with the enclosing record's id, and 1,870 selected instances (one per ElementId, newest, with volume). Against Revit's 1,321 `Tag`s, 1,314 are among them; the 7 missed are 4 curtain-panel walls (§5) and 3 `IfcElementAssembly`. + +By frame, whether the id from the enclosing record is a `Tag` of Revit's export (a Python reading of the correctly inflated stream with the same rule; a frame is not an element, and copies in `Partitions/3` count again): + +| category | first word `ff ff ff ff` | first word a count | +|---|---|---| +| walls | 80 true | 201 true | +| doors, windows | 69, 76 true | none | +| floors, roofs | 4 true (roofs) | 9 floors, 6 roofs true | +| stairs, runs, railings | 6, none, 14 true | 6 runs true | +| structural columns | 22 true | 109 true | +| mullions | 303 true | none | +| curtain panels | 99 true | 6 true, **96 false** | +| pipes, pipe fittings | 232, 222 true | 24, 17 true | +| generic models, equipment | 22, 6, 30 true | none | + +The frames of categories rvt-rs does not export read as false against the `Tag`s, and no category it exports does but the panels: rooms (`IfcSpace` carries no `Tag`) and four categories (-2008072, -2008045, -2008139, -2008141) whose elements sit next to every pipe, pipe fitting, conduit and conduit fitting with the id one higher. rvt-rs's export has 73 `IfcSpace` and no element that Revit's lacks. + +## 5. Curtain panels + +On this file the curtain panels of one wall are pairs of elements: an even id that Revit exports as an `IfcPlate`, and the odd id before it, which owns the even one (`Global/ElemTable`'s owner word) and whose record holds a second frame in the panel category. 96 of the 102 panel frames whose first word is a count are that second frame (64 ids, each record 699 bytes with a count of 37, the word at marker-6 3291), and none of those 64 is in Revit's export; the other 6 are panels it does write, as `IfcWallStandardCase` where the panel's type is a basic wall. All 99 whose first word is `ff ff ff ff` are in the export. The rule therefore leaves a panel frame with a counted first word without an id: 96 false frames avoided, 6 true ones lost, which are the 4 panel walls listed as missing in §7. The fields compared (first word, the word at marker-44, the word at marker-6) do not tell those 6 from the 96, so no rule is written for them. + +## 6. Levels + +All seven `OST_Levels` frames that are standalone placed records, container unset and the 2015 to 2018 Level placement kind `0xffffef6f` (RE-181), are frames that do not open with their declared id (first word `ff ff ff ff`). `find_level_records_32` read first-prologue frames only, so the storey set was never recovered: at the commit that gave elements their enclosing id and no more (`9ff5e950`, run 41707946855) 687 of the 1,136 elements sat on no storey, and rvt-rs wrote 9 storeys measured from element boxes, named `Elevation 0.000 ft` and so on, against Revit's 7. + +A Level frame that is a standalone placed record and sits in a declared record now takes that record's id, but the guard of §3 does not exist for it: a Level frame's counted reference list is empty (0 entries on all 24 Level-category frames of this file, those with their own id as well), and the Level's name block does not lie in its record (the records are 1.5 MB into `Partitions/0`, the name blocks 11.9 MB). What stands in its place is two checks that need no constant of this file: the frame's class word, 6 bytes before its record marker, is the tag of the schema's class `Level`, and the stream holds a name and elevation block of the enclosing record's id. Without them a stray frame of the Level category inside another element would add an id with no name, and `recovered_levels_are_a_storey_set` would then drop every storey. The probe prints the schema's `Level` tag and the class word of every Level-category frame: on this file `Level` is tag 2130 and the frames are 2130 standalone x7, 2130 other x16 (container members), and 2131 x1 (the type symbol); all 7 survive and 7 Levels are recovered with a name and an elevation (run 41707939060). The same holds on Autodesk's `rst_basic_sample_project` of 2018 to 2023, whose 9 standalone Level frames each carry `Level`'s tag (2130, 2164, 2198, 2199, 2299 and 2377 on 2018 to 2023, 54 of 54), though those open with their own id and are not held to the check. A frame that opens with its declared id is read as before. + +## 7. Result + +Run 41707934595, main's side `8664b94a`, head `01b5af37` (the code of this report; the later commits change the export warning's text, cap the option sets searched at 64, merge main and change documents, and §10 repeats the held-out and identity runs at the final head): + +| `ALD26 Before` | main | head | Revit's export | +|---|---:|---:|---:| +| building elements and spaces written | 0 | 1,209 | 1,394 | +| elements written and paired by `Tag` (spaces counted apart) | 0 | 1,136 | 1,321 | +| of those, in Revit's class or IFC4's for its IFC2X3 one | 0 | 1,127 | | +| written that Revit's export lacks | 0 | 0 | | +| storeys | 0 | 7 | 7 | +| elements on Revit's storey | 0 | 1,073 of 1,136 | | + +By class, of Revit's elements: 1,136 of 1,321 are written, **1,127 in Revit's class or IFC4's for it**. In the same class: beams 1, building element proxies 43, columns 114, doors 68, members 206, plates 67, railings 14, roofs 8, slabs 9, stairs 6, stair flights 6, walls (`IfcWall`) 12, windows 70 (624). In IFC4's class for Revit's IFC2X3 one, counted found by `Tag` as Measure counts them: 128 `IfcFlowFitting` as `IfcPipeFitting`, 140 `IfcFlowSegment` as `IfcPipeSegment`, 235 `IfcWallStandardCase` as `IfcWall` (503). **In another class: 9 `IfcCurtainWall`, written as `IfcWall`** (Revit's export has IfcCurtainWall; IFC4 does too). **Missing, 185**: 104 `IfcFlowSegment` (102 conduits, `OST_Conduits` -2008132, and 2 gutters, -2001391), 74 `IfcFlowFitting` (conduit fittings, -2008128), 4 `IfcWallStandardCase` (§5), 3 `IfcElementAssembly`. The probe attributes all 102 conduits, 74 conduit fittings and 2 gutters (the `Canalón` family, category -2001391) to Revit's `Tag`s, and they are not written, because their categories are not in `RECOVERED_CATEGORIES`. + +Of the 1,136 written, on a storey: 1,073 are Revit's; 26 are on one where Revit contains them in an `IfcSpace`; 21 are on another (14 columns, 4 pipe segments, 3 pipe fittings); 16 are on none (9 building element proxies, 6 railings, 1 stair flight), against 687 on none before the Level change. The 7 storeys are as many as Revit's; the run does not score their names or elevations, and their GlobalIds are not Revit's (#878). + +**Held out, counts only** (`ALD26 Project`, Revit's 1,520): main writes 0 elements and spaces. The first reading of the branch, before the filters (run 41713761592, commit `8fc583a1`), wrote 1,534 elements and spaces, **198 of them elements that Revit's export does not hold** (95 pipe segments, 90 pipe fittings, 8 building element proxies, 2 roofs, 2 windows, 1 wall). `ALD26 Before` has none. With the export phase and design options of §10 it writes 1,336 (1,263 elements and 73 spaces), **no element that Revit's export lacks**, with every class found before still found: the same classes in the same class (beams, proxies, columns, coverings, doors, members, plates, railings, roofs, slabs, stairs, stair flights, walls, windows), the same classes missing (74 `IfcFlowFitting`, 104 `IfcFlowSegment`, 3 `IfcElementAssembly`, 3 `IfcWallStandardCase`), 8 `IfcCurtainWall` written as `IfcWall`, so 1,255 of its 1,263 elements are in Revit's class or IFC4's. 1,181 of the 1,263 are on Revit's storey and 14 on none. The counts cannot say which of the two filters removed the 198, and the held-out file is not opened to find out. + +**Unchanged:** the IFC hashes of Frigento (2020), Gdynia (2021), Warmond (2021) and the held-out BIM Whale `AdvancedProject` (2021) are byte for byte main's, and BIM Whale's counts lines are main's (run 41707934595). + +Run 41707942875 (`-f samples=true -f nordic=true -f r2025=true`, main's side `8664b94a`, head `01b5af37`) exports the Autodesk sample projects of 2015 to 2027, Core Interior, Einhoven, both Nordic offices, the RE1 models and puzzbobb's 23: 92 models, 0 differ from main's IFC, the 33 Autodesk samples of 2015 to 2023 among them. No sample frame takes an enclosing id, no phase excludes an element on any sample, and the only diagnostics that change are the eight `rst_basic` saves of 2016 to 2023 (§10). + +## 8. Reproduce + +```bash +# counts, held-out models by counts only (BIM Whale among them) +gh workflow run measure.yml --ref main -f ref=01b5af37f7e7d186ac7b60e4c49b85ffdab36dd6 -f base=main -f holdout=true -f models="spent-* holdout-bimspeed-* holdout-bimwhale-*" -f keep_ifc=true +# the probe, on Before and the samples of 2018 to 2023 only +gh workflow run measure.yml --ref main -f ref=01b5af37f7e7d186ac7b60e4c49b85ffdab36dd6 -f base=main -f holdout=true -f samples=true -f sample_years="2018 2023" -f models="spent-*" -f keep_ifc=true -f probe=probe_re315_32bit_record_chain -f probe_scorer=instances_vs_ifc_tags +# byte identity +gh workflow run measure.yml --ref main -f ref=01b5af37f7e7d186ac7b60e4c49b85ffdab36dd6 -f base=main -f samples=true -f nordic=true -f r2025=true +# the final head: every held-out pair, counts only, and byte identity +gh workflow run measure.yml --ref main -f ref=68ae7904d5b97350cd2aae670d262b8e982edcaf -f base=main -f holdout=true +gh workflow run measure.yml --ref main -f ref=68ae7904d5b97350cd2aae670d262b8e982edcaf -f base=main -f samples=true -f nordic=true -f r2025=true -f scorers=none +# the held-out Project before the filters of §10 +gh workflow run measure.yml --ref main -f ref=8fc583a165267ef842138d299472e7daecdd2bfe -f base=none -f holdout=true -f models="holdout-bimspeed-vitoria-project-2018" +# the elements-only commit of §6 +gh workflow run measure.yml --ref main -f ref=9ff5e9502b874685bd89360155023fb8e739fac2 -f base=none -f holdout=true -f models="spent-*" +``` + +## 9. What stays missing + +- 178 conduits, conduit fittings and gutters, which are Revit's exact `Tag`s on this file and are not written because their categories are not recovered. They are the next category census (RE-33), measured on more than one file. +- 4 curtain-panel walls and the 96 panel shells (§5). +- 3 `IfcElementAssembly`, which are not instances. +- 9 curtain walls written as walls. +- Revit 2008 to 2017 and 2019 to 2023 files with frames of this kind: none is measured against an export. The records and the rule are the same bytes on every release of the 32-bit layout; the claim here is 2018's. +- Extras on the held-out pairs that the filters of §10 do not remove: Frigento 22 and Gdynia 3 elements that Revit's export lacks, their exports byte for byte main's. They are another cause, not looked into here. + +## 10. The export phase and design options on Revit 2016 to 2023 + +The first reading of the held-out `ALD26 Project` wrote 198 elements that Revit's export does not hold (95 pipe segments, 90 pipe fittings, 8 proxies, 2 roofs, 2 windows, 1 wall), `Before` none. The cause sought was a filter 2024 applies and the 32-bit path does not. Of what 2024 filters, the 32-bit path had the instance rule, the volume test and the nested-in-opening test, and a symbol frame never takes an enclosing id. It had neither of the other two, whether a frame opens with its id or sits in a record: + +- **The export phase** (RE-184). `native_phases::compute_phase_filter` reads 2015 to 2027 (RE-281), but only 2024 records were tested with `PhaseFilter::excludes`. **Evidence that applying it is right on a 32-bit file: the held-out `ALD26 Project`'s counts and nothing else.** On `ALD26 Before` the filter reads 1 phase and no element outside it (run 41707939060), and on every one of Autodesk's 33 samples of 2015 to 2023 it reads 2 or 3 phases and no element outside the last, with no read error, so neither can show it; the counts of `Project` fell from 198 extra elements to none and cannot separate the phase from the design options below. +- **Design options** (RE-40). 2024 left out the elements of a set's non-primary options. The bytes carry over with `u32` ids, read on Autodesk's `rst_basic_sample_project`, the one Autodesk model with an option set (2015 has none), where the 2024 and 2025 copies are the oracle: the set record has its category at `+14` and its list at `+58`; the name entry is the 2024 one with `u32` ids ("Option Set 1", options 875635, 875638 and 875641, primary 875635); and the `u32` 26 bytes before a frame's marker is its option, `ff ff ff ff` for the main model (the first `u32` of that 8-byte slot is an object a line belongs to, and is not read). **Evidence: eight saves of one model, 2016 to 2023: 1 set and 4 frames naming the non-primary option, 875638, on each (a type symbol and three instances, one of them with volume, a `Mass`); Revit does not export a `Mass`, so no export effect is observed, and no Revit export of these files is measured.** The 2024 and 2025 copies count 1 `Mass` in a non-primary option as well. `ALD26 Before` has no option set (the category ids occur only in a loaded family's document). A value that is not an option of a resolved set excludes nothing, and an unresolved set keeps its elements, as on 2024. + +Both are applied on Revit 2016 to 2023 only (`partition_element_records_2023::EXPORT_FILTER_RELEASES`): the layout is read on those releases' samples and the phase filter measured on a 2018 file. On 2008 to 2015 no set is read and no element is left out, because there is no file of those releases with a set or an element outside the phase to measure on. (BIMcollab's Revit 2015 example has 8 elements outside its last phase and none of them a written instance, so it shows nothing either way.) `recover_2023_records` drops a selected instance that either filter excludes, rooms never by phase. The counts of the diagnostics (`element_record_not_in_export_phase`, `element_record_in_non_primary_design_option`) are read on the same releases. + +**What changes on the eight `rst_basic` samples** (2018 to 2023: run 41707939060, 2016 and 2017: run 41707942875): the exported IFC is byte for byte main's and `building_elements` is 454 on each. The diagnostics change: a skipped item `element_record_in_non_primary_design_option` appears with 1 `Mass`; `decoded.production_class_counts` loses its `Mass` and `production_walker_elements` falls by 1 (530 to 529 on 2018 and 2019, 545 to 544 on 2020 and 2021, 546 to 545 on 2022, 547 to 546 on 2023, 513 to 512 on 2016, 530 to 529 on 2017); `decoded.mean_element_confidence` moves by less than 0.0001. The 2015 sample, which has no set, and the 2024 to 2027 samples do not change. + +Result (run 41707934595, head `01b5af37`, counts only for the held-out): + +- `ALD26 Project`: 1,336 elements and spaces written (1,263 elements and 73 spaces) against Revit's 1,520; **no element written that Revit's export lacks** (198 before the filters); every class found in the same class as before, none lost; 1,255 of the 1,263 elements in Revit's class or IFC4's, 8 curtain walls as walls; 1,181 of 1,263 on Revit's storey and 14 on none. +- `ALD26 Before`: unchanged by the filters and by the Level checks of §6, 1,136 of 1,321, none extra, 7 storeys; its IFC is the one of the previous head. +- Frigento, Gdynia, Warmond and BIM Whale: IFC byte for byte main's. Frigento (22) and Gdynia (3) keep elements Revit's export lacks. +- **The export warning** (`diagnostics.warnings`) said design options were not read for every 2008 to 2023 export. It now says per release: on 2016 to 2023 the export phase and design options are applied, on 2008 to 2015 neither is. A file's option sets are searched for at most 64 (`OPTION_SETS_MAX`); the sets past it, by ElementId, stay unresolved and keep their elements, as a set with no name entry does on 2024. The most any file here has is 2. + +**At the final head** (`68ae7904`, the merge of main `53e06c02` into `c04c92dc`): + +- Run 41722201615 (`-f holdout=true -f base=main`, no model filter: Core Interior, Einhoven, the four RE1 models, the spent `ALD26 Before` and every held-out pair of Measure, counts only for the held-out): the exported IFC of 23 of the 25 models is byte for byte main's. The two that move are the two Vitoria saves. The `counts.txt` of the other 17 held-out models is main's line for line, 0 lines differ: the Revit 2023 project `modelo_bim` (the only 2023 pair in Measure's held-out set; the other 2023 project of the 255ribeiro set, `Exemplo_data`, is local only and not measured here), the 2022 and 2024 to 2027 pairs and offices, Frigento, Gdynia, Warmond, BIM Whale, `AA-SingleDwellingHouse` and `teste_export_2025`. `ALD26 Before`'s class sets and storeys and `ALD26 Project`'s counts are those of run 41707934595. +- Run 41722205132 (`-f samples=true -f nordic=true -f r2025=true -f scorers=none`): 92 models, 0 differ from main's IFC. The diagnostics change on the 33 samples of 2015 to 2023 (the warning's text) and, in the skipped items, on the eight `rst_basic` saves of 2016 to 2023. diff --git a/src/ifc/mod.rs b/src/ifc/mod.rs index 7b5bf187..3e54f8f0 100644 --- a/src/ifc/mod.rs +++ b/src/ifc/mod.rs @@ -3826,7 +3826,22 @@ pub fn build_export_diagnostics_with_limits( layout.filter(|&version| crate::partition_element_records_2023::is_32bit_release(version)) { // RE-81, RE-102, RE-107, RE-109, RE-111 to RE-115, RE-180: what a - // 2015 to 2023 export carries, and what it does not (RE-181). + // 2008 to 2023 export carries, and what it does not (RE-181). The + // export phase and design options are applied on 2016 to 2023 only + // (RE-315). + let filters = if crate::partition_element_records_2023::applies_export_filters(version) { + format!( + "The export phase and design options are applied for {version}: elements that \ + do not exist in the last phase or are in a non-primary design option are left \ + out." + ) + } else { + format!( + "Neither the export phase nor design options are applied for {version} (they \ + are on 2016 to 2023): elements outside the last phase or in a non-primary \ + design option are written." + ) + }; warnings.push(format!( "Revit {version}: elements come from their records with their ElementId, category and \ bounding box, doors and windows with their host wall, rooms with their outline, \ @@ -3835,7 +3850,7 @@ pub fn build_export_diagnostics_with_limits( type names, and family instances their type's materials. Layers become \ material layer sets where their materials are named, a layer that takes its \ category's material taking the object styles' one, and walls keep their record box. \ - Wall joins and profiles, design options, parameters and \ + {filters} Wall joins and profiles, parameters and \ \"Export to IFC As\" overrides are not read for {version}, and components \ nested in doors and windows are left out." )); diff --git a/src/research/native_phases.rs b/src/research/native_phases.rs index 692d991e..345034f8 100644 --- a/src/research/native_phases.rs +++ b/src/research/native_phases.rs @@ -172,20 +172,26 @@ pub fn scan_phase_excluded_instances( RECOVERED_CATEGORIES, scan_category_records_multi, supports_revit_version, }; let mut by_class = BTreeMap::new(); - if !supports_revit_version(revit_version) { + let filtered_32bit = + crate::partition_element_records_2023::applies_export_filters(revit_version); + if !filtered_32bit && !supports_revit_version(revit_version) { return Ok(by_class); } let phases = rf.memoised_phase_filter(); if phases.excluded.is_empty() { return Ok(by_class); } - let declared: BTreeSet = match crate::elem_table::parse_records(rf) { - Ok(records) => crate::elem_table::declared_ids(&records), - Err(_) => return Ok(by_class), - }; let options = rf.memoised_design_options(); - let categories: Vec = RECOVERED_CATEGORIES.iter().map(|(c, _)| *c).collect(); - let records = scan_category_records_multi(rf, revit_version, &categories, &declared)?; + let records = if filtered_32bit { + crate::partition_element_records_2023::scan_records(rf, revit_version) + } else { + let declared: BTreeSet = match crate::elem_table::parse_records(rf) { + Ok(records) => crate::elem_table::declared_ids(&records), + Err(_) => return Ok(by_class), + }; + let categories: Vec = RECOVERED_CATEGORIES.iter().map(|(c, _)| *c).collect(); + scan_category_records_multi(rf, revit_version, &categories, &declared)? + }; let mut excluded: BTreeMap = BTreeMap::new(); for record in records .iter() diff --git a/src/research/partition_design_options.rs b/src/research/partition_design_options.rs index 478dd129..66282e88 100644 --- a/src/research/partition_design_options.rs +++ b/src/research/partition_design_options.rs @@ -36,12 +36,27 @@ //! ids, and a set lists its options in ascending id order, so neither says //! which option is primary. //! +//! # Revit 2016 to 2023 +//! +//! The same three pieces are there with `u32` ids (RE-315, read on Autodesk's +//! 2018 `rst_basic_sample_project`, whose set "Option Set 1" lists 875635, +//! 875638 and 875641 with 875635 primary): the set record has its category at +//! `+14` and the count at `+58`, the name entry writes each id as a `u32`, +//! and an element's option is the `u32` 26 bytes before its record marker, +//! `ff ff ff ff` for the main model. The same layout holds on the eight +//! `rst_basic_sample_project` saves of 2016 to 2023: 1 set and 4 frames in a +//! non-primary option each, a `Mass` Revit does not export. No Revit export +//! of them is measured, and no other file has a set. Other releases of the +//! 32-bit layout read no set +//! ([`crate::partition_element_records_2023::EXPORT_FILTER_RELEASES`]). +//! //! # Fail closed //! //! A set whose name entry is missing, or whose entries disagree about the //! primary, stays unresolved, and elements in its options are kept as -//! before. Only an option of a resolved set that is not its primary is -//! reported as non-primary. +//! before. So does every set past the first [`OPTION_SETS_MAX`] of a file, +//! by ElementId: each set costs a search of every partition. Only an option +//! of a resolved set that is not its primary is reported as non-primary. use crate::le_bytes; use crate::partition_element_records::{ @@ -66,6 +81,13 @@ pub const OPTIONS_PER_SET_MAX: usize = 64; /// Longest set name, in UTF-16 code units, read from a name entry. pub const SET_NAME_MAX_UNITS: usize = 256; +/// Most option sets a file's primaries are looked for. Each set costs a +/// search of every partition for its name entry, and the most any file here +/// has is 2 (Snowdon Towers). The sets past it, by ElementId, are left +/// unresolved: their options are neither primary nor non-primary, so their +/// elements are kept, as for a set with no name entry (RE-315). +pub const OPTION_SETS_MAX: usize = 64; + /// A design option set whose primary option was read. #[derive(Debug, Clone, PartialEq, Eq)] pub struct DesignOptionSet { @@ -152,10 +174,27 @@ pub fn option_set_members(buf: &[u8], start: usize, end: usize, set_id: u64) -> /// lists `options`. A primary that is not one of `options`, or a name with /// a control character or an unpaired surrogate, is not an entry. pub fn find_set_entries(buf: &[u8], options: &[u32]) -> Vec<(String, u32)> { - let mut needle = Vec::with_capacity(8 + options.len() * 8); + find_set_entries_of_width(buf, options, 8) +} + +/// [`find_set_entries`] for Revit 2008 to 2023, whose name entries write +/// each id as a `u32` (RE-315). +pub fn find_set_entries_32(buf: &[u8], options: &[u32]) -> Vec<(String, u32)> { + find_set_entries_of_width(buf, options, 4) +} + +fn find_set_entries_of_width(buf: &[u8], options: &[u32], id_bytes: usize) -> Vec<(String, u32)> { + let id = |value: u32| -> Vec { + if id_bytes == 8 { + u64::from(value).to_le_bytes().to_vec() + } else { + value.to_le_bytes().to_vec() + } + }; + let mut needle = Vec::with_capacity(8 + options.len() * id_bytes); needle.extend_from_slice(&(options.len() as u32).to_le_bytes()); for option in options { - needle.extend_from_slice(&u64::from(*option).to_le_bytes()); + needle.extend_from_slice(&id(*option)); } needle.extend_from_slice(&1u32.to_le_bytes()); let mut out = Vec::new(); @@ -178,9 +217,13 @@ pub fn find_set_entries(buf: &[u8], options: &[u32]) -> Vec<(String, u32)> { if name.trim().is_empty() || name.chars().any(char::is_control) { continue; } - let Some(primary) = - le_bytes::u64_at(buf, name_at + units * 2).and_then(|v| u32::try_from(v).ok()) - else { + let primary_at = name_at + units * 2; + let primary = if id_bytes == 8 { + le_bytes::u64_at(buf, primary_at).and_then(|v| u32::try_from(v).ok()) + } else { + le_bytes::u32_at(buf, primary_at) + }; + let Some(primary) = primary else { continue; }; if options.contains(&primary) { @@ -190,49 +233,129 @@ pub fn find_set_entries(buf: &[u8], options: &[u32]) -> Vec<(String, u32)> { out } +/// Offset, from the start of a 32-bit option-set partition record, of its +/// `BuiltInCategory`: after the 12-byte record header and two unread bytes. +pub const OPTION_SET_CATEGORY_OFFSET_32: usize = 14; + +/// Offset, from the start of a 32-bit option-set partition record, of the +/// `u32` count that opens the set's id list. The ids that follow are `u32`. +pub const OPTION_LIST_OFFSET_32: usize = 58; + +/// [`option_set_members`] for the 32-bit record of Revit 2008 to 2023 +/// (RE-315): the category and the list sit at their own offsets and each id +/// is a `u32`. +pub fn option_set_members_32( + buf: &[u8], + start: usize, + end: usize, + set_id: u32, +) -> Option> { + let category = le_bytes::i64_at(buf, start.checked_add(OPTION_SET_CATEGORY_OFFSET_32)?)?; + if category != OST_DESIGN_OPTION_SETS { + return None; + } + let list_at = start.checked_add(OPTION_LIST_OFFSET_32)?; + let count = le_bytes::u32_at(buf, list_at)? as usize; + if !(2..=OPTIONS_PER_SET_MAX + 1).contains(&count) { + return None; + } + let ids_at = list_at.checked_add(4)?; + if ids_at.checked_add(count.checked_mul(4)?)? > end { + return None; + } + if le_bytes::u32_at(buf, ids_at)? != set_id { + return None; + } + let mut options = Vec::with_capacity(count - 1); + for index in 1..count { + let id = le_bytes::u32_at(buf, ids_at + index * 4)?; + if id == set_id || options.contains(&id) { + return None; + } + options.push(id); + } + Some(options) +} + +/// A reader of the `(name, primary)` entries a set has in a buffer. +type SetEntries = fn(&[u8], &[u32]) -> Vec<(String, u32)>; + /// The design option sets of a whole file and their primary options. /// Empty where the release's record shape is not proven. /// `RevitFile::design_options` memoises it. pub fn compute_design_options(rf: &mut RevitFile) -> Result { - let mut out = DesignOptions::default(); let version = rf.layout_release()?; - let Some(marker) = crate::partition_element_records::file_bbox_marker(rf, version) else { - return Ok(out); - }; + let mut out = DesignOptions::default(); let streams = rf.partition_stream_names(); let mut members: BTreeMap> = BTreeMap::new(); - for stream in &streams { - let Ok(inflated) = rf.inflated_partition(stream) else { - continue; + let entries_of: SetEntries; + if crate::partition_element_records_2023::is_32bit_release(version) { + // Outside the releases measured no set is read, so no element is + // left out by option (RE-315). + if !crate::partition_element_records_2023::applies_export_filters(version) { + return Ok(out); + } + entries_of = find_set_entries_32; + let Some((_, header_tag)) = crate::partition_element_records_2023::record_marker(rf) else { + return Ok(out); }; - let buf = inflated.bytes(); - for span in partition_record_chain(buf, &marker) { - let Ok(set_id) = u32::try_from(span.element_id) else { + let declared: BTreeSet = match crate::elem_table::parse_records(rf) { + Ok(records) => crate::elem_table::declared_ids(&records), + Err(_) => return Ok(out), + }; + for stream in &streams { + let Ok(inflated) = rf.inflated_partition(stream) else { continue; }; - if let Some(options) = option_set_members(buf, span.start, span.end, span.element_id) { - // A set framed twice with different lists is unresolved. - match members.get(&set_id) { - Some(held) if *held != options => { - out.unresolved_sets.insert(set_id); - } - _ => { - members.insert(set_id, options); - } + let buf = inflated.bytes(); + for span in + crate::partition_element_records_2023::record_spans(buf, header_tag, &declared) + { + if let Some(options) = + option_set_members_32(buf, span.start, span.end, span.element_id) + { + note_set(&mut members, &mut out, span.element_id, options); + } + } + } + } else { + entries_of = find_set_entries; + let Some(marker) = crate::partition_element_records::file_bbox_marker(rf, version) else { + return Ok(out); + }; + for stream in &streams { + let Ok(inflated) = rf.inflated_partition(stream) else { + continue; + }; + let buf = inflated.bytes(); + for span in partition_record_chain(buf, &marker) { + let Ok(set_id) = u32::try_from(span.element_id) else { + continue; + }; + if let Some(options) = + option_set_members(buf, span.start, span.end, span.element_id) + { + note_set(&mut members, &mut out, set_id, options); } } } } + let mut searched = 0usize; for (set_id, options) in members { if out.unresolved_sets.contains(&set_id) { continue; } + if searched >= OPTION_SETS_MAX { + out.unresolved_sets.insert(set_id); + continue; + } + searched += 1; let mut entries: BTreeSet<(String, u32)> = BTreeSet::new(); for stream in &streams { let Ok(inflated) = rf.inflated_partition(stream) else { continue; }; - entries.extend(find_set_entries(inflated.bytes(), &options)); + entries.extend(entries_of(inflated.bytes(), &options)); } let primaries: BTreeSet = entries.iter().map(|(_, primary)| *primary).collect(); match (primaries.len(), entries.into_iter().next()) { @@ -255,6 +378,24 @@ pub fn compute_design_options(rf: &mut RevitFile) -> Result { Ok(out) } +/// Record the options a set record lists. A set framed twice with different +/// lists is unresolved. +fn note_set( + members: &mut BTreeMap>, + out: &mut DesignOptions, + set_id: u32, + options: Vec, +) { + match members.get(&set_id) { + Some(held) if *held != options => { + out.unresolved_sets.insert(set_id); + } + _ => { + members.insert(set_id, options); + } + } +} + /// Placed instances of the /// [`crate::partition_element_records::RECOVERED_CATEGORIES`] that are in a /// non-primary design option, one per ElementId, keyed by class name. The @@ -266,19 +407,25 @@ pub fn scan_non_primary_option_instances( ) -> Result> { use crate::partition_element_records::{RECOVERED_CATEGORIES, scan_category_records_multi}; let mut by_class = BTreeMap::new(); - if !supports_revit_version(revit_version) { + let filtered_32bit = + crate::partition_element_records_2023::applies_export_filters(revit_version); + if !filtered_32bit && !supports_revit_version(revit_version) { return Ok(by_class); } let options = rf.memoised_design_options(); if options.sets.is_empty() { return Ok(by_class); } - let declared: BTreeSet = match crate::elem_table::parse_records(rf) { - Ok(records) => crate::elem_table::declared_ids(&records), - Err(_) => return Ok(by_class), + let records = if filtered_32bit { + crate::partition_element_records_2023::scan_records(rf, revit_version) + } else { + let declared: BTreeSet = match crate::elem_table::parse_records(rf) { + Ok(records) => crate::elem_table::declared_ids(&records), + Err(_) => return Ok(by_class), + }; + let categories: Vec = RECOVERED_CATEGORIES.iter().map(|(c, _)| *c).collect(); + scan_category_records_multi(rf, revit_version, &categories, &declared)? }; - let categories: Vec = RECOVERED_CATEGORIES.iter().map(|(c, _)| *c).collect(); - let records = scan_category_records_multi(rf, revit_version, &categories, &declared)?; let mut excluded: BTreeMap = BTreeMap::new(); for record in records .iter() diff --git a/src/research/partition_element_records.rs b/src/research/partition_element_records.rs index 81635492..e08b5ad2 100644 --- a/src/research/partition_element_records.rs +++ b/src/research/partition_element_records.rs @@ -625,13 +625,16 @@ pub struct PartitionElementRecord { pub references: Vec, /// `true` when the frame carries no ElementId at `+0x00` (the second /// prologue, RE-30) and [`Self::element_id`] is the id of the partition - /// record the frame sits in ([`assign_second_prologue_ids`], RE-35). + /// record the frame sits in ([`assign_second_prologue_ids`], RE-35; on + /// Revit 2008 to 2023, which write the id as a `u32` further back, + /// [`crate::partition_element_records_2023::record_spans`], RE-315). /// `false` for a frame that starts its record, with the id at `+0x00`. #[serde(default)] pub id_from_enclosing_record: bool, /// The design option the element belongs to, from `+0x2a` - /// ([`DESIGN_OPTION_OFFSET`]). `None` for the main model, and for a - /// value outside the `u32` ElementId range. + /// ([`DESIGN_OPTION_OFFSET`]), or on Revit 2008 to 2023 from the `u32` + /// 26 bytes before the record marker (RE-315). `None` for the main + /// model, and for a value outside the `u32` ElementId range. /// /// Whether the option is its set's primary one is document-level /// knowledge: see [`crate::partition_design_options`]. diff --git a/src/research/partition_element_records_2023.rs b/src/research/partition_element_records_2023.rs index 90972404..d98a5b05 100644 --- a/src/research/partition_element_records_2023.rs +++ b/src/research/partition_element_records_2023.rs @@ -16,19 +16,31 @@ //! Measured on two 2023 projects against Revit's own IFC4 exports: every //! element Tag of each export is a placed instance here //! (`reports/element-framing/RE-81-2023-element-records.md`). What 2024 -//! decodes on top of the records (design options) is not established for -//! 2023 and is not read, except wall joins, whose reference-list partners -//! trim walls as on 2024 (RE-120); family instances' family and type -//! names are (RE-109), and so are system-family types and their layers -//! (RE-111, RE-112). Door and window hosts are: -//! the reference lists carry them as on 2024 (RE-85), and so they carry -//! each element's Level (RE-107). +//! decodes on top of the records is read for 2023 as far as wall joins, +//! whose reference-list partners trim walls as on 2024 (RE-120), and, on 2016 +//! to 2023, design options, whose id sits in the frame 26 bytes before the +//! marker and whose sets [`crate::partition_design_options`] reads +//! ([`EXPORT_FILTER_RELEASES`], RE-315). Family +//! instances' family and type names are read (RE-109), and so are +//! system-family types and their layers (RE-111, RE-112). Door and window +//! hosts are: the reference lists carry them as on 2024 (RE-85), and so they +//! carry each element's Level (RE-107). +//! +//! A frame with no declared ElementId 52 bytes before its marker (the +//! second prologue, RE-30) sits inside its element's partition record, as +//! on 2024 (RE-35). The record is `u32` ElementId, `u32` size, the tag of +//! `ElementHeader` and a `u16` count, the body, and a 12-byte trailer: a +//! `u32`, a flag word and the size again. [`record_spans`] walks them, and +//! [`scan_records`] gives a placed-instance frame the id of the declared +//! record around it when the frame's own reference list names that id +//! (RE-315, #1261). use crate::RevitFile; use crate::le_bytes; use crate::partition_element_records::{ BBOX_MARKER_OFFSET, BBOX_OFFSET, BUILTIN_CATEGORY_MAX, BUILTIN_CATEGORY_MIN, CLASS_TAG_OFFSET, - PLACEMENT_KIND_OFFSET, PartitionElementRecord, + CONTAINER_NONE, OST_CURTAIN_WALL_PANELS, PLACEMENT_KIND_INSTANCE, PLACEMENT_KIND_OFFSET, + PartitionElementRecord, }; use std::collections::BTreeSet; @@ -60,6 +72,19 @@ pub fn is_32bit_release(revit_version: u32) -> bool { REVIT_32BIT_RELEASES.contains(&revit_version) } +/// The releases whose elements outside the export phase and in a non-primary +/// design option are left out (RE-315): Revit 2016 to 2023. The design option +/// layout is read on Autodesk's `rst_basic_sample_project` of each of them, +/// and the phase filter's only evidence that it is right on a 32-bit file is +/// a 2018 project. 2008 to 2015 have no evidence, and keep every element. +pub const EXPORT_FILTER_RELEASES: std::ops::RangeInclusive = 2016..=2023; + +/// Whether `revit_version` leaves out the elements outside the export phase +/// and in a non-primary design option ([`EXPORT_FILTER_RELEASES`]). +pub fn applies_export_filters(revit_version: u32) -> bool { + EXPORT_FILTER_RELEASES.contains(&revit_version) +} + /// The schema tags the 32-bit readers match, each the tag of the named /// class in the release's own schema. The constants measured on 2023 /// (RE-111 to RE-116) are 2023's tags of these classes, and 2024's @@ -168,6 +193,12 @@ const CATEGORY_BEFORE_MARKER: usize = 38; /// 4,183 and 95, every container member among them /// (`probes/examples/probe_32bit_container.rs`). const CONTAINER_BEFORE_MARKER: usize = 22; +/// Bytes from the design option reference to the marker. The 8 bytes after +/// the category are two `u32` slots; the second holds the ElementId of the +/// design option the element belongs to, and `ff ff ff ff` for the main +/// model (RE-315). The first holds the id of an object a line belongs to on +/// the frames that set it, and is not read. +const DESIGN_OPTION_BEFORE_MARKER: usize = 26; /// Offset of the `ElementHeader` tag after the ElementId (after the `u32` /// id and the `u32` size). const HEADER_TAG_AFTER_ID: usize = 8; @@ -196,9 +227,124 @@ pub fn record_marker(rf: &mut RevitFile) -> Option<([u8; 8], u16)> { Some((marker, header)) } +/// Length of the header that opens a 32-bit partition record: the `u32` +/// ElementId, the `u32` record size, the tag of `ElementHeader` and a `u16` +/// count (RE-315). +pub const RECORD_HEADER_LEN: usize = 12; + +/// Length of the trailer that closes a 32-bit partition record: a `u32`, a +/// flag word and the record size again (RE-315). The flag word is `0`, +/// `0x0100_0000` or, on a record that holds more, another value +/// (`0x0100_11cc` on one record of a Revit 2018 project), so it is not read. +pub const RECORD_TRAILER_LEN: usize = 12; + +/// One record of a 32-bit partition: where it starts, where its body ends +/// (its trailer starts there) and the ElementId in its header. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct RecordSpan { + /// Offset of the record header. + pub start: usize, + /// Offset one past the record body, where its trailer starts. + pub end: usize, + /// The ElementId the header opens with, declared in `Global/ElemTable`. + pub element_id: u32, +} + +/// The record that starts at `start`, as `(element_id, end)`, or `None` when +/// `start` is not a record header whose trailer echoes its size. +fn record_at(buf: &[u8], start: usize, header_tag: u16) -> Option<(u32, usize)> { + let element_id = le_bytes::u32_at(buf, start)?; + let size = le_bytes::u32_at(buf, start.checked_add(4)?)? as usize; + if le_bytes::u16_at(buf, start.checked_add(HEADER_TAG_AFTER_ID)?)? != header_tag + || size < RECORD_HEADER_LEN + { + return None; + } + let end = start.checked_add(size)?; + let echo = le_bytes::u32_at(buf, end.checked_add(8)?)? as usize; + (echo == size).then_some((element_id, end)) +} + +/// The records of one inflated 32-bit partition whose ElementId is in +/// `declared`, in file order (RE-315). +/// +/// A partition is runs of back-to-back records ([`RECORD_HEADER_LEN`], +/// body, [`RECORD_TRAILER_LEN`]) with stretches between them that hold no +/// record whose trailer echoes its size. The walk starts at offset 0, steps +/// over each record that validates, and where one does not resumes at the +/// next place the `ElementHeader` tag sits 8 bytes into a record that +/// validates. A record of a loaded family's own document (RE-135) is stepped +/// over and left out, as its id is not declared. A record is never entered, +/// so the spans do not overlap and each step moves forward, which makes the +/// walk linear in `buf`. +pub fn record_spans(buf: &[u8], header_tag: u16, declared: &BTreeSet) -> Vec { + let needle = header_tag.to_le_bytes(); + let finder = memchr::memmem::Finder::new(&needle); + let mut spans = Vec::new(); + let mut at = 0usize; + loop { + if let Some((element_id, end)) = record_at(buf, at, header_tag) { + if declared.contains(&element_id) { + spans.push(RecordSpan { + start: at, + end, + element_id, + }); + } + at = end + RECORD_TRAILER_LEN; + continue; + } + let from = at + HEADER_TAG_AFTER_ID + 1; + let Some(rest) = buf.get(from..) else { + break; + }; + let Some(hit) = finder.find(rest) else { + break; + }; + at = from + hit - HEADER_TAG_AFTER_ID; + } + spans +} + +/// The span of `spans` (from [`record_spans`]) whose body holds `offset`. +pub fn enclosing_span(spans: &[RecordSpan], offset: usize) -> Option<&RecordSpan> { + let index = spans.partition_point(|span| span.start <= offset); + let span = spans.get(index.checked_sub(1)?)?; + (offset < span.end).then_some(span) +} + +/// The spans of one partition, walked the first time a frame needs them. +struct PartitionSpans<'a> { + buf: &'a [u8], + header_tag: u16, + declared: &'a BTreeSet, + spans: Option>, +} + +impl<'a> PartitionSpans<'a> { + fn new(buf: &'a [u8], header_tag: u16, declared: &'a BTreeSet) -> Self { + Self { + buf, + header_tag, + declared, + spans: None, + } + } + + fn enclosing_id(&mut self, offset: usize) -> Option { + let (buf, header_tag, declared) = (self.buf, self.header_tag, self.declared); + let spans = self + .spans + .get_or_insert_with(|| record_spans(buf, header_tag, declared)); + enclosing_span(spans, offset).map(|span| span.element_id) + } +} + /// Every 2008 to 2023 element record in the file whose ElementId is -/// declared in `Global/ElemTable`. Empty for any other release (fail -/// closed). +/// declared in `Global/ElemTable`: a frame that opens with its declared id, +/// and a placed-instance frame without one (the second prologue) that sits +/// in a declared record whose id its reference list names (RE-315). Empty +/// for any other release (fail closed). pub fn scan_records(rf: &mut RevitFile, revit_version: u32) -> Vec { if !is_32bit_release(revit_version) { return Vec::new(); @@ -216,6 +362,7 @@ pub fn scan_records(rf: &mut RevitFile, revit_version: u32) -> Vec Vec, header_tag: u16, shortfall: usize, + spans: &mut PartitionSpans<'_>, ) -> Option { let id_at = at.checked_sub(ID_BEFORE_MARKER - shortfall)?; - let element_id = le_bytes::u32_at(buf, id_at)?; - if !declared.contains(&element_id) { - return None; - } - let tag_at = id_at + HEADER_TAG_AFTER_ID; - if le_bytes::u16_at(buf, tag_at)? != header_tag { - return None; - } + let leading = le_bytes::u32_at(buf, id_at)?; + let own_id = (declared.contains(&leading) + && le_bytes::u16_at(buf, id_at + HEADER_TAG_AFTER_ID) == Some(header_tag)) + .then_some(leading); let category_at = at - (CATEGORY_BEFORE_MARKER - shortfall); let builtin_category = le_bytes::i64_at(buf, category_at)?; if !(BUILTIN_CATEGORY_MIN..=BUILTIN_CATEGORY_MAX).contains(&builtin_category) { @@ -282,6 +427,25 @@ fn decode_at_marker( } let list_at = at + 8 + 48; let references = u32_list(buf, list_at).unwrap_or_default(); + let design_option = at + .checked_sub(DESIGN_OPTION_BEFORE_MARKER) + .and_then(|from| le_bytes::u32_at(buf, from)) + .filter(|option| *option != u32::MAX); + let (element_id, id_from_enclosing_record) = match own_id { + Some(id) => (id, false), + None => { + let id = enclosing_instance_id( + spans, + id_at, + leading, + builtin_category, + container, + placement_kind, + &references, + )?; + (id, true) + } + }; Some(PartitionElementRecord { stream: stream.to_string(), offset: id_at, @@ -295,11 +459,43 @@ fn decode_at_marker( preceding_reference: None, owner_reference: None, references, - id_from_enclosing_record: false, - design_option: None, + id_from_enclosing_record, + design_option, }) } +/// The ElementId of a frame with no declared id of its own (the second +/// prologue, RE-30), from the declared record around it (RE-315). +/// +/// Only a placed instance gets one (no container reference and the +/// placed-instance kind, the RE-21 rule), and only when the frame's own +/// reference list names the record's id, as an element's does (RE-23). On +/// Revit 2018's `ALD26 Before` each of the 3,350 placed-instance frames +/// without a declared id that a declared record encloses names it. A curtain +/// panel frame that opens with a count instead of `ff ff ff ff` gets none: +/// 96 of the 102 such frames sit in the record of a panel Revit's export does +/// not write (a panel's shell), and the 6 panels it does write that they +/// hold are left out with them; all 99 that open with `ff ff ff ff` are +/// panels it writes. +fn enclosing_instance_id( + spans: &mut PartitionSpans<'_>, + id_at: usize, + leading: u32, + builtin_category: i64, + container: u64, + placement_kind: u32, + references: &[u64], +) -> Option { + if container != CONTAINER_NONE || placement_kind != PLACEMENT_KIND_INSTANCE { + return None; + } + if leading != u32::MAX && builtin_category == OST_CURTAIN_WALL_PANELS { + return None; + } + let id = spans.enclosing_id(id_at)?; + references.contains(&u64::from(id)).then_some(id) +} + /// A `u32`-counted list of `u32` ElementIds at `at`, widened to `u64`. fn u32_list(buf: &[u8], at: usize) -> Option> { let count = le_bytes::u32_at(buf, at)? as usize; diff --git a/src/research/partition_level_records.rs b/src/research/partition_level_records.rs index 76e57f07..bf2dcefc 100644 --- a/src/research/partition_level_records.rs +++ b/src/research/partition_level_records.rs @@ -847,6 +847,30 @@ const PLACEMENT_KIND_AFTER_CATEGORY_2023: usize = 24; /// Bytes from the category to the record marker. const MARKER_AFTER_CATEGORY_2023: usize = 38; +/// Bytes from a Level frame's record marker back to its class word, the +/// definition ordinal of its schema class (RE-315). Every Level instance +/// frame of Revit 2018's `ALD26 Before` and `rst_basic_sample_project` holds +/// the same word there (2130), and their type symbol another (2131). +const CLASS_BEFORE_MARKER_2023: usize = 6; + +/// What a Level frame with no ElementId of its own must show before it takes +/// the id of the record around it (RE-315). A Level frame's reference list is +/// empty and its name block lies elsewhere in the stream (1.5 MB from the +/// frame on `ALD26 Before`), so the proof is the frame's class and a name +/// and elevation block of that id in the stream: a stray frame of the Level +/// category inside another element adds no Level, and cannot drop the storeys +/// by adding an id with no name. +#[derive(Debug, Clone, Copy, Default)] +struct EnclosedLevels { + /// The tag of the schema class `Level`, which the frame's class word + /// must be. `None` when the schema has none: no frame takes an + /// enclosing id. + level_tag: Option, + /// The elevation marker of the file's name blocks + /// ([`level_name_marker`]); `None` leaves no block to find. + name_marker: Option<[u8; 6]>, +} + /// Every Revit 2023 `OST_Levels` record in one inflated stream (RE-107). /// It is the 2024 record without the 24 sentinel bytes after the /// category, opened by the 32-bit chain header 2023's element records @@ -862,12 +886,29 @@ pub fn find_level_records_2023( header_tag: u16, marker: &[u8; 6], ) -> Vec { - find_level_records_32(stream, buf, declared_ids, header_tag, marker, 0) + find_level_records_32( + stream, + buf, + declared_ids, + header_tag, + marker, + 0, + EnclosedLevels::default(), + ) } /// [`find_level_records_2023`] on a record envelope `shortfall` bytes /// shorter between the category and the placement kind, as Revit 2008's /// is ([`crate::partition_element_records_2023::envelope_shortfall`]). +/// +/// A frame that does not open with its declared id (the second prologue, +/// RE-30) takes the id of the declared record around it +/// ([`crate::partition_element_records_2023::record_spans`], RE-315) when it +/// is a standalone placed record, with no container reference and the +/// placed kind of its release, and `enclosed` vouches for it: its class word +/// is `Level`'s and the stream holds a name and elevation block of that id +/// ([`levels_from_records_and_blocks`] joins on it). On Revit 2018's +/// `ALD26 Before` all 7 of Revit's storeys are such frames. fn find_level_records_32( stream: &str, buf: &[u8], @@ -875,41 +916,107 @@ fn find_level_records_32( header_tag: u16, marker: &[u8; 6], shortfall: usize, + enclosed: EnclosedLevels, ) -> Vec { + use crate::partition_element_records::PLACEMENT_KIND_INSTANCE; let needle = (OST_LEVELS as u64).to_le_bytes(); - memchr::memmem::find_iter(buf, &needle) - .filter_map(|category| { - let id_at = category.checked_sub(ID_BEFORE_CATEGORY_2023)?; - let element_id = le_bytes::u32_at(buf, id_at)?; - if element_id == 0 || !declared_ids.contains(&element_id) { - return None; + let mut spans: Option> = None; + let mut block_ids: Option> = None; + let mut out = Vec::new(); + for category in memchr::memmem::find_iter(buf, &needle) { + let Some(id_at) = category.checked_sub(ID_BEFORE_CATEGORY_2023) else { + continue; + }; + let Some(leading) = le_bytes::u32_at(buf, id_at) else { + continue; + }; + let marker_at = category + MARKER_AFTER_CATEGORY_2023 - shortfall; + if buf.get(marker_at..marker_at + marker.len()) != Some(&marker[..]) { + continue; + } + let (Some(flags), Some(container), Some(placement_kind)) = ( + le_bytes::u32_at(buf, id_at + 4), + le_bytes::u32_at(buf, category + CONTAINER_AFTER_CATEGORY_2023 - shortfall), + le_bytes::u32_at( + buf, + category + PLACEMENT_KIND_AFTER_CATEGORY_2023 - shortfall, + ), + ) else { + continue; + }; + let own = leading != 0 + && declared_ids.contains(&leading) + && le_bytes::u16_at(buf, id_at + HEADER_TAG_AFTER_ID_2023) == Some(header_tag); + let element_id = if own { + leading + } else { + if container != u32::MAX + || !matches!( + placement_kind, + PLACEMENT_KIND_INSTANCE | LEVEL_PLACEMENT_KIND_2015 + ) + { + continue; } - if le_bytes::u16_at(buf, id_at + HEADER_TAG_AFTER_ID_2023)? != header_tag { - return None; + let class = marker_at + .checked_sub(CLASS_BEFORE_MARKER_2023) + .and_then(|at| le_bytes::u16_at(buf, at)); + if enclosed.level_tag.is_none() || class != enclosed.level_tag { + continue; } - let marker_at = category + MARKER_AFTER_CATEGORY_2023 - shortfall; - if buf.get(marker_at..marker_at + marker.len())? != marker { - return None; + let spans = spans.get_or_insert_with(|| { + crate::partition_element_records_2023::record_spans(buf, header_tag, declared_ids) + }); + let id = match crate::partition_element_records_2023::enclosing_span(spans, id_at) { + Some(span) if span.element_id != 0 => span.element_id, + _ => continue, + }; + let block_ids = block_ids.get_or_insert_with(|| { + enclosed + .name_marker + .map(|name_marker| { + find_name_blocks_sized( + buf, + declared_ids, + &name_marker, + 4, + OWNER_SENTINEL_RUN_LEN_2023, + ) + .into_iter() + .map(|block| block.element_id) + .collect::>() + }) + .unwrap_or_default() + }); + if !block_ids.contains(&id) { + continue; } - Some(PartitionLevelRecord { - stream: stream.to_string(), - offset: id_at, - element_id, - flags: le_bytes::u32_at(buf, id_at + 4)?, - container: match le_bytes::u32_at( - buf, - category + CONTAINER_AFTER_CATEGORY_2023 - shortfall, - )? { - u32::MAX => u64::MAX, - id => u64::from(id), - }, - placement_kind: le_bytes::u32_at( - buf, - category + PLACEMENT_KIND_AFTER_CATEGORY_2023 - shortfall, - )?, - }) - }) - .collect() + id + }; + out.push(PartitionLevelRecord { + stream: stream.to_string(), + offset: id_at, + element_id, + flags, + container: match container { + u32::MAX => u64::MAX, + id => u64::from(id), + }, + placement_kind, + }); + } + out +} + +/// The elevation marker of a release's Level name blocks: Revit 2008 to 2012 +/// frame the `Plane` reference ([`framed_plane_marker`]), 2013 and later +/// write it unframed ([`elevation_marker`], else the schema's). +fn level_name_marker(rf: &mut RevitFile, revit_version: u32) -> Option<[u8; 6]> { + if revit_version < FRAMED_PLANE_UNTIL { + framed_plane_marker(rf) + } else { + elevation_marker(revit_version).or_else(|| schema_elevation_marker(rf)) + } } /// The placement kind Revit 2015 to 2018 write on a Level element, where @@ -937,6 +1044,16 @@ fn level_records( }; let mut marker = [0u8; 6]; marker.copy_from_slice(&full[2..]); + let enclosed = EnclosedLevels { + level_tag: rf.memoised_schema_classes().ok().and_then(|classes| { + classes + .classes + .iter() + .find(|class| class.name == "Level") + .map(|class| class.tag) + }), + name_marker: level_name_marker(rf, revit_version), + }; let mut records = Vec::new(); for stream in rf.partition_stream_names() { let Ok(inflated) = rf.inflated_partition(&stream) else { @@ -949,6 +1066,7 @@ fn level_records( header_tag, &marker, crate::partition_element_records_2023::envelope_shortfall(revit_version), + enclosed, )); } if revit_version < 2019 { @@ -998,12 +1116,7 @@ pub fn scan_partition_levels( if !supports_revit_version(revit_version) || declared_ids.is_empty() { return Ok(Vec::new()); } - let marker = if revit_version < FRAMED_PLANE_UNTIL { - framed_plane_marker(rf) - } else { - elevation_marker(revit_version).or_else(|| schema_elevation_marker(rf)) - }; - let Some(marker) = marker else { + let Some(marker) = level_name_marker(rf, revit_version) else { return Ok(Vec::new()); }; let records = level_records(rf, revit_version, declared_ids); diff --git a/src/research/partition_schema_mvp/records_2023.rs b/src/research/partition_schema_mvp/records_2023.rs index 01cc3cba..97adf84e 100644 --- a/src/research/partition_schema_mvp/records_2023.rs +++ b/src/research/partition_schema_mvp/records_2023.rs @@ -8,8 +8,9 @@ use super::*; /// feeds. Components nested in doors and windows are left out /// ([`nested_in_openings`]). Doors and windows bind to their host wall by /// the 2024 rule (RE-85) on 2023's reference lists, and every element to -/// the one Level its list names (RE-107). No names, types, joins or design -/// options: their 2023 layouts are not decoded. +/// the one Level its list names (RE-107). On 2016 to 2023 an element in a +/// non-primary design option, or not in the export phase, is left out +/// (RE-315). pub(super) fn recover_2023_records( rf: &mut RevitFile, revit_version: u32, @@ -19,7 +20,16 @@ pub(super) fn recover_2023_records( let records = crate::partition_element_records_2023::scan_records(rf, revit_version); let names = type_and_family_names_2023(rf, &records); let system_types = system_type_names_2023(rf, revit_version, &records); - let selected = select_newest_instance_records(records); + let mut selected = select_newest_instance_records(records); + // Revit's export leaves out an element in a non-primary design option and + // one that does not exist in the export phase, as on 2024 (RE-40, RE-184). + // Rooms are never left out by phase. Applied on the releases measured only + // (RE-315). + if crate::partition_element_records_2023::applies_export_filters(revit_version) { + let options = rf.memoised_design_options(); + let phases = rf.memoised_phase_filter(); + selected.retain(|_, record| !options.excludes(record) && !phases.excludes(record)); + } // Only a family instance can be nested; a host wall and the doors it // hosts name each other too (RE-76's class tag, at the same place on // 2023). diff --git a/tools/re/instances_vs_ifc_tags.py b/tools/re/instances_vs_ifc_tags.py index b57f2d1d..8133da44 100755 --- a/tools/re/instances_vs_ifc_tags.py +++ b/tools/re/instances_vs_ifc_tags.py @@ -9,7 +9,9 @@ Usage: - python3 tools/re/instances_vs_ifc_tags.py + python3 tools/re/instances_vs_ifc_tags.py [--list] + +`--list` is what Measure passes to a probe's scorer; it changes nothing here. """ import collections @@ -19,6 +21,7 @@ def main(argv): + argv = [arg for arg in argv if arg != "--list"] if len(argv) != 3: print(__doc__, file=sys.stderr) return 2