Skip to content

docs(criteria): re-measure the 1.0 requirements from the 2026-10-27 main run (#195) #401

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
DrunkOnJava merged 5 commits into from
Oct 28, 2026
Merged

docs(criteria): re-measure the 1.0 requirements from the 2026-10-27 main run (#195) #401

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
f540e7e
docs(spec-coverage): move the rows the 2026-10-27 main run proves, an…
DrunkOnJava Oct 27, 2026
7d76edf
docs(criteria): re-measure the 1.0 requirements from the 2026-10-27 m…
DrunkOnJava Oct 27, 2026
27bc149
Merge #398's branch (docs/spec-coverage-refresh) so the criteria coun…
DrunkOnJava Oct 27, 2026
1c8d508
docs(criteria): R3 counts the rows #398 moves: 97 closes, 7 partial, …
DrunkOnJava Oct 27, 2026
76bd093
Merge remote-tracking branch 'origin/main' into HEAD
DrunkOnJava Oct 28, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions docs/release-1.0-criteria.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# 1.0.0 release criteria

Last measured 2026-10-21, on `main` at `b268e884`.
Last measured 2026-10-27, on `main` at `61255e9` ([CI run 40542129639](https://github.com/DrunkOnJava/dwg-rs/actions/runs/40542129639), artifact `corpus-measurement`).

1.0.0 is the first release with a stability commitment
([`RELEASE.md`](../RELEASE.md)). The maintainer set its scope on
Expand All @@ -27,13 +27,13 @@ status changes only in the pull request that links the new evidence.
| | Requirement | Bar | Measured now | Status |
|---|---|---|---|---|
| R1 | Measured in public | Every number README.md, STATUS.md and this page publish comes from a CI run on files anyone can fetch, with the run linked | README.md's and STATUS.md's real-file numbers come from the Corpus CI job on 52 public drawings, run linked; STATUS.md's other numbers and this page's do not yet (#94, #112) | **failed** |
| R2 | Every release opens | R13, R14, R2000, R2004, R2007, R2010, R2013 and R2018 files open and walk, on at least two corpus files per release | R14 to R2018 open and walk in CI on at least four drawings per release (eight each for R14 to R2013, four for R2018, all AutoCAD saves with recorded provenance); no R13 file (#119) | **failed** |
| R3 | Every section of the spec is read | Every row of [`spec-coverage.md`](spec-coverage.md) is `complete`: chapters 2 to 29 and all 104 prescriptions of §20.4 | 0 of 138 rows `complete`; 34 `missing`, 31 `partial`, 6 `decoder`, 67 `closes` (CI corpus, #225) (#118, #125, #131, #138) | **failed** |
| R4 | Values equal AutoCAD's | For every section, every value equals AutoCAD's DXF export of the same drawing, matched by handle | The DXF value oracle (`examples/dxf_oracle`, #99) compares the decoded fields of 46 sections on R2000-R2018 (entities also on R14) over the nextgis and ACadSharp exports in the Corpus CI job, which fails when an agreeing count falls below `tools/oracle/baseline.tsv` (#103): 19 values disagree (#241-#244), every other decoded value agrees; most sections still have DXF groups no decoder surfaces, and no R13 drawing is in the corpus | **failed** |
| R2 | Every release opens | R13, R14, R2000, R2004, R2007, R2010, R2013 and R2018 files open and walk, on at least two corpus files per release | R14 to R2018 open and walk in CI on at least four drawings per release (eight each for R14 to R2013, four for R2018, all AutoCAD saves with recorded provenance; 52 drawings, 15,541 records decoded, 0 errored, `corpus-results.json` of [the run](https://github.com/DrunkOnJava/dwg-rs/actions/runs/40542129639)); no R13 file (#119) | **failed** |
| R3 | Every section of the spec is read | Every row of [`spec-coverage.md`](spec-coverage.md) is `complete`: chapters 2 to 29 and all 104 prescriptions of §20.4 | 0 of 138 rows `complete`; 28 `missing`, 7 `partial`, 6 `decoder`, 97 `closes` (`docs/data/spec-coverage.tsv` with #398, whose seven moves to `closes` and §22 to `partial` are proven by [CI run 40532346172](https://github.com/DrunkOnJava/dwg-rs/actions/runs/40532346172); `closes` is measured by the Corpus CI job, #225) (#118, #125, #131, #138) | **failed** |
| R4 | Values equal AutoCAD's | For every section, every value equals AutoCAD's DXF export of the same drawing, matched by handle | The DXF value oracle (`examples/dxf_oracle`, #99) compares the decoded fields of 53 sections on R2000-R2018 (entities also on R14) over the nextgis and ACadSharp exports in the Corpus CI job, which fails when an agreeing count falls below `tools/oracle/baseline.tsv` (#103): 22 values disagree (#241-#244, #381; `oracle-summary.md` of [the run](https://github.com/DrunkOnJava/dwg-rs/actions/runs/40542129639)), every other decoded value agrees; most sections still have DXF groups no decoder surfaces, and no R13 drawing is in the corpus | **failed** |
| R5 | Classes beyond the spec | Every class the corpus holds that v5.4.1 does not prescribe (VISUALSTYLE, MATERIAL, surfaces and the rest) meets R3 and R4 | VISUALSTYLE and the view styles close; the DXF oracle finds every VISUALSTYLE value of 1,008 records (R2000-R2018) equal to the export but `format_version` on R2013-R2018 and Realistic's face lighting quality on R2000-R2007 (264 values, #250), and does not score the view styles yet; MATERIAL, PROPERTYSET_DATA, MESH and others partial or unmeasured (#152) | **failed** |
| R6 | A corpus that holds everything | Every (section, release) pair has at least one corpus record with a DXF export, from sources with a documented right to fetch them | Two public sources fetched in CI: nextgis (7 simple drawings, R14 to R2013) and ACadSharp (one dense drawing, R14 to R2018, plus three R2018 drawings); AutoCAD's DXF beside every save but R14 and two of the extra R2018 drawings; no R13 (#153) | **failed** |
| R7 | Held-out files | At least two files per release band, used by no development work and measured once before any fix, meet R3 and R4 | None reserved (#158, #196) | **not measured** |
| R8 | Nothing silently wrong | A record that does not decode is reported (diagnostics name its type and why); no decoder returns plausible values for a layout it did not verify | 308 records error honestly on the public corpus (#226, #227, #228, #71); the DXF oracle found five decoders that closed on their boundaries with wrong values, all fixed (#235, #236, #238, #251, #253); AppInfo is misread (#137) | **failed** |
| R8 | Nothing silently wrong | A record that does not decode is reported (diagnostics name its type and why); no decoder returns plausible values for a layout it did not verify | 0 records error on the public corpus and 15,541 decode (the 308 that errored at #237 were fixed: #226, #227, #228, #71), and 533 are skipped for want of a decoder, each named by kind in the skip histogram (`corpus-results.json` of [the run](https://github.com/DrunkOnJava/dwg-rs/actions/runs/40542129639)); the DXF oracle found five decoders that closed on their boundaries with wrong values, all fixed (#235, #236, #238, #251, #253); AppInfo is misread (#137) | **failed** |

## Writing

Expand All @@ -47,7 +47,7 @@ status changes only in the pull request that links the new evidence.

| | Requirement | Bar | Measured now | Status |
|---|---|---|---|---|
| X1 | DXF | DWG to DXF equals AutoCAD's own DXF of the same drawing, for every target version the source supports | Written for R12 to R2018; checked only syntactically (#169) | **not measured** |
| X1 | DXF | DWG to DXF equals AutoCAD's own DXF of the same drawing, for every target version the source supports | Written for R12 to R2018. The HEADER, TABLES, ENTITIES and BLOCKS sections each have a probe in the Corpus CI job that pairs every written record with AutoCAD's export of the same drawing by handle and compares it group by group; on the 42 drawings with an export of their release (10 are skipped for want of one), 0 groups disagree and none is written that the export omits: HEADER 9,583 variables (9,410 agree, 173 hold a stated relation), TABLES 1,239 records (8,831 of 8,840 groups agree, 9 by a stated relation), ENTITIES 803 records (5,627 of 5,627 groups), BLOCKS 1,532 records (12,577 of 12,583 groups, 6 by a stated relation, #243; `dxf-*-probe.log` of [the run](https://github.com/DrunkOnJava/dwg-rs/actions/runs/40542129639)). Not equal yet: 342 ENTITIES records of 18 kinds are not written (DIMENSION 66, MULTILEADER 90, HATCH 48, TOLERANCE 18, MLINE 18 and others) and the OBJECTS section is not written on `main` (#169) | **failed** |
| X2 | SVG and PDF | Rendered geometry equals the decoded geometry for every 2D entity type, within a stated tolerance | Rendered; not measured (#171) | **not measured** |
| X3 | glTF | Every export passes the Khronos validator; solids drawn from their ACIS data | 3DSOLID is a box placeholder (#172, #173) | **failed** |
| X4 | Viewer | No stubs; browser tests open every corpus file; zero network requests after load | Selection, section box, clipping, worker and progressive open are stubs (#174) | **failed** |
Expand All @@ -59,7 +59,7 @@ status changes only in the pull request that links the new evidence.
|---|---|---|---|---|
| S1 | The stable surface is named | RELEASE.md lists what 1.0 commits to and what it does not | Every module public; nothing named (#192) | **failed** |
| S2 | Breaking changes are caught | cargo-semver-checks and the Python API listing fail CI on an unannounced break | Not checked (#192) | **failed** |
| S3 | Hostile input | One ParseLimits holds every cap; every fuzz target has run an hour with no findings; no panic on 64- or 32-bit targets | Separate limit structs; no recorded fuzz durations; no 32-bit job (#184) | **failed** |
| S3 | Hostile input | One ParseLimits holds every cap; every fuzz target has run an hour with no findings; no panic on 64- or 32-bit targets | Separate limit structs; no recorded fuzz durations; the library compiles for `wasm32-unknown-unknown` in the Clippy job (step "The library builds where usize is 32 bits", success in the run), but no job runs tests or a corpus walk on a 32-bit target (#184) | **failed** |
| S4 | Published | crates.io, docs.rs, PyPI and the release binaries checked from what was published | Nothing published (#109, #182) | **failed** |

## Outside evidence
Expand Down
Loading