diff --git a/.gitignore b/.gitignore index 256b9114..8d32c9af 100644 --- a/.gitignore +++ b/.gitignore @@ -1,6 +1,6 @@ # Cargo build artifacts /target/ -# Nested target/ for out-of-workspace sub-crates (e.g. tools/synthetic-dwg). +# Nested target/ for out-of-workspace sub-crates (wasm/, fuzz/). **/target/ # Separate CARGO_TARGET_DIR used to verify the MSRV without evicting the # stable toolchain's incremental cache: diff --git a/CHANGELOG.md b/CHANGELOG.md index f7c7bba5..632b7157 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -182,7 +182,7 @@ No library behaviour changes in this entry. - **CI gates.** Perf benches the PR's merge-base and the PR in one job on one host, instead of comparing against a baseline cached from another runner, which had measured hardware and flagged PRs that touched no benchmarked code (#217). A Spec coverage job runs `tools/ci/spec_coverage.py`, so `docs/spec-coverage.md` keeps one row for every section of ODA spec v5.4.1 and never drifts from `docs/data/spec-coverage.tsv` (#219). The PII guard also refuses merge-conflict markers (#309), and the Spec coverage job refuses ARCHITECTURE.md sections that repeat or skip a letter (#334). The nightly fuzz run covers its whole budget and fails on a crash (#232). Every cargo workflow sets `CARGO_NET_RETRY=10` (#268), and the wasm-pack install retries up to five times on a transient download failure (#342). `main` requires Corpus and Spec coverage (#292); docs/maintaining.md describes the `main-protection` ruleset, with no bypass, as it is (#220). `tools/maintainer/clean_caches.py`, which `audit.py` already pointed to, removes the Actions caches of closed pull requests and the superseded ones per key family (#361), the cache having stood at 9.95 GB of its 10 GB. The five actionlint findings on `main` are cleared without changing what a workflow does (#269): the PII guard's backslash-ended patterns are double-quoted, the docs.rs docstring ratio counts with `awk` and quoted arrays, and `publish-pypi` in `release.yml` stays off through a condition that is never true rather than a literal `if: false`, which actionlint rejects. - **Merge train.** `tools/maintainer/train/train.sh` lands queued decoder PRs one at a time, regenerating the generated files (`tools/corpus/baseline.tsv`, `docs/spec-coverage.md`, the spec-example fixtures) and renumbering colliding ARCHITECTURE.md sections instead of leaving every open PR stale after each merge (#298). It ratchets the DXF oracle baseline as well (#300), defers a PR whose build is refused a governor lease instead of dropping it from the queue (#304), resolves both-sides appends to CLEANROOM.md (#327), never leaves auto-merge on a PR it stops landing (#336), resets its worktree before each checkout (#338), ignores oracle rows the PR itself renamed (#344) and, after main carried three `7n` sections, numbers ARCHITECTURE.md's newer sections in landing order (#322). - **Documentation.** CLAUDE.md for coding agents and contributor docs brought up to date, including the decoder registration steps and the rule that evidence is real-file, not unit tests (#218, #257); a 10-minute contributor quickstart at the top of CONTRIBUTING.md (#18); a CLEANROOM.md entry that a class layout the spec never prints is derived from AutoCAD's own DWG and its DXF export, accepted only when it closes every record's boundary on every release and every decoded value equals the twin's group (#321); AGENTS.md gives a reviewer the clean-room rule (#369). -- **Fixtures and benchmarks.** `examples/build_fixtures` writes the four canonical R2004-family fixtures, five entities each, byte-deterministically, so the coverage-smoke job has a corpus to run against instead of skipping, with a 95 % floor on the decoded ratio and a noise floor for the Criterion job (#19). +- **Fixtures and benchmarks.** `examples/build_fixtures` writes the four canonical R2004-family fixtures, five entities each, byte-deterministically, so the coverage-smoke job has a corpus to run against instead of skipping, with a 95 % floor on the decoded ratio and a noise floor for the Criterion job (#19). `tools/synthetic-dwg`, an out-of-workspace R14 generator that no CI job built, is removed (#20); `examples/build_fixtures` supersedes it, and the four byte-identical sample files it produced for the viewer stay, with their provenance recorded in `wasm/examples/samples/README.md`. - **Dependencies.** thiserror 1.0.69 to 2.0.20 (#3) and 2.0.21 (#93), criterion 0.5.1 to 0.7.0 (#4), the cargo patch group (anyhow, clap, serde, serde_json) in #11 and #93, libfuzzer-sys 0.4.13 in `/fuzz` (#200), js-sys and serde in `/wasm` (#201); the Actions `upload-artifact` 7.0.1 (#5), `configure-pages` 6.0.0 (#6), `download-artifact` 8.0.1 (#7) and 8.0.2 (#202), `cache/save` 6.1.0 (#85), `softprops/action-gh-release` 3.0.3 (#86), `gitleaks-action` (#87, #92 to 3.0.0), `deploy-pages` 5.0.1 (#90), `checkout` 7.0.1 (#91) and `upload-pages-artifact` 5.0.0 (#203). - **Direct commits before the PRs, 2026-04-20 to 2026-04-25.** Pre-alpha claims toned down across the public surfaces and the uniqueness framing removed (58753a1, 566711b, 7ec595b), HATCH path coverage documented (fc3ee5c), rustdoc intra-doc links fixed and two let-chains in `svg.rs` split so the crate compiles on the 1.85 MSRV (de37ead), wasm-opt run with bulk-memory enabled so `wasm-pack build` works (b0f49b9), and the wasm attestation (V-19) scoped to the Import section with Pages auto-enabled (be3a71d). diff --git a/CLAUDE.md b/CLAUDE.md index a66c0653..1f799254 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -59,7 +59,7 @@ python3 tools/streaming/verify.py target/debug/examples/stream_header target/deb Features: `lz77-trace` (plus `DWG_LZ77_TRACE=1` at runtime) traces the LZ77 decoder; `dhat-heap` enables `examples/dhat_profile.rs`. Benches are Criterion (`cargo bench`); the Perf workflow fails a PR only when a bench is at least 20% and at least 5 µs slower than `main` and repeats on a re-run. Measure before touching `bitcursor.rs`, `bitwriter.rs` or `lz77*.rs`. -`wasm/` (`wasm-pack build --target web --release`), `fuzz/` (`cargo +nightly fuzz run `, targets in `fuzz/fuzz_targets/`) and `tools/synthetic-dwg/` are separate crates, not workspace members. `python3 tools/maintainer/audit.py` runs the read-only GitHub housekeeping checks from `docs/maintaining.md` (needs `gh`). +`wasm/` (`wasm-pack build --target web --release`) and `fuzz/` (`cargo +nightly fuzz run `, targets in `fuzz/fuzz_targets/`) are separate crates, not workspace members. `python3 tools/maintainer/audit.py` runs the read-only GitHub housekeeping checks from `docs/maintaining.md` (needs `gh`). ## Real-file corpus diff --git a/Cargo.toml b/Cargo.toml index 552e1556..325fb04a 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -18,9 +18,9 @@ exclude = [ "fuzz/", # Developer-experience scaffolding. `templates/` contains a # cargo-generate template for entity decoders (handlebars - # placeholders, not valid Rust); `tools/` contains auxiliary - # binaries like the synthetic-DWG fixture generator. Neither - # belongs in the published library crate. + # placeholders, not valid Rust); `tools/` contains the corpus, + # oracle, streaming and maintainer scripts. Neither belongs in the + # published library crate. "templates/", "tools/", # Note: CHANGELOG.md, README.md, SECURITY.md, ARCHITECTURE.md, and diff --git a/STATUS.md b/STATUS.md index 052b813f..f610c12f 100644 --- a/STATUS.md +++ b/STATUS.md @@ -252,8 +252,8 @@ the current capability matrix. - 11 launch posts (blog, HN, r/rust, r/cad, LinkedIn, Reddit, Twitter, LibreCAD/FreeCAD/QCAD forums, ODA community). - 11 DEV docs (CONTRIBUTING, CLEANROOM, THREAT_MODEL, EXTENDING_DECODERS, - entity-decoder cargo-generate template, synthetic DWG generator, - GitHub Discussions, issue + PR templates). + entity-decoder cargo-generate template, GitHub Discussions, + issue + PR templates). ### Security (SEC-series) - `DecompressLimits` + LZ77 output cap (SEC-01, 02, 03, 04). diff --git a/tools/maintainer/audit.py b/tools/maintainer/audit.py index c529a6c6..641f0416 100644 --- a/tools/maintainer/audit.py +++ b/tools/maintainer/audit.py @@ -30,12 +30,12 @@ # A research or Measure branch with no pull request is kept while its work is # live; one whose last commit is older than this is stale (docs/maintaining.md). BRANCH_STALE_DAYS = 14 -# Directories with a lockfile that Dependabot deliberately does not update, and why -# (docs/maintaining.md says the same). # The 1.0 release decision; every open issue under the 1.0.0 milestone serves one of # its requirements (docs/maintaining.md, housekeeping item 8). RELEASE_CRITERIA = ROOT / "docs" / "release-1.0-criteria.md" -UNWATCHED = {"/tools/synthetic-dwg": "superseded by examples/build_fixtures.rs; its removal is #20"} +# Directories with a lockfile that Dependabot deliberately does not update, mapped to +# why. None today. +UNWATCHED = {} results = [] diff --git a/tools/synthetic-dwg/Cargo.lock b/tools/synthetic-dwg/Cargo.lock deleted file mode 100644 index e6d37475..00000000 --- a/tools/synthetic-dwg/Cargo.lock +++ /dev/null @@ -1,79 +0,0 @@ -# This file is automatically @generated by Cargo. -# It is not intended for manual editing. -version = 4 - -[[package]] -name = "byteorder" -version = "1.5.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "1fd0f2584146f6f2ef48085050886acf353beff7305ebd1ae69500e27c67f64b" - -[[package]] -name = "dwg" -version = "0.1.0-alpha.1" -dependencies = [ - "byteorder", - "thiserror", -] - -[[package]] -name = "proc-macro2" -version = "1.0.106" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "8fd00f0bb2e90d81d1044c2b32617f68fcb9fa3bb7640c23e9c748e53fb30934" -dependencies = [ - "unicode-ident", -] - -[[package]] -name = "quote" -version = "1.0.45" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "41f2619966050689382d2b44f664f4bc593e129785a36d6ee376ddf37259b924" -dependencies = [ - "proc-macro2", -] - -[[package]] -name = "syn" -version = "2.0.117" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "e665b8803e7b1d2a727f4023456bbbbe74da67099c585258af0ad9c5013b9b99" -dependencies = [ - "proc-macro2", - "quote", - "unicode-ident", -] - -[[package]] -name = "synthetic-dwg" -version = "0.0.0" -dependencies = [ - "dwg", -] - -[[package]] -name = "thiserror" -version = "1.0.69" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "b6aaf5339b578ea85b50e080feb250a3e8ae8cfcdff9a461c9ec2904bc923f52" -dependencies = [ - "thiserror-impl", -] - -[[package]] -name = "thiserror-impl" -version = "1.0.69" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "4fee6c4efc90059e10f81e6d42c60a18f76588c3d74cb83a0b242a2b6c7504c1" -dependencies = [ - "proc-macro2", - "quote", - "syn", -] - -[[package]] -name = "unicode-ident" -version = "1.0.24" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75" diff --git a/tools/synthetic-dwg/Cargo.toml b/tools/synthetic-dwg/Cargo.toml deleted file mode 100644 index 7246c033..00000000 --- a/tools/synthetic-dwg/Cargo.toml +++ /dev/null @@ -1,8 +0,0 @@ -[package] -name = "synthetic-dwg" -version = "0.0.0" -edition = "2024" -publish = false - -[dependencies] -dwg = { path = "../.." } diff --git a/tools/synthetic-dwg/src/main.rs b/tools/synthetic-dwg/src/main.rs deleted file mode 100644 index d2389a33..00000000 --- a/tools/synthetic-dwg/src/main.rs +++ /dev/null @@ -1,220 +0,0 @@ -//! `synthetic-dwg` — build a minimal valid DWG file from scratch. -//! -//! Useful as a test-fixture generator for unit tests and fuzz seeds -//! without requiring a real AutoCAD / Autodesk-produced corpus file -//! on disk. Produces a file that [`dwg::DwgFile::open`] accepts — it -//! is NOT guaranteed to open in AutoCAD. -//! -//! # Chosen version: R14 (AC1014) -//! -//! The binary emits an **R14 (AC1014)** file, the simplest of the -//! supported version families. R2004+ would require a correctly- -//! masked R2004 header block, a Section Page Map with valid page -//! numbers, LZ77-framed section payloads, and CRC-32 over the -//! decrypted header — all doable but ~10× the code for no extra -//! test coverage. R2007 has its own layout and isn't supported by -//! the reader yet anyway. R14's flat locator table matches what the -//! reader's `R13R15Header::parse` consumes (§3.2.6): magic + common -//! prefix + u32 count + N × 9-byte locator records. -//! -//! # Layout emitted -//! -//! ```text -//! 0x00 "AC1014" magic (6 bytes) -//! 0x06 00 00 00 00 00 zero padding -//! 0x0B maint_version (1) arbitrary (0) -//! 0x0C 00 -//! 0x0D image_seeker u32 0 (no preview) -//! 0x11 00 00 reserved -//! 0x13 codepage u16 30 (ANSI_1252, the common default) -//! 0x15 locator_count u32 5 -//! 0x19 locator records 5 × 9 bytes -//! ... a small "fake LINE entity" payload stored at the -//! offset the OBJECT_MAP locator points at. The byte shape -//! matches the entity-stream preamble expected by the LINE -//! decoder (zflag=2D, start=(0,0,0), end=(100,100,0)). -//! ``` - -use dwg::DwgFile; -use dwg::bitwriter::BitWriter; -use std::env; -use std::error::Error; -use std::fs; -use std::path::PathBuf; -use std::process::ExitCode; - -const MAGIC: &[u8; 6] = b"AC1014"; - -fn print_help() { - println!( - "synthetic-dwg — build a minimal valid DWG file\n\ - \n\ - USAGE:\n\ - \x20 synthetic-dwg [--read-back]\n\ - \n\ - ARGS:\n\ - \x20 where to write the generated .dwg file\n\ - \n\ - FLAGS:\n\ - \x20 --read-back after writing, re-open the file via DwgFile::open\n\ - \x20 and print the parsed metadata, to verify round-trip\n\ - \x20 -h, --help show this message\n\ - \n\ - VERSION EMITTED:\n\ - \x20 AC1014 (AutoCAD R14 / 1997). Chosen for minimal layout —\n\ - \x20 R2004+ files require encrypted headers + LZ77 section\n\ - \x20 frames which aren't needed for a test fixture.\n\ - \n\ - NOTE:\n\ - \x20 The output is a test fixture — it satisfies dwg-rs's reader\n\ - \x20 but is NOT guaranteed to open in AutoCAD.\n" - ); -} - -/// Build a synthetic R14 DWG file in memory. Returns the raw bytes. -fn build_minimal_r14() -> Vec { - // --- Fake LINE entity payload (matches entities/line.rs::decode) --- - // - // zflag=true (2D), start.x=0, dx=100 (so end.x=100), - // start.y=0, dy=100 (end.y=100), thickness=default, extrusion=default. - let mut w = BitWriter::new(); - w.write_b(true); // zflag: 2D line - w.write_rd(0.0); // start.x - w.write_bd(100.0); // delta to end.x (=> end.x=100) - w.write_rd(0.0); // start.y - w.write_bd(100.0); // delta to end.y (=> end.y=100) - w.write_b(true); // thickness: default (0.0) - w.write_b(true); // extrusion: default (0,0,1) - let line_payload = w.into_bytes(); - - // --- Compute file layout offsets --- - // - // Header prefix is 0x19 bytes, then 5 × 9-byte locator records = - // 45 bytes, ending at 0x19 + 45 = 0x46. - const HEADER_END: usize = 0x19 + 5 * 9; // 0x46 - let object_map_offset: u32 = HEADER_END as u32; // 0x46 - let object_map_size: u32 = line_payload.len() as u32; - - // --- Build the 0x19-byte header prefix --- - let mut bytes = Vec::with_capacity(256); - // 0x00: magic - bytes.extend_from_slice(MAGIC); - // 0x06-0x0A: zero padding (5 bytes) - bytes.extend_from_slice(&[0u8; 5]); - // 0x0B: maint_version - bytes.push(0x00); - // 0x0C: zero - bytes.push(0x00); - // 0x0D-0x10: image_seeker u32 (little-endian) — 0, no preview - bytes.extend_from_slice(&0u32.to_le_bytes()); - // 0x11-0x12: reserved - bytes.extend_from_slice(&[0u8; 2]); - // 0x13-0x14: codepage u16 — 30 = ANSI_1252 (common default) - bytes.extend_from_slice(&30u16.to_le_bytes()); - debug_assert_eq!(bytes.len(), 0x15); - - // 0x15-0x18: locator_count u32 = 5 - bytes.extend_from_slice(&5u32.to_le_bytes()); - debug_assert_eq!(bytes.len(), 0x19); - - // 0x19+: 5 × 9-byte locator records. Record format: - // u8 number | u32 seeker | u32 size (9 bytes total) - // Records 0-4 are HEADER, CLASSES, OBJECT_MAP, UNKNOWN_C3, MEASUREMENT. - for i in 0u8..5 { - let (seeker, size): (u32, u32) = match i { - // Point OBJECT_MAP (record 2) at the line entity payload we - // append after the header. The other records are stubs with - // zero size — valid for the parser, which doesn't traverse - // the pointed-at byte range during open. - 2 => (object_map_offset, object_map_size), - _ => (0, 0), - }; - bytes.push(i); - bytes.extend_from_slice(&seeker.to_le_bytes()); - bytes.extend_from_slice(&size.to_le_bytes()); - } - debug_assert_eq!(bytes.len(), HEADER_END); - - // --- Append the fake entity payload + trailing CRC sentinel --- - bytes.extend_from_slice(&line_payload); - // Trailing 4-byte zero "CRC sentinel". R14 stores a per-section - // CRC-8 + a file-level CRC-32, neither of which the reader - // verifies at open time (the parser only uses them for strict - // validation paths). Write zeros so file size looks plausible. - bytes.extend_from_slice(&[0u8; 4]); - - bytes -} - -fn run(out_path: PathBuf, read_back: bool) -> Result<(), Box> { - let bytes = build_minimal_r14(); - fs::write(&out_path, &bytes)?; - eprintln!( - "synthetic-dwg: wrote {} bytes to {}", - bytes.len(), - out_path.display() - ); - - if read_back { - let file = DwgFile::open(&out_path) - .map_err(|e| format!("read-back failed: DwgFile::open errored: {e}"))?; - println!(); - println!("round-trip OK:"); - println!(" version: {}", file.version()); - println!(" release: AutoCAD {}", file.version().release()); - println!(" file_size: {} bytes", file.file_size()); - println!(" section_count: {}", file.sections().len()); - for s in file.sections() { - println!( - " - {:<18} kind={:<12} offset={:>6} size={:>6}", - s.name, - s.kind.short_label(), - s.offset, - s.size - ); - } - } - Ok(()) -} - -fn main() -> ExitCode { - // Minimal arg parsing — we intentionally don't depend on clap so - // this binary stays tiny and doesn't drag in the dwg crate's - // `cli` feature set. - let args = env::args().skip(1); - let mut out_path: Option = None; - let mut read_back = false; - for arg in args { - match arg.as_str() { - "-h" | "--help" => { - print_help(); - return ExitCode::SUCCESS; - } - "--read-back" => read_back = true, - other if other.starts_with("--") => { - eprintln!("synthetic-dwg: unknown flag: {other}"); - print_help(); - return ExitCode::FAILURE; - } - other => { - if out_path.is_some() { - eprintln!("synthetic-dwg: unexpected extra arg: {other}"); - return ExitCode::FAILURE; - } - out_path = Some(PathBuf::from(other)); - } - } - } - let Some(out_path) = out_path else { - eprintln!("synthetic-dwg: missing "); - print_help(); - return ExitCode::FAILURE; - }; - match run(out_path, read_back) { - Ok(_) => ExitCode::SUCCESS, - Err(e) => { - eprintln!("synthetic-dwg: {e}"); - ExitCode::FAILURE - } - } -} diff --git a/wasm/examples/samples/README.md b/wasm/examples/samples/README.md index f41bfe49..530460df 100644 --- a/wasm/examples/samples/README.md +++ b/wasm/examples/samples/README.md @@ -1,26 +1,28 @@ # dwg-wasm sample fixtures -Tiny synthetic DWG files bundled with `dwg-wasm` so a browser viewer -can load a real file without the user having to supply one. Served by -the V-20 `DwgFile.openSample(name)` JS entry point (see -`wasm/src/sample_loader.rs`). +Tiny synthetic DWG files meant to let the browser viewer load a real file +without the user having to supply one (V-20). `.github/workflows/pages.yml` +copies this directory into the Pages site, so they are served from +`/samples/`. Nothing in the tree fetches them yet: the +`DwgFile.openSample(name)` entry point that V-20 describes is not implemented +(`wasm/src/no_upload.rs` notes it needs a one-line CSP relaxation when it +ships), and `docs/viewer/index.html` reads only a file the user picks. ## Provenance -Generated at build time by `tools/synthetic-dwg` (see -`tools/synthetic-dwg/src/main.rs`). That binary emits a minimal valid -R14 (AC1014) file — magic + common prefix + five 9-byte locator +Committed on 2026-04-20 (`f5bd374`), generated by `tools/synthetic-dwg`, a small +crate outside the workspace that has since been removed (#20): `examples/build_fixtures.rs` +supersedes it, and nothing in the tree builds or reads it. Its source is in +git history (`git show e7c53e1:tools/synthetic-dwg/src/main.rs`). It emitted a +minimal R14 (AC1014) file: magic + common prefix + five 9-byte locator records + a fake LINE entity payload. The file round-trips through -`dwg::DwgFile::open` but is NOT guaranteed to open in AutoCAD; these -are test fixtures, not AutoCAD-authored drawings. +`dwg::DwgFile::open` but is NOT guaranteed to open in AutoCAD; these are +test fixtures, not AutoCAD-authored drawings. -To regenerate: - -```bash -cd tools/synthetic-dwg -cargo build --release -./target/release/synthetic-dwg ../../wasm/examples/samples/line_r14.dwg --read-back -``` +The four files are committed as they were generated and nothing in the tree +regenerates them. The reader's real fixtures are `tests/fixtures/canonical/` +(written by `examples/build_fixtures.rs`, R2000 to R2018) and the public +corpus pinned in `tools/corpus/pins.tsv`. ## Files @@ -31,15 +33,9 @@ cargo build --release | `square_r14.dwg` | R14 (AC1014) | 107 B | Four-LINE square outline | | `triangle_r14.dwg` | R14 (AC1014) | 107 B | Three-LINE triangle | -NOTE: the current `synthetic-dwg` generator emits the same skeleton -file for each invocation — the name-to-content mapping above is the -**intended** future shape once the generator grows per-shape entity -output. All four files are byte-identical today. The four names still -exercise the sample-loader's `fetch` path + URL routing, which is the -purpose of V-20. - -## License - -These fixtures are generated output — not copied from Autodesk or any -other vendor's drawings. They ship under the same Apache-2.0 license -as the rest of `dwg-rs`. +NOTE: the generator emitted the same skeleton file for each invocation, so +the name-to-content mapping above is the shape the names were meant to have, +not what the files hold. All four files are byte-identical (SHA-256 +`1a6e92f734f0a747b434cc617953e907ca35b2c394962f5149d3ad9e5358c342`). The four +names are there for the loader's `fetch` path and URL routing, once V-20's +loader exists.