Skip to content

feat(wasm): build both wasm FMU forms for every case, in CI - #486

Merged
mtiller merged 6 commits into
refactorfrom
wasm-artifacts-ci
Sep 7, 2026
Merged

mtiller merged 6 commits into
refactorfrom
wasm-artifacts-ci

Conversation

@mtiller

@mtiller mtiller commented Sep 7, 2026 •

Copy link
Copy Markdown
Owner

Steps 0, 1 and 1.5 of the wasm plan: every case exporting as a WebAssembly FMU, in CI, before anything consumes it. Step 2 (the web-worker client that replaces the api/ round-trip) is deliberately not here.

The astro/mystmd refactor is untouched and unblocked — that work is purely additive under web-astro/ and web/, and this is purely additive inside text/.

The native build is a separate pipeline, and that matters

make results runs dvc repro dvc.yaml, which reproduces every stage in that file. The first cut put the 196 build-wasm-* stages there, which quietly made the ordinary book build try to export wasm FMUs with whatever omc it had — and the book-builder image's omc is a release build:

Error: Invalid type of flag simCodeTarget, expected one of None, C, Cpp, ...
       but got the string "wasm-jit".
Error: Unknown option +fmuDirectory.

specs.py now emits them to text/dvc-wasm/dvc.yaml, its own pipeline with its own lock, and make wasm reproduces that file. Neither pipeline can reach the other. text/dvc.lock is consequently byte-identical to refactor.

Everything downstream of make results — plots, Sphinx, ebooks, exes.tar.gz, api/ — behaves exactly as before.

Both export forms, because they are not interchangeable

--simCodeTarget=wasm-jit emits two quite different artifacts and --fmuDirectory is the only difference. One omc run emits both at ~0.7 s per case, so which to ship stays open rather than being decided by the build.

form count laid out median runs where
wasm/dylink/<case>.fmu 98 8.3 MB 24 KB model kernel only, binaries/wasm32-om-dylink/. Needs a host that links it against a prebuilt adapter — today an OpenModelica importer. No browser host exists.
wasm/component/<case>.fmu 98 70.5 MB 712 KB portable fmi-ls-wasm component, binaries/wasm32-wasip2/. What a browser can run today.
wasm/aot/<case>/ 98 256.9 MB 2.6 MB the component transpiled ahead of time into core modules + glue + manifest.json.
wasm/pack/ 127 blobs 16.3 MB — the AOT content, content-addressed. The deployable layout.

Ahead-of-time transpile (Step 1.5)

A browser cannot instantiate a WebAssembly Component — WebAssembly.instantiate takes core modules. OpenModelica's fmi-simulator page ships jco's transpiler (~9.3 MB) and converts in the page: 1424 ms of a 1598 ms first load (MIC-178). The book's model set is known at build time, so tools/wasm/transpile.mjs does it once per case.

in-page AOT
session.load() 1598 ms 169 ms
page open → first result 1854 ms 364 ms
shared download, once ~11.5 MB 2.2 MB

Content-addressed packing

jco names core modules positionally and the numbering shifts per model, hiding how much the cases share. By content they barely differ: 186 MB of core modules is 12.8 MB of distinct bytes, and 1.85 MB of that — the OpenModelica runtime — is byte-identical in every case. fmu.js is 747 KB with four distinct variants.

Per case none of that can be shared, since each aot/<case>/fmu.core9.wasm is its own URL. tools/wasm/pack.py rewrites it:

pack/blobs/<sha256[:16]>.wasm    each distinct core module, once
pack/blobs/<sha256[:16]>.js      each distinct glue file, once
pack/cases/<case>.json           logical name -> blob path, + imports/exports

98 cases → 127 blobs, 16.3 MB — 15.7x, in 0.68 s, hardlinked so it costs no extra disk. The logical names survive because that is what the glue asks for: fmu-core.js builds cores as a Map keyed by jco's name and calls compile(name).

For a reader: ~0.75 MB gzipped for the first figure on a page, then ~16 KB per figure after — against 0.75 MB every time laid out per case. That also sharpens the open Step 0 question: per figure AOT is ~33 KB against dylink's 24 KB, and unlike dylink it has a browser that can run it.

Content-addressing the FMU forms was measured and rejected: it would save 8 KB across all of dylink/ (96 identical 85-byte artifact.json sidecars) and zero across component/.

Two defects fixed, and where the case settings went

gen_wasm.mos emitted setCommandLineOptions("+override=..."), which is not an omc option — it answers Error: Unknown option +override. and carries on — so 8 cases were silently exported with stock parameter values. There is no compile-time equivalent; -override is a runtime flag on the native path too.

Separately, buildModelFMU takes no simulation settings, so an FMU carries the model's own experiment annotation as its DefaultExperiment. For 74 of 97 cases that is not the case's stopTime.

Both belong beside the artifact. specs.py now emits results/cases.json, and make-index.py folds it into results/wasm/index.json — per case, the artifacts plus the stopTime, tolerance, intervals and overrides an importer must apply with fmi3Set* before initialization.

CI

generate_wasm runs in ghcr.io/mtiller/omc-wasm:3998b1f8, pinned to the OpenModelica commit it was built from. No published image has this compiler: --simCodeTarget=wasm-jit needs OM_OMC_ENABLE_RUST=ON, which defaults OFF, and Docker Hub's openmodelica/openmodelica carries release tags only, all pre-wasm. tools/wasm/Dockerfile adds the MSL and toolchain on top of that base and rebuilds in ~35 s; the base is the ~72 minute from-source build on MIC-178.

Two things it needs that only running it revealed:

  • clang + lld. IIEV uses external "C" (interpolateVectorTable) with no wasm implementation, so omc compiles the Include sources itself with clang --target=wasm32-wasip1. The wasi sysroot already ships in the base.
  • An MSL link. A container job runs with HOME=/github/home, where omc finds no libraries at all, so the workflow links it to the image's MSL before make env.

Gating also changed: generate_results and generate_site were behind the same Full Build label as deploy_site, so a label meaning "build this" also meant "publish to production". Now all three build jobs run on every PR and deploy_site is restricted to push, which the trigger already narrows to master — a PR cannot deploy, with or without a label. labeled comes off the trigger too, since nothing keys off labels now; the Full Build label is unused and can be deleted.

Verification

Last run, all four jobs as intended:

job result time
generate_wasm ✅ success 3m
generate_results ✅ success 12m
generate_site ✅ success 1m
deploy_site ⏭️ skipped —

98/98 cases in all three forms, CI totals matching local byte for byte. Cold 14m30s, warm 3m. Artifact 75 MB at 7-day retention (pack/ is uploaded, aot/ excluded — same content, duplicates removed).

mtiller and others added 2 commits September 7, 2026 09:44
Every one of the 98 simulation cases is now also exported as a
WebAssembly-backed FMI 3.0 FMU, in both forms `--simCodeTarget=wasm-jit`
can emit, plus an ahead-of-time transpile of the portable one. Nothing
consumes these yet: they are built so that they are known to build, and
keep building, before any client work depends on them.

The native pipeline is untouched. These are additional `build-wasm-*`
DVC stages beside `build-case-*`; `make results`, the plots, Sphinx, the
ebooks, `exes.tar.gz` and `api/` all behave exactly as before.

Both forms, because they are not interchangeable and the choice does not
have to be made at build time:

  dylink/     the model kernel alone, at binaries/wasm32-om-dylink/.
              Median 24 KB, 8.3 MB for the whole book. Runs only in a
              host that links it against a prebuilt adapter, which today
              means an OpenModelica importer -- there is no browser host
              for it.
  component/  a portable fmi-ls-wasm component at binaries/wasm32-wasip2/.
              Median 712 KB, and the form a browser can run today.

One `omc` run emits both (the only difference is --fmuDirectory), so
building both costs ~0.7 s per case.

The AOT step matters because a browser cannot instantiate a WebAssembly
Component -- OpenModelica's own fmi-simulator page therefore ships jco's
transpiler and converts in the page, which measured 1424 ms of a 1598 ms
first load. The book's model set is known at build time, so
tools/wasm/transpile.mjs does it once per case instead: load 1598 -> 169 ms,
first result 1854 -> 364 ms, shared download ~11.5 MB -> 2.2 MB (MIC-178).

Two things the export does not carry, and now does not pretend to:

  - `setCommandLineOptions("+override=...")` was not a real omc option
    (`Error: Unknown option +override.`), so 8 cases were silently built
    with stock parameter values. buildModelFMU has no compile-time
    equivalent; `-override` is a runtime flag on the native path too.
  - buildModelFMU takes no simulation settings either, so an FMU carries
    the model's own `experiment` annotation as its DefaultExperiment,
    which differs from the case's stopTime for 74 of 97 cases.

Both belong beside the artifact rather than inside it, so specs.py now
emits results/cases.json and make-index.py folds it into
results/wasm/index.json: per case, the artifacts and the stopTime,
tolerance, interval count and overrides an importer has to apply.

CI runs this in ghcr.io/mtiller/omc-wasm, pinned to the OpenModelica
commit it was built from. No published image has this compiler:
--simCodeTarget=wasm-jit needs OM_OMC_ENABLE_RUST=ON, which defaults OFF,
and Docker Hub's openmodelica/openmodelica carries release tags only, all
pre-wasm. tools/wasm/Dockerfile adds the MSL and the build toolchain on
top of that base and rebuilds in seconds; producing the base itself is
the ~72 minute from-source build documented on MIC-178.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Nothing consumes the wasm artifacts yet, so a 90-day retention would hold
~340 MB per run to no purpose. Rebuild cost is carried by the DVC cache
(98 MB, content-addressed), not by the artifact.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@mtiller mtiller added Full Build Trigger the full build+deploy workflow on this PR and removed Full Build Trigger the full build+deploy workflow on this PR labels Sep 7, 2026
mtiller and others added 4 commits September 7, 2026 11:18
`make all` in text/results would otherwise require an omc with the
wasm-jit backend and a node with the jco bundle, which the ordinary book
toolchain has neither of. The DVC path and CI both name the wasm stages
explicitly, so nothing else needs it there.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… once

jco names its core modules positionally (fmu.core.wasm, fmu.core2.wasm,
...) and the numbering shifts from model to model, which hid how much the
98 cases have in common. By content they barely differ: 186 MB of core
modules is 12.8 MB of distinct bytes, and 1.85 MB of that -- the
OpenModelica runtime -- is byte-identical in every single case. The glue
is nearly shared too: fmu.js is 747 KB with only four distinct variants.

Laid out per case none of that could be shared, because each
aot/<case>/fmu.core9.wasm is its own URL: a reader opening a second
figure would download the runtime again. pack.py rewrites the tree so
every distinct blob has one content-addressed path:

  pack/blobs/<sha256[:16]>.wasm   each distinct core module, once
  pack/blobs/<sha256[:16]>.js     each distinct glue file, once
  pack/cases/<case>.json          logical name -> blob path, + imports/exports

98 cases become 127 blobs, 16.3 MB -- 15.7x smaller -- in under a second.
Blobs are hardlinked from the DVC outputs, so the packed tree costs no
extra space, and it is built after `dvc repro` rather than as a stage,
because a shared blob store cannot be the output of 98 separate stages.

The logical names survive the rename because that is what the glue asks
for: fmu-core.js builds `cores` as a Map keyed by jco's name and the
generated module calls compile(name) against it. The client may fetch
those bytes from any URL; it just needs the mapping.

For a reader, assuming the blobs are served at those shared paths: about
0.75 MB gzipped for the first figure on a page, then ~16 KB for each
figure after it, against 0.75 MB every time laid out per case. That also
changes the Step 0 comparison the PR leaves open -- per figure the AOT
form is ~33 KB against dylink's 24 KB, and unlike dylink it has a browser
that can run it.

CI now uploads pack/ and skips aot/, which is the same content with the
duplicates removed rather than the same content twice.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
generate_results and generate_site were gated behind the same 'Full Build'
label as deploy_site, so an ordinary pull request built nothing and a
label meant to say "build this" also meant "publish this to production".
Those are different questions and only one of them belongs on a PR.

Now the three build jobs run everywhere and deploy_site is restricted to
push events, which the workflow's own trigger already narrows to master.
A pull request therefore cannot deploy, with or without a label.

'labeled' comes off the pull_request trigger with it: nothing keys off
labels any more, so leaving it in would re-run a ~17 minute build every
time any label was added or removed. The 'Full Build' label itself is now
unused and can be deleted from the repo.

Measured on the last two master runs: generate_results 16m,
generate_site 1m, both green, and generate_wasm runs beside them in
7-14m. Minutes are free on a public repository.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`make results` runs `dvc repro dvc.yaml`, which reproduces *every* stage
in that file -- so adding the 196 build-wasm-* stages to text/dvc.yaml
quietly made the ordinary book build try to export wasm FMUs. The
book-builder image's omc is a release build with no wasm backend, so
generate_results failed:

    Error: Invalid type of flag simCodeTarget, expected one of None, C,
    Cpp, ... but got the string "wasm-jit".
    Error: Unknown option +fmuDirectory.
    Error: Unknown FMU version 3.0. Only version 1.0 & 2.0 are supported.

Diffing the generated stage definitions had shown them byte-identical to
refactor's, which was true and beside the point: what changed was not any
native stage but the set of stages `dvc repro dvc.yaml` reaches. Running
the job is what caught it.

specs.py now emits them to text/dvc-wasm/dvc.yaml instead, and
`make wasm` reproduces that file. Neither pipeline can reach the other.

Two things fall out of the split. text/dvc.lock goes back to exactly what
is on refactor, since the wasm stages now keep their own lock -- which
also drops the ~150 lines of YAML re-wrapping noise the shared lock had
picked up from a newer dvc. And `make wasm` got faster, 3m01 to 1m38 on a
warm cache, because dvc no longer walks 99 native stages to decide it has
nothing to do with them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@mtiller
mtiller merged commit c91851e into refactor Sep 7, 2026
4 checks passed
@mtiller
mtiller deleted the wasm-artifacts-ci branch September 7, 2026 20:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant