Repository navigation
feat(parser): wasm parser-module loader, hardening, and authoring target - #173
facontidavide wants to merge 2 commits into
Conversation
8dfaa32 to
6bc55f6
Compare
ad01f03 to
babd073
Compare
6bc55f6 to
d02344d
Compare
0279ed5 to
21f11b7
Compare
babd073 to
4e42f8b
Compare
21f11b7 to
93931f0
Compare
4e42f8b to
08503f7
Compare
93931f0 to
f977260
Compare
f977260 to
878cb93
Compare
|
Postponed — the core PR (#172) merges first on its own as This was the explicit rationale for splitting the SDK work in two: 1b is a second loader for an ABI that #172 already freezes, so deferring wasm delays wasm, never the architecture. Nothing here blocks #172, and #172 ships a complete native module story on its own. State of this branch while it waits:
🤖 Generated with Claude Code |
08503f7 to
3501581
Compare
|
Note for revival: When picking this back up, after #172 merges: rebase onto Deliberately not rebased now to avoid resolving those conflicts twice. 🤖 Generated with Claude Code |
3501581 to
817130e
Compare
878cb93 to
8a315d4
Compare
…le caps, in-tree executor Review rework of the wasm parser-module loader (PR #173). No ABI or protocol change; the installed host C++ API of 0.22/0.23 is unchanged except for additions, and NativeParserModule::load(path, sink) behaves exactly as in 0.22. - The wasm instance wrapper now has the native wrapper's fault contract: it classifies traps, metering exhaustion, malformed descriptors, and bad splices as contract violations and returns them. Strike recording, quarantine, create/bind replay, and disabling are host policy through the shared ParserModuleStrikeTracker, exactly as for native modules. This deletes the per-instance self-recreation (which was claim-wide in state but per-instance in replay, and left sibling instances on poisoned stores) and the loader-owned tracker every instance of a module raced on. - ParserModuleStrikeTracker is thread-safe (scalar and object routes of one claim run on different threads) and a contract violation while quarantined — the replay itself faulting — disables the claim instead of leaving it quarantined forever. - ParserModuleSessionBudgetTracker counts resources under opaque reservation ids instead of manifest ids: two loads of one artifact, or native and wasm builds of one source, are two reservations; duplicate-provider policy stays in the catalog. It is thread-safe because wrappers release reservations from destructors on arbitrary threads. The process-global default budget is gone: both loaders take an optional budget and do no accounting without one. - WasmParserModule::load has one signature (path + WasmParserModuleLoadOptions) instead of four overloads. - pj_base validateParserModuleWasmArtifact() is the single static admission audit used by the loader and pj-wasm-embed-manifest. It now parses the table section: every table must declare a maximum and the aggregate is capped (default 65536 elements) — table storage is host memory outside the linear memory budget and is allocated at instantiation, before any metering. - Compilation prefers Wasmer's Singlepass backend (linear compile time on untrusted input; no cancellable compile API exists). Docs now say metering is an instruction budget, not a wall-clock deadline. - The executor is an in-tree component: pj_plugin_host links it through BUILD_INTERFACE only, it is not installed or exported, and the never-taken PJ_SDK_WITH_WASMER block in plotjuggler_sdkConfig.cmake.in is removed. Installed packages stay wasmer-free and ship the authoring preset and tool. - Tests: token-zero creation error now reaches the guest (three-claim manifest on a two-claim module); table-cap loader rejections; host-driven quarantine loop; concurrent instances under one tracker; concurrent budget accounting; tracker disables on replay fault. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017U391nLF7Motf4FehVRiXz
7e3b64d to
8873b0d
Compare
…le caps, in-tree executor Review rework of the wasm parser-module loader (PR #173). No ABI or protocol change; the installed host C++ API of 0.22/0.23 is unchanged except for additions, and NativeParserModule::load(path, sink) behaves exactly as in 0.22. - The wasm instance wrapper now has the native wrapper's fault contract: it classifies traps, metering exhaustion, malformed descriptors, and bad splices as contract violations and returns them. Strike recording, quarantine, create/bind replay, and disabling are host policy through the shared ParserModuleStrikeTracker, exactly as for native modules. This deletes the per-instance self-recreation (which was claim-wide in state but per-instance in replay, and left sibling instances on poisoned stores) and the loader-owned tracker every instance of a module raced on. - ParserModuleStrikeTracker is thread-safe (scalar and object routes of one claim run on different threads) and a contract violation while quarantined — the replay itself faulting — disables the claim instead of leaving it quarantined forever. - ParserModuleSessionBudgetTracker counts resources under opaque reservation ids instead of manifest ids: two loads of one artifact, or native and wasm builds of one source, are two reservations; duplicate-provider policy stays in the catalog. It is thread-safe because wrappers release reservations from destructors on arbitrary threads. The process-global default budget is gone: both loaders take an optional budget and do no accounting without one. - WasmParserModule::load has one signature (path + WasmParserModuleLoadOptions) instead of four overloads. - pj_base validateParserModuleWasmArtifact() is the single static admission audit used by the loader and pj-wasm-embed-manifest. It now parses the table section: every table must declare a maximum and the aggregate is capped (default 65536 elements) — table storage is host memory outside the linear memory budget and is allocated at instantiation, before any metering. - Compilation prefers Wasmer's Singlepass backend (linear compile time on untrusted input; no cancellable compile API exists). Docs now say metering is an instruction budget, not a wall-clock deadline. - The executor is an in-tree component: pj_plugin_host links it through BUILD_INTERFACE only, it is not installed or exported, and the never-taken PJ_SDK_WITH_WASMER block in plotjuggler_sdkConfig.cmake.in is removed. Installed packages stay wasmer-free and ship the authoring preset and tool. - Tests: token-zero creation error now reaches the guest (three-claim manifest on a two-claim module); table-cap loader rejections; host-driven quarantine loop; concurrent instances under one tracker; concurrent budget accounting; tracker disables on replay fault. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017U391nLF7Motf4FehVRiXz
|
@copilot resolve the merge conflicts in this pull request |
Merged
Full Conan build isn't runnable in this sandbox (no network access to conancenter), so I verified with |
… for experimental parser modules Integrates PR #173 (originally 75ca1a0 "wasm parser-module loader, hardening, and authoring target" and 8873b0d "host-driven fault policy, resource-keyed budgets, table caps, in-tree executor") on top of main after the parser-module kit moved to experimental/parser_module/ (#230). - WasmParserModule / WasmParserModuleInstance execute WASI reactors through a statically linked, pinned Wasmer 7.0.1 C API. Admission runs one static audit (validateParserModuleWasmArtifact, shared with the pj-wasm-embed-manifest tool) before compilation: exactly one manifest section, an empty import allow-list, the frozen export ABI, _initialize and no start function, exported memory and tables with bounded declared maxima (table cap 65536 elements). Singlepass is preferred when available. - Execution is store-per-instance with per-call instruction metering (an instruction budget, not a wall-clock deadline). Linear memory is reacquired and every returned range revalidated after each guest call; splices resolve against the original host payload. - The wasm wrapper has the native wrapper's fault contract: traps, metering exhaustion, malformed descriptors and bad splices are returned contract violations; strikes, quarantine, create/bind replay and disabling stay host policy through ParserModuleStrikeTracker, which is now thread-safe and disables a claim whose replay faults. - ParserModuleSessionBudgetTracker (optional, thread-safe) bounds modules, artifact size, claims, active instances and aggregate declared memory under opaque reservation ids; both loaders accept a null budget and then do no accounting. - pj_add_parser_module(... TARGETS native wasm) builds both artifacts from one source with wasi-sdk 27, embeds the manifest and audits post-link; the toy module and adversarial guests (trap, infinite loop, memory growth, table cap, quarantine replay) dogfood it. The parser_module_wasm_audit test tool is replaced by pj-wasm-embed-manifest. - CI installs Wasmer 7.0.1 and runs ctest -L wasmer. The feature lives entirely under experimental/parser_module/: targets pj_wasm_parser_module_host, pj_wasm_embed_manifest and the wasm tests are defined in its CMakeLists.txt (Wasmer gated on PJ_WASMER_ROOT, wasi-sdk on PJ_WASI_SDK_ROOT). Nothing is installed or packaged, so the SDK version, changelog, feature floors and Conan recipe are unchanged and installed packages are Wasmer-free. Documentation is in experimental/parser_module/README.md ("WebAssembly loader") and AUTHORING.md. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
25bca78 to
08fb398
Compare
…sted module metadata A Codex review found that the wasm loader bounds what the guest may use but not what the host allocates or computes while reading a module's metadata. - Scalar output descriptors: field names may alias, so 4096 fields pointing at one 1 MiB string passed every range check and then cost ~4 GiB of copies. The shared decoder now caps the field count (2^20) and the cumulative bytes of names and string values (64 MiB) before any owned copy, on the native and wasm paths. A bad_alloc in wasm parse() is now a classified data error instead of escaping the wrapper. - Static inspection copied the resolved signature into every export. Exports and imports now hold a type index and resolve through the type table. - Duplicate export names were found with a quadratic scan; it is now a hash set, and type, import and export counts are capped at the WebAssembly JS-API limits. - Manifest decoding built claims (each copying the module id) before any bound. It now bounds the JSON size (4 MiB), every string (1024 bytes), the claim count (4096, the session budget's cap) and digests per claim (64) first. load() contains allocation failure. - Session budget: two instances of one module shared the module's reservation id, so releasing one twice freed the other's charge. Each instance now gets its own id, consumed once. - The wasm build did not depend on headers outside the kit glob (time_math.hpp), so wasm artifacts could go stale. It now uses a compiler-generated depfile. Each code fix has a regression test, seen failing before the fix. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
SDK PR 1b of the parser-extensibility v4 architecture (spec, PlotJuggler/pj-official-plugins#272) — follows #172 (
0.22.0, merged). A second loader for the already-frozen ABI: wasmer execution, hardening, budgets, and the wasm authoring target. Ships as 0.24.0 (0.23.0went to the symbol-provenance fix, #175).Rebased onto
main(2026-08-30) after #174–#176: the budget-awareNativeParserModule::loadnow keeps #175's defining-object symbol provenance (recorded_path), the CHANGELOG gained a0.24.0entry above0.23.0, and rootCLAUDE.md/ the plugin skill'sparser-module.md/ kit README /ARCHITECTURE.mdno longer describe wasm as unavailable. 80/80 Debug+ASAN with wasi-sdk 27 + wasmer 7.0.1.Review rework (2026-08-30, commit
7e3b64d) — after a simplicity-first review (Claude + Codex, findings cross-verified):ParserModuleStrikeTracker. Removes the per-instance self-recreation (claim-wide state, per-instance replay) and the loader-owned tracker that concurrent instances raced on.ParserModuleStrikeTrackerandParserModuleSessionBudgetTrackerare thread-safe; a fault during replay now disables instead of leaving a claim quarantined forever.NativeParserModule::load(path, sink)behaves exactly as in 0.22.WasmParserModule::load(path, options)— one signature instead of four overloads.pj_baseaudit shared withpj-wasm-embed-manifest.BUILD_INTERFACE, not installed); the never-takenPJ_SDK_WITH_WASMERconfig block is gone. Installed/Conan packages stay wasmer-free and ship the authoring preset + tool.The pre-existing Conan
plugin_hostcomponent gap (missingpj_parser_module_host/pj_parser_claim_catalogsince 0.22) is fixed separately onmain.Exit-criterion prototype findings (spec §13 gate)
libwasmer.a(7.0.1, static) declares but does not exportwasm_module_share/obtain— the loader uses engine-owned module reuse: one compilation, store-per-bound-instance, isolated state verified.wasmer_metering_*C API but no interrupt/epoch/deadline surface — execution limits are enforceable instruction metering (fresh allowance per guest call), not a wall-clock watchdog that couldn't fire.Surface
_initialize; start/_startrejected), export names and signatures audited via the sharedpj_basewasm inspector, empty frozen import allow-list (a parser module may import nothing)pj-wasm-embed-manifestinstalled CLI (embed/verify);pj_add_parser_module(... TARGETS native wasm)builds both artifacts from one source with a post-link audit — dogfooded on the toy moduleplugin_hostcomponent;plugin_sdkconsumers stay wasmer-free.PJ_WASMER_ROOTgate with graceful skip; CI installs wasmer 7.0.1 and runs the wasmer-labeled testsVerification
🤖 Generated with Claude Code