Skip to content

refactor(parser-module)!: move the functional parser-module kit to experimental/ (1.0.0) - #230

Merged
facontidavide merged 1 commit into
mainfrom
refactor/experimental-parser-module
Nov 13, 2026
Merged

facontidavide merged 1 commit into
mainfrom
refactor/experimental-parser-module

Conversation

@facontidavide

@facontidavide facontidavide commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

What

Moves the functional parser-module kit out of the installed SDK and into experimental/parser_module/. It is still built and tested in this repository, but it is no longer installed, no longer packaged, and no longer covered by the compatibility promise.

Why

1.0.0 would otherwise freeze an export ABI, a manifest schema and about 5,000 lines of authoring headers that nothing uses: no parser module exists, and no host loads one. The pre-1.0 review found three things in module ABI 1 that would need an ABI 2 (no timestamp on the object output, the payload copied inline per message, no contract-violation result code). Keeping the kit outside the promise lets those be fixed without a MAJOR.

Why experimental/ here, and not another repository

  • The native loader uses the plugin loaders' private pj_plugins/src/detail/library_loader.hpp, so the host half cannot leave without exporting or duplicating it.
  • The export ABI is a host-to-module contract. The host would be PJ4 and the authors would be plugin writers; this repository is the only place both can depend on.
  • ObjectWriter is coupled to the builtin codecs and splice table and should keep being tested against them in the same commit.

If the kit should leave the repository entirely, the follow-up is git rm -r experimental/parser_module.

Interaction with #173

Draft PR #173 (feat/parser-extensibility-1b, the wasm parser-module loader) builds on this kit and touches the same files. Whichever lands second needs a rebase across the path moves. The two agree in direction: #173 already keeps its wasm executor in-tree and out of the installed package, and with this PR the wasm loader would land under experimental/parser_module/ beside the native one.

Removed from the installed SDK (breaking)

  • Component plotjuggler_sdk::parser_module and pj_add_parser_module() (PjParserModule.cmake)
  • pj_base/parser_module/*.hpp, pj_base/parser_module_abi.h, pj_base/parser_module_manifest.hpp
  • pj_plugins/host/native_parser_module.hpp, pj_plugins/host/parser_module_runtime.hpp
  • The pj_parser_module_host archive that plotjuggler_sdk::plugin_host linked; the PJ::parser_module codecs leave libpj_base.a
  • ParserClaimCatalog::ingestModuleManifest, decodeParserModuleManifest, ParserModuleManifest
  • PJ_MODULE_PARSE_INPUT_FLAG_HAS_TIMESTAMP from the feature_floors.json baseline

No consumer references any of these: I grepped PlotJuggler, pj-official-plugins, plotjuggler-ros-plugins, PJ-plugins, mosaico, plotjuggler-agent and WASM, including their worktrees.

Unchanged

ParserClaimCatalog claim admission, synthesizeParserPluginClaims and ParserRouteResolver, which PJ4's ParserRoutingService uses.

Notes for review

  • Almost every file is a pure git mv. Headers keep their pj_base/... and pj_plugins/host/... include spellings, so promoting the kit back later is a move, not a rewrite.
  • The manifest decoder used the catalog's private claim validator. In experimental/ it validates by admitting into a scratch ParserClaimCatalog and reading the claims back, so the stable catalog gains no new public API. ingestParserModuleManifest() is now a free function that takes the diagnostic sink as an argument.
  • The module ABI static_assert pins moved from abi_layout_sentinels_test.cpp into the experimental ABI test.
  • New option PJ_BUILD_EXPERIMENTAL, default follows PJ_BUILD_TESTS. Conan exports experimental/* so with_tests=True still configures; nothing from it is packaged.
  • test_sdk_install.sh now fails if any parser-module file reaches the install tree.
  • The plugin-authoring skill no longer offers parser modules as an option; its reference moved to experimental/parser_module/AUTHORING.md.

Verification

  • ./build.sh --debug (Debug + ASAN): builds
  • ./test.sh: 108 of 108 pass, including the 8 parser-module tests from their new location
  • pre-commit (clang-format 22): clean
  • tools/feature_floors/check_feature_floors.py: OK, 140 surfaces
  • ./test_sdk_install.sh (Release): passes, including the new leak check
  • conan create (Release, with_tests=False) plus a throwaway consumer that links plotjuggler_sdk::plugin_host and asserts the parser_module target and pj_add_parser_module are absent: passes

Not run locally: the abidiff gate (libabigail is not installed here) and the WASI conformance job (no wasi-sdk here). Both run in CI; the WASI target name is unchanged.

🤖 Generated with Claude Code

…perimental/ (1.0.0)

The kit (authoring headers, export ABI and block codecs, native loader and
runtime, pj_add_parser_module) has no module and no host using it, so 1.0
would freeze an ABI nobody has exercised. It moves to
experimental/parser_module/: still built and tested here, never installed or
packaged, and outside the compatibility promise.

It stays in this repository because the native loader depends on the plugin
loaders' private detail/library_loader.hpp, the export ABI is a host-to-module
contract that needs one home, and ObjectWriter is coupled to the builtin
codecs and splice table it is tested against.

Removed from the installed SDK:
- component plotjuggler_sdk::parser_module and PjParserModule.cmake
- pj_base/parser_module/*.hpp, parser_module_abi.h, parser_module_manifest.hpp
- pj_plugins/host/native_parser_module.hpp, parser_module_runtime.hpp
- the pj_parser_module_host archive linked by plugin_host
- ParserClaimCatalog::ingestModuleManifest, decodeParserModuleManifest and
  ParserModuleManifest (now parser_module_claims.hpp in experimental, which
  validates through the public admitClaims path)
- PJ_MODULE_PARSE_INPUT_FLAG_HAS_TIMESTAMP from the feature-floor baseline

Claim admission, plugin-claim synthesis and ParserRouteResolver are unchanged.

test_sdk_install.sh now fails if any parser-module file reaches the install
tree. PJ_BUILD_EXPERIMENTAL (default: follows PJ_BUILD_TESTS) gates the build.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@facontidavide
facontidavide force-pushed the refactor/experimental-parser-module branch from c100f7e to be1d7c4 Compare November 13, 2026 23:22
@facontidavide
facontidavide merged commit f1b091c into main Nov 13, 2026
7 checks passed
@facontidavide
facontidavide deleted the refactor/experimental-parser-module branch November 13, 2026 23:37
facontidavide added a commit that referenced this pull request Nov 14, 2026
… for experimental parser modules

Integrates PR #173 (originally 75ca1a0 "wasm parser-module loader,
hardening, and authoring target" and 8873b0d "host-driven fault policy,
resource-keyed budgets, table caps, in-tree executor") on top of main after
the parser-module kit moved to experimental/parser_module/ (#230).

- WasmParserModule / WasmParserModuleInstance execute WASI reactors through a
  statically linked, pinned Wasmer 7.0.1 C API. Admission runs one static
  audit (validateParserModuleWasmArtifact, shared with the
  pj-wasm-embed-manifest tool) before compilation: exactly one manifest
  section, an empty import allow-list, the frozen export ABI, _initialize and
  no start function, exported memory and tables with bounded declared
  maxima (table cap 65536 elements). Singlepass is preferred when available.
- Execution is store-per-instance with per-call instruction metering (an
  instruction budget, not a wall-clock deadline). Linear memory is
  reacquired and every returned range revalidated after each guest call;
  splices resolve against the original host payload.
- The wasm wrapper has the native wrapper's fault contract: traps, metering
  exhaustion, malformed descriptors and bad splices are returned contract
  violations; strikes, quarantine, create/bind replay and disabling stay
  host policy through ParserModuleStrikeTracker, which is now thread-safe and
  disables a claim whose replay faults.
- ParserModuleSessionBudgetTracker (optional, thread-safe) bounds modules,
  artifact size, claims, active instances and aggregate declared memory
  under opaque reservation ids; both loaders accept a null budget and then
  do no accounting.
- pj_add_parser_module(... TARGETS native wasm) builds both artifacts from
  one source with wasi-sdk 27, embeds the manifest and audits post-link;
  the toy module and adversarial guests (trap, infinite loop, memory growth,
  table cap, quarantine replay) dogfood it. The parser_module_wasm_audit test
  tool is replaced by pj-wasm-embed-manifest.
- CI installs Wasmer 7.0.1 and runs ctest -L wasmer.

The feature lives entirely under experimental/parser_module/: targets
pj_wasm_parser_module_host, pj_wasm_embed_manifest and the wasm tests are
defined in its CMakeLists.txt (Wasmer gated on PJ_WASMER_ROOT, wasi-sdk on
PJ_WASI_SDK_ROOT). Nothing is installed or packaged, so the SDK version,
changelog, feature floors and Conan recipe are unchanged and installed
packages are Wasmer-free. Documentation is in
experimental/parser_module/README.md ("WebAssembly loader") and AUTHORING.md.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant