Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
Expand Up @@ -12,3 +12,7 @@ tools/extension-qualification/Cargo.toml text eol=lf
tools/extension-qualification/*.rs text eol=lf
tools/extension-qualification/src/*.rs text eol=lf
tools/extension-qualification/src/bin/*.rs text eol=lf

# Golden model outputs are digested bytes; never convert their line endings.
rust/crates/morphir-python-binding/tests/golden/* -text
rust/crates/morphir-python-binding/tests/fixtures/** text eol=lf
4 changes: 4 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,11 @@ repos:
rev: v5.0.0
hooks:
- id: trailing-whitespace
# Golden payloads are exact digested bytes with no final newline.
exclude: ^rust/crates/morphir-python-binding/tests/golden/
- id: end-of-file-fixer
# Golden payloads are exact digested bytes with no final newline.
exclude: ^rust/crates/morphir-python-binding/tests/golden/
- id: check-yaml
- id: check-toml
- id: check-added-large-files
Expand Down
2 changes: 2 additions & 0 deletions openspec/changes/qualify-mep-model-line/.openspec.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
schema: html-design
created: 2026-10-31
83 changes: 83 additions & 0 deletions openspec/changes/qualify-mep-model-line/design.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>qualify-mep-model-line: design</title>
<!--
Agent document template (docs/templates/agent-document.html). Specs, plans and reports that a person reviews
start from this file: copy it, keep the <head> as it is, and write the content inside <main>.
Styles and scripts are files in docs/templates/assets/, referenced and not embedded. In a copy, change the
"assets/" prefix of each href and src to the relative path from the copy to that directory, for example
"../../docs/templates/assets/" from .dev/beads-specs/. See AGENTS.md, "Specs and plans are HTML documents".
Code goes in <pre><code class="language-python"> (or json, toml, bash, diff). Give sections an id and
list them in nav.toc.
-->
<link rel="stylesheet" href="../../../docs/templates/assets/agent-document.css">
<link rel="stylesheet" href="../../../docs/templates/assets/highlight/styles/github.min.css" media="(prefers-color-scheme: light)">
<link rel="stylesheet" href="../../../docs/templates/assets/highlight/styles/github-dark.min.css" media="(prefers-color-scheme: dark)">
<script defer src="../../../docs/templates/assets/highlight/highlight.min.js"></script>
<script defer src="../../../docs/templates/assets/agent-document.js"></script>
</head>
<body>
<main>
<h1>qualify-mep-model-line: design</h1>
<p class="meta">OpenSpec change · Bead <code>morphir-ukvn.1.6.9</code> · 2026-10-31 · in progress</p>

<nav class="toc" aria-label="Contents">
<ul>
<li><a href="#context">Context</a></li>
<li><a href="#goals">Goals and non-goals</a></li>
<li><a href="#decisions">Decisions</a></li>
<li><a href="#risks">Risks and trade-offs</a></li>
<li><a href="#migration">Migration plan</a></li>
</ul>
</nav>

<h2 id="context">Context</h2>
<p>See proposal.md, "Why". The model line is in <code>model_transport.rs</code> on <code>morphir-extension-sdk</code> <code>0.4.0</code>. The SDK selects the intersection of the host offer and the guest support in the host's order, and the guest admits each call against the session tuples. These facts were true in <code>extension/python/v0.5.0</code>:</p>
<table>
<thead><tr><th>Item</th><th>0.5.0</th></tr></thead>
<tbody>
<tr><td>Protocol versions</td><td><code>0.2.0-draft.1</code>, <code>0.1</code></td></tr>
<tr><td>Claims version</td><td><code>0.1.0-draft.2</code></td></tr>
<tr><td>Model tuples</td><td>compileModel and generateModel × <code>morphir-ir</code> / <code>morphir-json</code> / <code>0.1.0-draft.1</code> / IR <code>3.0.0</code> and <code>4.0.0</code></td></tr>
<tr><td>Ion</td><td>Not offered. Ion evidence is refused on WASM and native alike.</td></tr>
</tbody>
</table>

<h2 id="goals">Goals and non-goals</h2>
<ul>
<li>Goal: tests that fail if negotiation, admission or the model outputs change.</li>
<li>Goal: evidence that the MoonBit worker host runs the model line from both CLIs.</li>
<li>Non-goal: Ion, dependencies in <code>compileModel</code>, or any change to finos/morphir-moonbit or finos/morphir.</li>
</ul>

<h2 id="decisions">Decisions</h2>

<h3 id="tst-1">TST-1: golden files hold the exact payload bytes</h3>
<p>The envelope golden file replaces <code>payload.content</code> with a placeholder, and a second file holds the decoded payload bytes as the guest wrote them. The SDK decodes the payload only after it checks length and SHA-256, so the payload file's digest equals the envelope digest. Pretty-printed IR 3 was 215 KB against 18 KB of exact bytes, and it would not catch a change in serialization. The files are <code>-text</code> in <code>.gitattributes</code> and excluded from the whitespace hooks.</p>
<p>Set <code>MORPHIR_UPDATE_GOLDEN=1</code> to write them again after an intended change.</p>

<h3 id="tst-2">TST-2: run golden outputs through a negotiated session</h3>
<p>Calling <code>compile_model</code> directly skips the SDK's session admission. The golden test sends JSON-RPC messages to a session that a host initialized with all four tuples, so it also covers selection, admission and serialization of the result.</p>

<h3 id="qual-1">QUAL-1: qualify without changing finos/morphir-moonbit</h3>
<p>The host's Python suite pins <code>0.4.1</code> and <code>0.4.2</code> and checks MEP <code>0.1</code> only. A separate script outside both repositories drives the CLIs built from a clean <code>main</code> checkout with the same contract <code>0.2.0</code> WASM pin shape, the same environments (no Node on the native CLI's PATH) and the same checks as the host's Elm model-line suite. The receipt records the CLI program digests and the commit, which prove that the host is unchanged.</p>

<h3 id="rel-1">REL-1: release 0.6.0 as a minor version</h3>
<p>The protocol surface is the same as in <code>0.5.0</code>, but the compiler now builds with Ruff <code>0.0.20</code> and current dependencies. That can change parsing or generated text, so it is a new minor version, not a patch. The extension tag builds its native wheel from source in the same run, so it needs no native or CLI release.</p>

<h2 id="risks">Risks and trade-offs</h2>
<ul>
<li>[A Ruff update changes generated text] → the golden test fails and shows the difference; update the files only after review.</li>
<li>[The published CLI <code>morphir-python</code> <code>0.1.2</code> keeps compiler <code>0.5.0</code>] → the documentation says which compiler each package pairs with; a native and CLI release is a separate decision.</li>
<li>[The MoonBit host offers model tuples for IR <code>4.0.0</code> only, and refuses <code>--ir-version 3</code> for any guest that supports the model line (<code>apps/morphir/cli/extension.mbt</code>)] → the qualification records that refusal as an expected row; IR 3 model operations are covered by the Rust session and golden tests. With <code>0.4.2</code>, which declares only MEP <code>0.1</code>, the host compiled IR 3. This is a host change for finos/morphir-moonbit, tracked in its own bead.</li>
<li>[On the npm CLI a WASM worker runs on a Node worker thread] → its documented stop method is <code>thread-termination</code>; the qualification still requires an acknowledged shutdown and exit code 0. The process executable and the native CLI's WASM worker stop by themselves (<code>exited-after-exit</code>).</li>
</ul>

<h2 id="migration">Migration plan</h2>
<p>No migration. Hosts on MEP <code>0.1</code> keep working: the finos/morphir Rust host pins <code>extension/python/v0.4.2</code>, and the <code>0.1</code> suite repeats for <code>0.4.1</code> and <code>0.4.2</code>. To roll back, hosts keep or restore their <code>0.5.0</code> pin; released bytes do not change.</p>
</main>
</body>
</html>
32 changes: 32 additions & 0 deletions openspec/changes/qualify-mep-model-line/proposal.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# Proposal

## Why

Every Morphir extension must be on the latest MEP, `0.2.0-draft.1`. The Python extension `0.5.0` already declares it, with `0.1` kept, and serves `morphir.frontend.compileModel` and `morphir.backend.generateModel` for the tuples `morphir-ir` / `morphir-json` / envelope `0.1.0-draft.1` / IR `3.0.0` and `4.0.0`. Three gaps remain. No test shows the guest agreeing on JSON when the host's default is Ion but the host also offers JSON (the negotiation rule that finos/morphir-gleam#31 tests). The model operations have no golden outputs. The MoonBit worker host (finos/morphir-moonbit `main`) qualified Python `0.4.1` and `0.4.2` on MEP `0.1` only, not the model line.

## What Changes

- Add session tests: host that prefers Ion but also offers JSON; host that offers only Ion; compile admission that does not grant generation; wrong envelope version in each direction; the process executable serving both model operations and stopping by itself.
- Add golden outputs for `compileModel` (envelope and exact payload bytes) and `generateModel` (artifacts) for IR `3.0.0` and `4.0.0`, through a real negotiated session.
- Add the host-default scenario to `python/model-transport`.
- Qualify the candidate on the MoonBit worker host from the native and npm CLIs: `extension describe`, `compile` and `generate` on the model line with a contract `0.2.0` pin, recompile of generated sources, and worker shutdown. Repeat the existing `0.1`-only suite for `0.4.1` and `0.4.2`.
- Release `extension/python/v0.6.0` with the current dependencies (Ruff `0.0.20`). No protocol, claim or tuple changes. No Ion.

## Capabilities

### New Capabilities

None.

### Modified Capabilities

- `python/model-transport`: the admission requirement gains a scenario for a host whose default format the guest does not serve.

## Beads

- **Tracked by**: `morphir-ukvn.1.6.9`
- **Related**: `morphir-5hxy` (delivered the model line in `0.5.0`); `morphir-1q2c` (MEP spec: model-format default and required rule); `morphir-hv6m` (MCK check of model-format selection order); `morphir-ukvn.5.64.36.7` (MoonBit worker host qualification of published extensions).

## Impact

Tests and golden files in `rust/crates/morphir-python-binding` and `rust/crates/morphir-python-extension`, the compiler crate version (`0.6.0`), `.gitattributes` and pre-commit excludes for byte-exact golden files, and the extension documentation after release. finos/morphir keeps its `extension/python/v0.4.2` pin; its bundle pins move in a separate PR. Merge and tag need maintainer approval.
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
## MODIFIED Requirements

### Requirement: Model operation admission is bilateral
Each model operation SHALL require an initialized, advertised tuple matching operation, profile, codec, envelope version and semantic IR version. Session tuples and limits MUST remain within both peers' offers and permitted static claims. Unknown required guarantees and unadmitted tuples MUST be refused. (bead: morphir-5hxy)

#### Scenario: Compile support does not grant generation
- **WHEN** only a compile tuple was admitted and the host requests model generation
- **THEN** the operation is refused without invoking the generator

#### Scenario: Independent lists would produce an unsupported combination
- **WHEN** an operation combines otherwise known tuple members in a combination not admitted as a complete tuple
- **THEN** the extension refuses that combination rather than forming a Cartesian product

#### Scenario: Host default not supported
- **WHEN** the host offers an Ion tuple first, as its default, and also offers a JSON tuple this extension serves, without requiring Ion
- **THEN** the session agrees on the JSON tuple in the host's order, and the model operations for that tuple succeed

#### Scenario: Host offers only formats the extension does not serve
- **WHEN** the host offers only Ion tuples
- **THEN** the session agrees on no model tuple, model operations are refused, no payload is converted to JSON, and the MEP `0.1` operations stay available
31 changes: 31 additions & 0 deletions openspec/changes/qualify-mep-model-line/tasks.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# Tasks

Qualification receipts for 3.1 and 3.2 (finos/morphir-moonbit `main` 40623b8, macOS arm64) are on bead `morphir-ukvn.1.6.9.3`: the `0.1` suite passed 32 of 32 rows, and the candidate model line passed 32 of 32 rows for the WASM and process guests from both CLIs. The host refuses IR 3 on the model line; see design.html, Risks.

## 1. Test negotiation, golden outputs and refusals (morphir-ukvn.1.6.9.1)

- [x] 1.1 Add session tests for an Ion-preferring host that also offers JSON, an Ion-only host, compile-only admission and a wrong envelope version; verify `cargo test -p morphir-python-binding --test model_session` passes.
- [x] 1.2 Add golden outputs for `compileModel` and `generateModel` for IR `3.0.0` and `4.0.0` through a negotiated session, with byte-exact payload files kept out of line-ending conversion and end-of-file hooks; verify `cargo test -p morphir-python-binding --test model_golden` passes and each payload file's SHA-256 equals its envelope digest.
- [x] 1.3 Add a process test that runs both model operations in a `0.2.0-draft.1` session and checks that the process stops by itself; verify `cargo test -p morphir-python-extension` passes.

## 2. Specify and document (morphir-ukvn.1.6.9.2)

- [x] 2.1 Add the host-default and Ion-only scenarios to `python/model-transport`; verify `openspec validate qualify-mep-model-line --strict` passes.
- [x] 2.2 Set the compiler crate version to `0.6.0`; verify `mise run rust:check` and `mise run check` pass.

## 3. Qualify on the MoonBit worker host (morphir-ukvn.1.6.9.3)

- [x] 3.1 Repeat the `0.1`-only Python suite of finos/morphir-moonbit `main` (`mise run test:cli-extism-python`) for `0.4.1` and `0.4.2`; verify every row passes from the native and npm CLIs.
- [x] 3.2 Run `extension describe`, model-line `compile` (IR 4), `generate`, recompile and a consumer check of the generated Python for the candidate WASM with a contract `0.2.0` pin from the native and npm CLIs; verify protocol `0.2.0-draft.1`, the JSON tuple, equal outputs across CLIs and `exited-after-exit` worker shutdown, and record the receipt on the epic bead.

## 4. Release (morphir-ukvn.1.6.9.4)

- [ ] 4.1 Open the PR and get CI green; check and handle each review finding.
- [ ] 4.2 Run the Release workflow manually on the candidate branch; verify every required job passes and no release is published.
- [ ] 4.3 After maintainer approval, merge, push `extension/python/v0.6.0` at the reviewed commit and verify the published assets, attestations, `release.json` descriptors and `.sha256` files; requalify the published WASM on the MoonBit worker host.
- [ ] 4.4 Point the extension documentation and `site/integrations.json` at `0.6.0` with its published digests; verify `mise run docs:test`.

## Workflow follow-up

- Archive the change after the release and the documentation update merge.
- Damian's main session moves the finos/morphir bundle pins in a separate PR.
2 changes: 1 addition & 1 deletion rust/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion rust/crates/morphir-python-binding/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "morphir-python-binding"
version = "0.5.0"
version = "0.6.0"
edition.workspace = true
rust-version.workspace = true
authors.workspace = true
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
{
"model": {
"envelopeVersion": "0.1.0-draft.1",
"modelId": "acme",
"revision": "sha256:01d40fe60c313fd0de6114693531c412366b0d4eb704697bdce4850158e24b72",
"irVersion": "3.0.0",
"profile": "morphir-ir",
"payload": {
"codec": "morphir-json",
"layout": "single-document",
"decodedByteLength": 18206,
"sha256": "01d40fe60c313fd0de6114693531c412366b0d4eb704697bdce4850158e24b72",
"content": "<base64 of the payload golden file>"
},
"evidence": []
},
"diagnostics": [],
"modules": [
"acme/functions",
"acme/models"
]
}

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
{
"model": {
"envelopeVersion": "0.1.0-draft.1",
"modelId": "acme",
"revision": "sha256:96647cf9745ad08392a039c2b7445380568b0ba17f5173dc2e18b586906365b0",
"irVersion": "4.0.0",
"profile": "morphir-ir",
"payload": {
"codec": "morphir-json",
"layout": "single-document",
"decodedByteLength": 5955,
"sha256": "96647cf9745ad08392a039c2b7445380568b0ba17f5173dc2e18b586906365b0",
"content": "<base64 of the payload golden file>"
},
"evidence": []
},
"diagnostics": [],
"modules": [
"acme/functions",
"acme/models"
]
}
Loading
Loading