feat(ir): read and write IR 3.1.0 with a v3 Specs distribution

.gitignore

@@ -70,3 +70,6 @@ yarn-error.log*
 
 # Beads runtime lock at repo root (bd creates this; do not commit)
 .beads.gate.lock
+
+# behave writes the pretty formatter to this file when its output is piped
+pretty.output

AGENTS.md

@@ -23,7 +23,8 @@ morphir-python/
 
 **`morphir` (core library)**
 - Morphir IR (Intermediate Representation) models, in two versions:
-  `morphir.ir.v3` (format 3.0.0; morphir-elm is normative) and `morphir.ir.v4`
+  `morphir.ir.v3` (formats 3.0.0 and 3.1.0; morphir-elm is normative for 3.0.0,
+  finos/morphir for the 3.1.0 `Specs` distribution) and `morphir.ir.v4`
   (draft upstream; `morphir.ir` re-exports it). Read `formatVersion` only through
   `morphir.ir.format_version`, convert only through `morphir.ir.migrate`, and keep
   diagnostic codes in `morphir.ir.diagnostics`. See

features/ir_interchange.feature

@@ -1,7 +1,7 @@
 Feature: IR interchange
   As a Morphir tooling user
   I want to inspect and convert versioned morphir-ir.json files
-  So that I can move models between IR format 3.0.0 and draft IR v4
+  So that I can move models between IR format 3 (3.0.x and 3.1.x) and draft IR v4
 
   Scenario: Inspect a morphir-elm format 3 file
     Given the golden fixture "elm-simpleValueTree.json"
@@ -28,8 +28,41 @@ Feature: IR interchange
     Then the command exits with code 1
     And the errors contain "missing_format_version"
 
-  Scenario: Specs cannot be downgraded
-    Given a file with the JSON '{"formatVersion": 4, "distribution": {"Specs": {"packageName": "a"}}}'
+  Scenario: Inspect a v3 Specs distribution of IR 3.1.0
+    Given the golden fixture "mck-distributions-0011-specs.json"
+    When I run "ir inspect" on it
+    Then the command exits with code 0
+    And the output contains "format: 3.1.0"
+    And the output contains "kind: Specs"
+    And the output contains "package: my/pkg"
+
+  Scenario: Convert a v3 Specs distribution to draft v4 and back
+    Given the golden fixture "rust-classic-specs-transport.json"
+    When I convert it to "v4"
+    Then the command exits with code 0
+    And the converted file declares formatVersion 4
+    When I inspect the converted file
+    Then the output contains "kind: Specs"
+    When I convert the converted file to "v3"
+    Then the command exits with code 0
+    And the converted file declares formatVersion "3.1.0"
+    And the converted file has the same JSON as the fixture
+
+  Scenario: A Specs distribution declared as format 3 is refused
+    Given a file with the JSON '{"formatVersion": 3, "distribution": ["Specs", [["a"]], [], {"modules": []}]}'
+    When I run "ir inspect" on it
+    Then the command exits with code 1
+    And the errors contain "specs_before_3_1"
+
+  Scenario: A later 3.x minor is refused
+    Given a file with the JSON '{"formatVersion": "3.2.0", "distribution": []}'
+    When I run "ir inspect" on it
+    Then the command exits with code 1
+    And the errors contain "unsupported_format_version_minor"
+    And the errors contain "3.0.x, 3.1.x and draft 4.0.x"
+
+  Scenario: Application cannot be downgraded
+    Given a file with the JSON '{"formatVersion": 4, "distribution": {"Application": {"packageName": "a", "entryPoints": {}}}}'
     When I convert it to "v3"
     Then the command exits with code 1
     And the errors contain "unsupported_v4_downgrade"

features/steps/ir_interchange_steps.py

@@ -102,6 +102,12 @@ def step_declares_version(context: Context, version: int) -> None:
     assert json.loads(context.output.read_text())["formatVersion"] == version
 
 
+@then('the converted file declares formatVersion "{version}"')
+def step_declares_release(context: Context, version: str) -> None:
+    """Check the converted file's formatVersion release string."""
+    assert json.loads(context.output.read_text())["formatVersion"] == version
+
+
 @then("the converted file has the same JSON as the fixture")
 def step_same_json(context: Context) -> None:
     """Check the converted file matches the fixture JSON."""

openspec/changes/archive/2026-10-27-add-ir-3-1-0/.openspec.yaml

@@ -0,0 +1,2 @@
+schema: html-design
+created: 2026-10-27

openspec/changes/archive/2026-10-27-add-ir-3-1-0/proposal.md

@@ -0,0 +1,41 @@
+# Proposal
+
+## Why
+
+IR format `3.1.0` is a minor revision of the classic IR (finos/morphir decision record 0018, issue finos/morphir#970). It adds a v3 `Specs` distribution, which publishes the public face of a package without its definitions. The reference support table is now `[3.0.0,3.2.0),[4.0.0,4.1.0)`, and morphir-rust reads and writes `3.1.0` (finos/morphir-rust `cd9f7fb`). morphir-python still declares `[3.0.0,3.1.0)` for v3, so it refuses every `3.1.0` file with `unsupported_format_version_minor`, and it cannot write a v4 `Specs` distribution as v3. Issue [finos/morphir-python#28](https://github.com/finos/morphir-python/issues/28) asks to close that gap.
+
+## What Changes
+
+- Raise the support table to `[3.0.0,3.2.0),[4.0.0,4.1.0)`. `"3.1.0"` (and every `3.1.x`) is supported; `"3.2.0"` still fails with `unsupported_format_version_minor`. **BREAKING** for a caller that relies on `3.1.0` being refused.
+- Add a v3 `Specs` distribution to `morphir.ir.v3`: `["Specs", package, dependencies, { modules }]`, where the fourth element is a package specification. A v3 distribution is now `Library | Specs`. **BREAKING** for a caller that types the v3 decoder result as `Library` only.
+- Refuse a `Specs` distribution that declares a release below `3.1.0` (`3`, `"3"`, `"3.0.x"`) with the new diagnostic `specs_before_3_1`. Refuse a module definition among a `Specs` distribution's own modules.
+- Writers emit the lowest version that fits: a v3 `Library` keeps `formatVersion: 3`, even when it was read as `"3.1.0"`; a v3 `Specs` writes `"3.1.0"`.
+- Migrations: a v3 `Specs` migrates to a v4 `Specs` (`formatVersion` 4), and a v4 `Specs` downgrades to a v3 `Specs` with the same rules and diagnostics as the dependency specifications of a `Library`. An `Application` still has no v3 form.
+- `morphir-tools ir inspect` reports a v3 `Specs`; `ir convert` converts `Specs` in both directions.
+- Document the change in `packages/morphir/README.md` ("Morphir IR versions") and the `morphir-tools` README.
+- v3 document trees, the other half of decision 0018, stay out of scope: morphir-python has no document tree for v4 either. Follow-up bead `morphir-ukvn.8.17` tracks them.
+
+## Capabilities
+
+### New Capabilities
+
+- (none)
+
+### Modified Capabilities
+
+- `ir/format-version`: the support table becomes `[3.0.0,3.2.0),[4.0.0,4.1.0)`.
+- `ir/v3`: the v3 model and codec gain the `Specs` distribution, the `specs_before_3_1` refusal, the strict reading of a `Specs` distribution's own modules, and the lowest-version writer rule; finos/morphir is normative for what `3.1.0` adds.
+- `ir/migrate`: `Specs` migrates in both directions; only `Application` is refused on downgrade.
+- `tools/ir-interchange`: inspect and convert handle a v3 `Specs` and `3.1.0` input; the scope limit moves from `3.1.x` to later 3.x minors.
+
+## Beads
+
+- **Tracked by**: `morphir-ukvn.8.13`
+- **Related**: `morphir-ukvn.8` (parent epic, first-class IR v3 alongside v4), `morphir-f22s` (closed finos/morphir work that specified v3 document trees, finos/morphir#970), `morphir-ukvn.8.14` (realigns the v4 model in parallel; this change does not touch the v4 model or codec), `morphir-ukvn.8.17` (follow-up: v3 and v4 document trees)
+
+## Impact
+
+- `packages/morphir`: `ir.format_version` (support table, messages), `ir.diagnostics` (`specs_before_3_1`), `ir.v3.distribution` and `ir.v3.codec` (`Specs`), `ir.migrate` (`Specs` path only). The v4 model and codec do not change. The core package stays dependency-free.
+- `packages/morphir-tools`: `ir inspect` and `ir convert`.
+- Tests and fixtures: new `3.1.0` fixtures in `tests/fixtures/ir/v3/` with provenance (MCK case distributions-0011 and the morphir-rust test documents); BDD scenarios in `features/ir_interchange.feature`.
+- Users: files with `formatVersion` `"3.1.0"` now load; a reader that still declares `[3.0.0,3.1.0)` (morphir-elm today) refuses the `Specs` files this package writes, as decision 0018 intends.

openspec/changes/archive/2026-10-27-add-ir-3-1-0/specs/ir/format-version/spec.md

@@ -0,0 +1,22 @@
+## REMOVED Requirements
+
+### Requirement: Support table is checked before decoding
+**Reason**: The table now includes IR 3.1, so the scenario that refused `"3.1.0"` no longer holds.
+**Migration**: See the requirement "Support table with IR 3.1 is checked before decoding".
+
+## ADDED Requirements
+
+### Requirement: Support table with IR 3.1 is checked before decoding
+The reader SHALL declare the support table `[3.0.0,3.2.0),[4.0.0,4.1.0)` and MUST finish recognition and the support check before invoking any distribution decoder or migration. A release outside the table SHALL fail with `unsupported_format_version_major` or `unsupported_format_version_minor`. (bead: morphir-ukvn.8.13)
+
+#### Scenario: IR 3.1.0 is supported
+- **WHEN** a document declares `formatVersion` `"3.1.0"` or `"3.1.4"`
+- **THEN** the support check passes and the distribution is decoded
+
+#### Scenario: A later 3.x minor is refused
+- **WHEN** a document declares `formatVersion` `"3.2.0"`
+- **THEN** reading fails with `unsupported_format_version_minor` before the distribution is decoded, and the message names the accepted table `[3.0.0,3.2.0),[4.0.0,4.1.0)`
+
+#### Scenario: Historical versions are refused
+- **WHEN** a document declares `formatVersion` `1` or `2`
+- **THEN** reading fails with `unsupported_format_version_major` and a message that formats 1 and 2 are follow-up work

openspec/changes/archive/2026-10-27-add-ir-3-1-0/specs/ir/migrate/spec.md

@@ -0,0 +1,57 @@
+## REMOVED Requirements
+
+### Requirement: Reverse migrate v4 to v3 with known failure paths
+**Reason**: A v4 Specs now downgrades to a v3 Specs, so the scenario that refused Specs no longer holds.
+**Migration**: See the requirement "Reverse migrate v4 Library and Specs to v3 with known failure paths".
+
+## ADDED Requirements
+
+### Requirement: Reverse migrate v4 Library and Specs to v3 with known failure paths
+The library SHALL migrate v4 to v3. When the input uses concepts v3 cannot represent, migration SHALL fail with structured errors and MUST NOT invent or drop IR content, except as allowed by the dropped-information requirement. (bead: morphir-ukvn.8.13)
+
+#### Scenario: Application does not downgrade
+- **WHEN** a v4 Application distribution is migrated to v3
+- **THEN** migration fails with `unsupported_v4_downgrade` naming the distribution kind, because format 3 has no Application
+
+#### Scenario: V4-only nodes are reported
+- **WHEN** a v4 Library contains a `Hole`, `Native`, or `External` value, or a `DocumentLiteral`
+- **THEN** migration fails with `unsupported_v4_downgrade` naming the construct and its location
+
+#### Scenario: V4-only definitions and types are reported
+- **WHEN** a v4 Library or Specs contains a native, external or incomplete value-definition body, an incomplete type definition, a `Union` type, or a specification with annotations
+- **THEN** migration fails with `unsupported_v4_downgrade` naming the construct and its location
+
+#### Scenario: Missing types are reported
+- **WHEN** a v4 value, pattern, or value-definition input has no `inferred_type`
+- **THEN** migration fails with `missing_inferred_type` at that location
+
+#### Scenario: Semantic attributes are reported
+- **WHEN** v4 type attributes carry a source, constraints, or extensions, or value attributes carry extensions
+- **THEN** migration fails with `unsupported_attributes_downgrade`
+
+#### Scenario: Absent member documentation is reported
+- **WHEN** a v4 type or value member has no documentation
+- **THEN** migration fails with `unsupported_documentation_downgrade`, because v3 member documentation is a required string
+
+#### Scenario: Compatible Library downgrades
+- **WHEN** a v4 Library uses only concepts v3 can represent
+- **THEN** migration succeeds with a v3 Library
+
+### Requirement: Specs migrate between v3 and v4
+The library SHALL migrate a v3 `Specs` to a v4 `Specs` and a v4 `Specs` to a v3 `Specs`. Each specification SHALL map as the dependency specifications of a `Library` map, with the same diagnostics, warnings and caller-supplied package version. (bead: morphir-ukvn.8.13)
+
+#### Scenario: V3 Specs migrates forward
+- **WHEN** a v3 Specs is migrated to v4
+- **THEN** the result is a v4 Specs whose package name, dependencies and module specifications match the source, with no annotations
+
+#### Scenario: Compatible v4 Specs downgrades
+- **WHEN** a v4 Specs uses only concepts v3 can represent
+- **THEN** migration succeeds with a v3 Specs, and a package version is dropped with `package_version_dropped`
+
+#### Scenario: Incompatible v4 Specs is reported
+- **WHEN** a v4 Specs has a module member without documentation
+- **THEN** migration fails with `unsupported_documentation_downgrade` at that member
+
+#### Scenario: V3 Specs survives v3 to v4 to v3
+- **WHEN** a v3 Specs is migrated to v4 and back
+- **THEN** the result is semantically equal to the original

openspec/changes/archive/2026-10-27-add-ir-3-1-0/specs/ir/v3/spec.md

@@ -0,0 +1,88 @@
+## MODIFIED Requirements
+
+### Requirement: morphir-elm is normative for v3 content
+The v3 model and JSON shapes SHALL follow morphir-elm `Morphir.IR.*` types and `*.Codec` modules for format 3.0.0. Where finos/morphir v3 schemas or docs disagree with morphir-elm on 3.0.0 content, morphir-elm MUST win. What format 3.1.0 adds, which morphir-elm does not implement, SHALL follow the finos/morphir v3 specification and decision record 0018. (bead: morphir-ukvn.8.13)
+
+#### Scenario: Elm output decodes
+- **WHEN** a `morphir-ir.json` written by morphir-elm `morphir make` is decoded
+- **THEN** decoding succeeds without a Python-specific JSON variant
+
+#### Scenario: The finos 3.1.0 example decodes
+- **WHEN** the v3 `Specs` document of the finos/morphir kit case distributions-0011 is decoded and encoded again
+- **THEN** decoding succeeds and the output is equal to the kit's canonical JSON
+
+### Requirement: V3 model mirrors morphir-elm concepts
+The library SHALL expose a v3 model whose concepts match morphir-elm: names as word lists, paths, qualified and fully qualified names, access control, documented values, literals, types, values, patterns, modules, packages, and the `Library` distribution, plus the `Specs` distribution of format 3.1.0. A v3 `Library` SHALL carry type attributes of unit and value attributes of a v3 type, as in elm. (bead: morphir-ukvn.8.13)
+
+#### Scenario: Library distribution is representable
+- **WHEN** a caller constructs a v3 Library with a package name, dependency specifications, and a package definition whose values carry types
+- **THEN** the model keeps those parts without any v4-only construct
+
+#### Scenario: Library has no package version
+- **WHEN** a caller constructs a v3 Library
+- **THEN** the model has no package release version, because format 3.0.0 has none
+
+#### Scenario: Specs distribution is representable
+- **WHEN** a caller constructs a v3 Specs with a package name, dependency specifications, and a package specification
+- **THEN** the model keeps those parts, holds no definitions, and has no package release version
+
+### Requirement: V3 JSON matches morphir-elm format 3.0.0
+The library SHALL encode and decode versioned distribution JSON with the shapes of morphir-elm `Morphir.IR.FormatVersion.Codec` and the codecs it uses. Node codecs SHALL take attribute codecs as parameters, as elm does. Version recognition SHALL use the shared format-version reader. (bead: morphir-ukvn.8.13)
+
+#### Scenario: Round trip versioned morphir-ir.json
+- **WHEN** a valid morphir-elm `morphir-ir.json` with `formatVersion` 3 is decoded and encoded again
+- **THEN** the output declares `formatVersion` 3 and its distribution is semantically equal to the input
+
+#### Scenario: String version is accepted on read
+- **WHEN** an otherwise valid format-3 document declares `formatVersion` `"3.0.0"`
+- **THEN** it decodes, and encoding writes the integer `3`
+
+#### Scenario: Wrong family at the v3 entry point
+- **WHEN** a document with `formatVersion` 4 is passed to the v3 decoder
+- **THEN** decoding fails with `unsupported_format_version_major` naming the found version and that only 3.0.x and 3.1.x are accepted
+
+#### Scenario: Unknown distribution kind
+- **WHEN** a format-3 document has a distribution tag other than `Library` or `Specs`
+- **THEN** decoding fails with `unknown_node` naming the tag and both known kinds
+
+## ADDED Requirements
+
+### Requirement: V3 Specs distribution JSON
+The library SHALL encode and decode a v3 `Specs` distribution as `["Specs", package, dependencies, specification]`, where `specification` is a package specification whose modules are spelled as a dependency's modules are. (bead: morphir-ukvn.8.13)
+
+#### Scenario: Specs round trips
+- **WHEN** a document with `formatVersion` `"3.1.0"` and a `Specs` distribution is decoded and encoded again
+- **THEN** the output declares `formatVersion` `"3.1.0"` and its distribution is semantically equal to the input
+
+#### Scenario: A Specs distribution with dependencies
+- **WHEN** a `Specs` distribution lists dependency package specifications
+- **THEN** they decode as the dependencies of a `Library` do
+
+### Requirement: A Specs distribution needs 3.1.0 or later
+The v3 reader SHALL refuse a `Specs` distribution whose document declares a release below `3.1.0`, such as `3`, `"3"` or `"3.0.1"`, with `specs_before_3_1` at the `formatVersion` member. A `Library` SHALL stay readable at every supported 3.x release. (bead: morphir-ukvn.8.13)
+
+#### Scenario: Specs declared as 3 is refused
+- **WHEN** a document declares `formatVersion` `3`, `"3"`, `"3.0.0"` or `"3.0.1"` and holds a `Specs` distribution
+- **THEN** decoding fails with `specs_before_3_1`
+
+#### Scenario: Library declared as 3.1.0 is read
+- **WHEN** a document declares `formatVersion` `"3.1.0"` and holds a `Library` distribution
+- **THEN** it decodes as a v3 Library
+
+### Requirement: A Specs distribution holds specifications only
+The v3 reader SHALL refuse a member other than `types`, `values` and `doc` in a module of a `Specs` distribution's own package specification with `unknown_member`, so a module definition is not read as a specification. Dependency modules keep the morphir-elm reading. (bead: morphir-ukvn.8.13)
+
+#### Scenario: A module definition in a Specs is refused
+- **WHEN** a `Specs` distribution's own module is spelled `{"access": "Public", "value": {...}}`
+- **THEN** decoding fails with `unknown_member` at that module
+
+### Requirement: Writers emit the lowest version that fits
+The v3 writer SHALL emit `formatVersion` `3` for a `Library` and `"3.1.0"` for a `Specs`, whatever version the input declared, so every reader of 3.0.0 still reads every `Library` this library writes. (bead: morphir-ukvn.8.13)
+
+#### Scenario: Library read as 3.1.0 is written as 3
+- **WHEN** a `Library` read from a document that declares `"3.1.0"` is encoded
+- **THEN** the output declares the integer `formatVersion` `3`
+
+#### Scenario: Specs is written as 3.1.0
+- **WHEN** a v3 `Specs` is encoded
+- **THEN** the output declares `formatVersion` `"3.1.0"`