You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
As an OpenShell policy author and integrator, I want every runtime and policy-analysis consumer to interpret the same authored policy document consistently, so that a policy has the same fields, defaults, validation outcome, and serialized meaning everywhere it is used without requiring YAML-only consumers to depend on protobuf, gRPC, or solver infrastructure.
Problem Statement
OpenShell does not currently have one public representation for the authored policy format. The runtime policy engine owns private serde types and converts them to protobuf, while policy-analysis paths maintain separate serde projections. Maximum-policy containment in #3289 adds another projection.
These representations already disagree about observable behavior. Examples include the effective value of an omitted filesystem_policy, whether version may be omitted, port width, and the fallback from an empty network-rule name to its map key. Partial policy deserializers for middleware and MCP/JSON-RPC fragments add further schema ownership outside the main parser.
The inconsistency also makes it difficult for dependency-sensitive consumers to work with policy YAML directly. The existing public parsing API returns protobuf types and therefore brings in dependencies unrelated to parsing or serialization.
Impact / Why This Matters
Today, the same authored policy can be accepted, rejected, defaulted, or classified differently depending on whether it is loaded by runtime enforcement, policy analysis, or maximum-policy containment. That creates security and compatibility risk: an operator cannot rely on one policy document having one meaning across validation, activation, proof, and round-trip serialization.
Maintainers must duplicate schema changes, defaults, validation rules, parser limits, and tests across consumers. New fields can silently disappear from a partial projection, and consumers that only need authored YAML must either duplicate the format themselves or compile the protobuf/gRPC dependency stack. Cross-consumer tests reduce some risk but do not establish a single ownership boundary.
Proposed Design
Provide one public, dependency-light representation of the complete authored OpenShell policy format in openshell-policy-schema. It must preserve authoring distinctions whose meaning depends on presence, including an omitted section versus a present empty object, and expose stable, consumer-oriented policy types.
The schema crate owns intrinsic authored-language meaning, including access-preset vocabulary and HTTP/WebSocket method expansion, effective ports and rule names, protocol classification, filesystem defaults, and lexical path normalization. Consumer-specific risk classification, runtime composition, and containment modeling remain outside the schema crate.
Every authored-policy consumer should parse and serialize through this representation, then adapt the typed document into its runtime or analysis model. Protobuf messages and prover models remain valid derived representations, but they must not independently define the authored YAML or JSON schema.
The shared parser should expose explicit consumer profiles:
Runtime loading remains strict: unsupported schema versions, unknown fields in closed policy objects, and managed-maximum-only annotations are rejected.
Maximum-policy containment can retain recognized managed annotations and unknown or valid-but-unmodeled policy shapes so the checker can report them as unsupported evidence rather than misclassifying them as malformed input.
User-defined namespaces—including middleware configuration and nested query, persisted-query, and MCP parameter maps—remain open data maps and must not be mistaken for unknown schema extensions.
Parsing and file loading must enforce documented resource limits while consuming input, including input length, nesting, aliases, nodes/events, scalar bytes, and collection sizes. Diagnostics must remain bounded and Unicode-safe.
Conversions between the authored representation and protobuf must distinguish invalid authored values, expected canonicalization, reversible representation changes, and intentionally omitted runtime provenance. Validation of raw protobuf presence must occur before a conversion can erase a meaningful presence distinction.
Existing runtime parsing and serialization entry points should remain available as compatibility adapters. The intentional alignment of previously divergent defaults or validation outcomes must be documented in the implementing pull request and user-facing policy documentation.
Acceptance Criteria
A public authored-policy API can parse, inspect, compare, and serialize every currently supported policy field without depending on OpenShell protobuf/gRPC, runtime, control-plane, or solver crates.
openshell-policy-schema owns intrinsic authored-language behavior, including access-preset vocabulary and method expansion.
Runtime policy loading and policy-analysis consumers use the same authored representation and agree on known field values, presence-sensitive defaults, effective ports, effective rule names, and access presets.
Authored policies require the supported explicit schema version consistently across consumers.
An omitted filesystem_policy resolves to the runtime-effective include_workdir: true, while a present empty filesystem_policy: {} remains distinct and resolves to false; the analysis behavior correction is documented.
Unknown fields in closed policy objects produce a field-specific runtime parse error, while containment reports syntactically valid unknown or valid-but-unmodeled authority as unsupported_policy_shape rather than invalid_input.
Middleware configuration, query and persisted-query maps, and recursively nested MCP parameter keys round-trip as user data, including ambiguous nested-map cases.
Malformed schema-owned fields remain invalid input. Containment text and JSON results distinguish invalid input (exit 2) from unsupported policy shape (exit 3).
Protobuf conversion rejects invalid authored values such as overflowing ports, validates meaningful raw message presence before canonicalization, and permits documented canonicalization and omission of runtime-only provenance.
Parser and file-loading limits are enforced before unbounded allocation, with coverage for oversized, deeply nested, alias-heavy, large-scalar, and growing-file inputs.
Round-trip, cross-consumer parity, raw-protobuf-presence, and non-ASCII diagnostic tests cover the complete compatibility contract.
The relevant policy/sandbox end-to-end path passes and demonstrates canonical parsing through sandbox creation or update, including omitted-versus-empty filesystem behavior.
Policy architecture and user-facing schema/prover documentation identify the authored schema owner, validation profiles, resource limits, normalization behavior, and intentional compatibility changes.
Keep separate serde projections and share only semantic helpers. Shared access-preset helpers reduce one source of drift, but each parser can still differ in accepted shape, defaults, presence, unknown-field behavior, limits, and future field coverage.
Use protobuf as the canonical authored model. This preserves the current adapter API but keeps YAML-only consumers coupled to generated/runtime dependencies and cannot faithfully preserve every authoring-time presence distinction.
Use one runtime and solver model everywhere. Runtime protobuf messages and prover models serve different purposes and carry different state. Sharing the authored representation removes syntax drift without forcing consumers into an unsafe universal execution model.
Agent Investigation
crates/openshell-policy/src/lib.rs currently owns the runtime serde document and YAML/protobuf conversion.
crates/openshell-policy/src/middleware.rs and MCP/JSON-RPC normalization paths deserialize partial authored-policy projections separately.
crates/openshell-prover/src/policy.rs maintains a separate permissive serde projection for policy analysis.
User Story
As an OpenShell policy author and integrator, I want every runtime and policy-analysis consumer to interpret the same authored policy document consistently, so that a policy has the same fields, defaults, validation outcome, and serialized meaning everywhere it is used without requiring YAML-only consumers to depend on protobuf, gRPC, or solver infrastructure.
Problem Statement
OpenShell does not currently have one public representation for the authored policy format. The runtime policy engine owns private serde types and converts them to protobuf, while policy-analysis paths maintain separate serde projections. Maximum-policy containment in #3289 adds another projection.
These representations already disagree about observable behavior. Examples include the effective value of an omitted
filesystem_policy, whetherversionmay be omitted, port width, and the fallback from an empty network-rule name to its map key. Partial policy deserializers for middleware and MCP/JSON-RPC fragments add further schema ownership outside the main parser.The inconsistency also makes it difficult for dependency-sensitive consumers to work with policy YAML directly. The existing public parsing API returns protobuf types and therefore brings in dependencies unrelated to parsing or serialization.
Impact / Why This Matters
Today, the same authored policy can be accepted, rejected, defaulted, or classified differently depending on whether it is loaded by runtime enforcement, policy analysis, or maximum-policy containment. That creates security and compatibility risk: an operator cannot rely on one policy document having one meaning across validation, activation, proof, and round-trip serialization.
Maintainers must duplicate schema changes, defaults, validation rules, parser limits, and tests across consumers. New fields can silently disappear from a partial projection, and consumers that only need authored YAML must either duplicate the format themselves or compile the protobuf/gRPC dependency stack. Cross-consumer tests reduce some risk but do not establish a single ownership boundary.
Proposed Design
Provide one public, dependency-light representation of the complete authored OpenShell policy format in
openshell-policy-schema. It must preserve authoring distinctions whose meaning depends on presence, including an omitted section versus a present empty object, and expose stable, consumer-oriented policy types.The schema crate owns intrinsic authored-language meaning, including access-preset vocabulary and HTTP/WebSocket method expansion, effective ports and rule names, protocol classification, filesystem defaults, and lexical path normalization. Consumer-specific risk classification, runtime composition, and containment modeling remain outside the schema crate.
Every authored-policy consumer should parse and serialize through this representation, then adapt the typed document into its runtime or analysis model. Protobuf messages and prover models remain valid derived representations, but they must not independently define the authored YAML or JSON schema.
The shared parser should expose explicit consumer profiles:
User-defined namespaces—including middleware configuration and nested query, persisted-query, and MCP parameter maps—remain open data maps and must not be mistaken for unknown schema extensions.
Parsing and file loading must enforce documented resource limits while consuming input, including input length, nesting, aliases, nodes/events, scalar bytes, and collection sizes. Diagnostics must remain bounded and Unicode-safe.
Conversions between the authored representation and protobuf must distinguish invalid authored values, expected canonicalization, reversible representation changes, and intentionally omitted runtime provenance. Validation of raw protobuf presence must occur before a conversion can erase a meaningful presence distinction.
Existing runtime parsing and serialization entry points should remain available as compatibility adapters. The intentional alignment of previously divergent defaults or validation outcomes must be documented in the implementing pull request and user-facing policy documentation.
Acceptance Criteria
openshell-policy-schemaowns intrinsic authored-language behavior, including access-preset vocabulary and method expansion.filesystem_policyresolves to the runtime-effectiveinclude_workdir: true, while a present emptyfilesystem_policy: {}remains distinct and resolves tofalse; the analysis behavior correction is documented.unsupported_policy_shaperather thaninvalid_input.Alternatives Considered
Keep separate serde projections and share only semantic helpers. Shared access-preset helpers reduce one source of drift, but each parser can still differ in accepted shape, defaults, presence, unknown-field behavior, limits, and future field coverage.
Use protobuf as the canonical authored model. This preserves the current adapter API but keeps YAML-only consumers coupled to generated/runtime dependencies and cannot faithfully preserve every authoring-time presence distinction.
Use one runtime and solver model everywhere. Runtime protobuf messages and prover models serve different purposes and carry different state. Sharing the authored representation removes syntax drift without forcing consumers into an unsafe universal execution model.
Agent Investigation
crates/openshell-policy/src/lib.rscurrently owns the runtime serde document and YAML/protobuf conversion.crates/openshell-policy/src/middleware.rsand MCP/JSON-RPC normalization paths deserialize partial authored-policy projections separately.crates/openshell-prover/src/policy.rsmaintains a separate permissive serde projection for policy analysis.mainand contains only the canonical schema and migrations for consumers present onmain.Checklist