Skip to content

Remove governed envelopes; keep frame types 8 and 9 reserved #46

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
TeoSlayer merged 1 commit into from
Oct 1, 2026
Merged

Remove governed envelopes; keep frame types 8 and 9 reserved #46

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,27 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Removed

- Governed delivery. The signed-envelope frames `TypeGoverned` (8) and
`TypeGovernedFileStream` (9) carried decisions issued by the hosted control
plane, which has been retired; the daemon stopped configuring the receiver
gates in v1.14.0, so the code could no longer be reached. Removed: every
exported `Governed*` identifier (the frame and stream-INIT envelopes and
their codecs, the verifier and receipt-recorder interfaces, the payload
hash helpers, `GovernedRetentionPolicy`), `DecisionFrameVerifier`,
`Client.SendGoverned*`, `BuildStreamInitPayload`, and the `ServiceConfig`
fields `RequireGoverned`, `GovernedVerifier`, `GovernedStreamVerifier`,
`RequireGovernedReceipts`, `GovernedReceiptRecorder`,
`GovernedContentInspector`, `RequireGovernedContentInspection`,
`GovernedTransferQuota`, `GovernedRetentionPolicies`, `RetentionStateDir`
and `RetentionSweepInterval`. The module no longer imports
`github.com/pilot-protocol/common/decision`.
- Frame types 8 and 9 stay reserved and keep their constants and names. A
receiver refuses them like any unsupported type: nothing is stored and the
sender gets `ERR GOVERNED save failed: unsupported frame type 8` (or
`ERR GOVERNED_FILESTREAM ... type 9`); the connection stays open.

### Fixed

- The inbox byte cap no longer deletes the whole inbox. With the default
Expand Down
107 changes: 18 additions & 89 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,10 +30,9 @@ rt.Register(dataexchange.NewService(dataexchange.ServiceConfig{}))

| File | What it does |
|---|---|
| `dataexchange.go` | Wire format: `Frame`, `WriteFrame`, `ReadFrame`, `TraceFrame`, `TypeText/Binary/JSON/File/Trace/Governed/Tagged`, `TypeName`. |
| `dataexchange.go` | Wire format: `Frame`, `WriteFrame`, `ReadFrame`, `TraceFrame`, `TypeText/Binary/JSON/File/Trace/Tagged`, `TypeName`. |
| `tagged.go` | Optional request/reply correlation on the wire: `Frame.MessageID` / `Frame.ReplyTo` (`TypeTagged`), `NewMessageID`, `ValidMessageID`. |
| `client.go` | `Client` — `Dial`, `Send` (waits for the ACK, falls back for older receivers) and send helpers. |
| `governed.go` | Signed decision envelope, receiver-side verifier, and enforceable transport constraints. |
| `server.go` | `Server` — accept loop and handler dispatch. |
| `service.go` | `*Service` — `coreapi.Service` adapter. Build tag `!no_dataexchange`. |
| `inbox_budget.go` | Inbox caps: running byte total, oldest-first eviction to 90% of the byte cap, file-count cap. |
Expand Down Expand Up @@ -96,8 +95,8 @@ a frame can carry two optional fields:
| `Frame.ReplyTo` | `reply_to` | The `message_id` of the message this one answers. |

IDs are 1–128 characters of `A-Z a-z 0-9 - _ . :`. They are correlation
metadata only: they are not authenticated (a governed frame's signature does
not cover them), so never use them for authorization.
metadata only: they are not authenticated, so never use them for
authorization.

**If you run a service** (an agent that answers requests): when a request's
inbox record (or `message.received` event) has a `message_id`, send your
Expand Down Expand Up @@ -147,8 +146,8 @@ untagged (`SendResult.Tagged == false`), and a raw `WriteFrame` returns
`ErrTaggedFrameTooLarge` without writing anything.

Whenever `Tagged` is false, the reply cannot carry `reply_to`; match it on
the sender alone. The fields can accompany text, JSON, binary, file,
governed and trace frames. On `TypeFileStream` frames they are ignored.
the sender alone. The fields can accompany text, JSON, binary, file and
trace frames. On `TypeFileStream` frames they are ignored.

Wire layout of a tagged frame, for other languages:

Expand All @@ -169,89 +168,19 @@ without a `MessageID` are always stored, as before, unless the operator sets
`ServiceConfig.DedupeContentWindow` (off by default) to also drop
byte-identical copies from the same sender inside a short window.

## Governed delivery

`TypeGoverned` wraps one text, JSON, binary, or single-frame file delivery in
the sender's signed `decision.Intent` and signed `decision.Decision`. The
intent payload hash binds the frame type, filename, and exact bytes; the
receiver verifies the signatures, tenant authority state, local deterministic
ceiling, exact local destination, and any applicable transport constraints
before it writes to disk. A workflow-approved action uses the same short-lived
execution Decision as an ordinary allowed action—there is no reusable
transport permit.

Large resumable files use `TypeGovernedFileStream`: the signed envelope binds
the exact `TypeFileStream` INIT (filename, declared length, full SHA-256,
chunk size, and transfer ID). Required receivers admit later chunks only for
that verified transfer on the same connection. Compute the file hash first,
call `BuildStreamInitPayload`, sign an Intent using
`GovernedStreamPayloadHash`, then call `SendGovernedFileStream`.

For a required typed-disclosure profile, build a `decision.DisclosureBinding`
whose content hash, byte length, filename, and stream transfer ID match the
file, bind its canonical hash in the signed Intent, and use
`SendGovernedWithDisclosure` or `SendGovernedFileStreamWithDisclosure`. A
`DecisionFrameVerifier` with `RequireDisclosure` rejects governed messages,
single-frame files, and resumable stream INITs that omit this evidence.
When a receipt recorder is configured, typed deliveries require its V2
disclosure-evidence method; the resulting signed receipt binds the canonical
disclosure hash without retaining the file or message body.

Set `ServiceConfig.GovernedVerifier` to `DecisionFrameVerifier` (or an
equivalent local verifier). Once all senders have been upgraded, set
`ServiceConfig.RequireGoverned` to reject unsigned legacy deliveries. Roll out
in that order: upgraded receiver with verification available, upgraded
senders, then required mode. An older receiver does not understand
`TypeGoverned`; a required receiver intentionally rejects `TypeTrace` and
raw `TypeFileStream` INIT frames. Governed stream INITs are supported.

For auditable enterprise ingress, configure `GovernedReceiptRecorder` and set
`RequireGovernedReceipts`. The service writes the received message/file first,
then requires the recorder to durably capture the exact signed Intent and
Decision before it emits the success ACK or delivery event. If recording fails,
the staged file is removed and the sender receives an error rather than a
successful but unreceipted delivery.

### Local content inspection

`ServiceConfig.GovernedContentInspector` is an optional receiver-local hook
that runs after the signed envelope and local policy ceiling are verified but
before a message/file is released. Set
`RequireGovernedContentInspection` to make startup fail unless the hook is
present; an inspection error removes a staged file or rejects the message.
`decision.PresidioInspector` is the included OSS adapter for bounded text,
JSON, XML, YAML, and form content. It rejects unsupported binary/document
types rather than truncating or silently skipping them. The inspector is local
to the receiver; neither the decision authority nor the sender's authority
receives plaintext for this check.

Typed disclosure binding V2 adds a tenant-defined `retention_class` to the
same Intent hash. A signed policy can select allowed classes; the receiver sees
the bound metadata. Configure `GovernedRetentionPolicies` to map those classes
to local expiry durations. The service writes an owner-only retention journal
before a governed message/file becomes accepted (and before a streamed file's
final rename), then removes the content after expiry across restarts. An
unknown, V1, or unconfigured class is rejected when retention is enabled.
This is deletion retention, not a legal-hold or WORM-storage implementation.

### Per-agent transfer quotas

`ServiceConfig.GovernedTransferQuota` admits a bounded number of bytes and/or
actions for each signed `Intent.AgentID` in a fixed local window. It is charged
only after governed verification, including the declared bytes of a verified
stream INIT; a peer address cannot select or reset another agent's budget.
Quota is deliberately charged for an admitted attempt even if later local DLP
or receipt persistence rejects it, so repeatedly failing submissions cannot
turn the scanner into an unmetered denial-of-service target.

The v1 action mapping is:

| Frame | Intent action |
|---|---|
| text | `data.send.text` |
| JSON | `data.send.json` |
| binary | `data.send.binary` |
| file | `file.share` |
## Reserved frame types

Types 8 (`TypeGoverned`) and 9 (`TypeGovernedFileStream`) carried deliveries
wrapped in an envelope signed by the hosted control plane. That control plane
has been retired and the envelope code was removed with it. The two numbers
stay reserved and will not be given to a new frame type, because senders built
before the removal can still emit them.

A receiver treats both like any frame type it does not implement: it stores
nothing, publishes no event, answers
`ERR GOVERNED save failed: unsupported frame type 8` (or
`ERR GOVERNED_FILESTREAM save failed: unsupported frame type 9`), and keeps
the connection open for further frames.

## Build tags

Expand Down
113 changes: 3 additions & 110 deletions client.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@ import (
"strings"
"time"

"github.com/pilot-protocol/common/decision"
"github.com/pilot-protocol/common/driver"
"github.com/pilot-protocol/common/protocol"
)
Expand Down Expand Up @@ -65,8 +64,9 @@ const duplicateAckSuffix = " (duplicate)"
// "ACK UNKNOWN(10) <n> bytes": a success ACK for a frame it dropped.
// - dataexchange v0.2.2 and later, before TypeTagged, reject an unknown
// type and answer "ERR UNKNOWN(10) save failed: unsupported frame type 10".
// A RequireGoverned receiver answers "ERR UNKNOWN(10) save failed:
// unsigned legacy frame ..." instead, also without storing it.
// A receiver of that era configured to require governed envelopes (since
// removed) answers "ERR UNKNOWN(10) save failed: unsigned legacy frame ..."
// instead, also without storing it.
//
// A receiver that knows TypeTagged unwraps the frame in ReadFrame and
// acknowledges the inner type (which can never be TypeTagged), so its ACK
Expand Down Expand Up @@ -177,113 +177,6 @@ func (c *Client) SendFile(name string, data []byte) error {
return WriteFrame(c.conn, &Frame{Type: TypeFile, Filename: name, Payload: data})
}

// SendGoverned sends a frame with exact signed intent/decision evidence.
// The remote service must have RequireGoverned enabled to enforce it.
func (c *Client) SendGoverned(frame *Frame, intent decision.Intent, result decision.Decision) error {
governed, err := NewGovernedFrame(frame, intent, result)
if err != nil {
return err
}
envelope, err := EncodeGovernedFrame(governed)
if err != nil {
return err
}
return WriteFrame(c.conn, envelope)
}

// SendGovernedWithDisclosure sends a single governed message or file with a
// typed disclosure binding. The Intent and Decision must already have been
// obtained for the exact binding.
func (c *Client) SendGovernedWithDisclosure(frame *Frame, intent decision.Intent, result decision.Decision, disclosure decision.DisclosureBinding) error {
governed, err := NewGovernedFrameWithDisclosure(frame, intent, result, disclosure)
if err != nil {
return err
}
envelope, err := EncodeGovernedFrame(governed)
if err != nil {
return err
}
return WriteFrame(c.conn, envelope)
}

// SendGovernedFileStream sends a large resumable file with a signed authority
// decision bound to its exact INIT metadata. Callers compute the intent payload
// hash with GovernedStreamPayloadHash over the deterministic INIT payload (use
// BuildStreamInitPayload after calculating the file hash), then pass the same
// signed intent and decision here. The receiver accepts chunks only for that
// verified transfer ID and records its receipt on successful completion.
func (c *Client) SendGovernedFileStream(name string, r io.ReadSeeker, size int64, intent decision.Intent, result decision.Decision, stepTimeout time.Duration) (*StreamResult, error) {
return c.SendGovernedFileStreamWithAuthorizer(name, r, size, func(_ []byte) (decision.Intent, decision.Decision, error) {
return intent, result, nil
}, stepTimeout)
}

// SendGovernedFileStreamWithDisclosure sends a governed resumable file whose
// Intent and authority Decision bind the supplied typed disclosure metadata.
// The metadata must match the final INIT (content hash, size, filename, and
// transfer ID) exactly; callers can derive these with BuildStreamInitPayload.
func (c *Client) SendGovernedFileStreamWithDisclosure(name string, r io.ReadSeeker, size int64, intent decision.Intent, result decision.Decision, disclosure decision.DisclosureBinding, stepTimeout time.Duration) (*StreamResult, error) {
return streamSendWithInit(c.conn, name, r, size, stepTimeout, func(id [transferIDLen]byte, declaredSize uint64, hash [32]byte, chunkSize uint32, filename string) (*Frame, error) {
init := encodeInit(id, declaredSize, hash, chunkSize, filename)
governed, err := NewGovernedStreamInitWithDisclosure(init, intent, result, disclosure)
if err != nil {
return nil, err
}
return EncodeGovernedStreamInit(governed)
})
}

// GovernedStreamAuthorizer receives the exact FileStream INIT payload after
// its stable transfer ID and content hash have been calculated. It must return
// a short-lived signed Intent and Decision bound to that exact payload. The
// callback runs before any stream frame is written, so a deny or unavailable
// authority cannot produce a partial transfer.
type GovernedStreamAuthorizer func(initPayload []byte) (decision.Intent, decision.Decision, error)

// GovernedStreamDisclosureAuthorizer returns typed disclosure evidence for
// the exact INIT. Hosted federation clients use this after uploading the full
// file content and before the first stream byte is released to the peer.
type GovernedStreamDisclosureAuthorizer func(initPayload []byte) (decision.Intent, decision.Decision, decision.DisclosureBinding, error)

// SendGovernedFileStreamWithAuthorizer obtains an exact signed decision only
// after the resumable stream's INIT bytes are known. This is the preferred
// sender API when the decision comes from an online authority.
func (c *Client) SendGovernedFileStreamWithAuthorizer(name string, r io.ReadSeeker, size int64, authorize GovernedStreamAuthorizer, stepTimeout time.Duration) (*StreamResult, error) {
if authorize == nil {
return nil, fmt.Errorf("dataexchange: governed stream authorizer is required")
}
return streamSendWithInit(c.conn, name, r, size, stepTimeout, func(id [transferIDLen]byte, declaredSize uint64, hash [32]byte, chunkSize uint32, filename string) (*Frame, error) {
init := encodeInit(id, declaredSize, hash, chunkSize, filename)
intent, result, err := authorize(append([]byte(nil), init.Payload...))
if err != nil {
return nil, err
}
governed, err := NewGovernedStreamInit(init, intent, result)
if err != nil {
return nil, err
}
return EncodeGovernedStreamInit(governed)
})
}

func (c *Client) SendGovernedFileStreamWithDisclosureAuthorizer(name string, r io.ReadSeeker, size int64, authorize GovernedStreamDisclosureAuthorizer, stepTimeout time.Duration) (*StreamResult, error) {
if authorize == nil {
return nil, fmt.Errorf("dataexchange: governed stream disclosure authorizer is required")
}
return streamSendWithInit(c.conn, name, r, size, stepTimeout, func(id [transferIDLen]byte, declaredSize uint64, hash [32]byte, chunkSize uint32, filename string) (*Frame, error) {
init := encodeInit(id, declaredSize, hash, chunkSize, filename)
intent, result, disclosure, err := authorize(append([]byte(nil), init.Payload...))
if err != nil {
return nil, err
}
governed, err := NewGovernedStreamInitWithDisclosure(init, intent, result, disclosure)
if err != nil {
return nil, err
}
return EncodeGovernedStreamInit(governed)
})
}

// SendTrace wraps data in a TypeTrace frame with the current nanosecond clock.
// Returns sentAtNs so the caller can correlate it against the timing ACK.
func (c *Client) SendTrace(innerType uint32, data []byte) (sentAtNs int64, err error) {
Expand Down
15 changes: 8 additions & 7 deletions dataexchange.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,13 +33,14 @@ const (
// peer that does not understand TypeFileStream never sends INIT-ACK, so
// the sender falls back to TypeFile.
TypeFileStream uint32 = 7
// TypeGoverned carries a regular data frame together with the sender's
// signed intent and authority decision. It is opt-in; enterprise receivers
// can require this envelope before persisting a message or file.
TypeGoverned uint32 = 8
// TypeGovernedFileStream carries the signed authorization evidence for a
// TypeFileStream INIT. Subsequent chunks are accepted only while bound to
// that verified transfer ID on the same connection.
// TypeGoverned and TypeGovernedFileStream are reserved. They carried a
// data frame (8) or a TypeFileStream INIT (9) inside an envelope signed
// by the hosted control plane, which has been retired; the envelope
// codec and its receiver gates were removed with it. The numbers must
// not be reused: senders built before the removal can still emit them,
// and a receiver refuses both as unsupported frame types without
// storing anything.
TypeGoverned uint32 = 8
TypeGovernedFileStream uint32 = 9
// TypeTagged wraps another frame with optional correlation metadata
// (Frame.MessageID / Frame.ReplyTo). Callers never set it directly:
Expand Down
2 changes: 1 addition & 1 deletion dedupe.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -181,7 +181,7 @@ func (d *deliveryDedupe) len() int {
// Streams have their own resume protocol and trace frames are diagnostics.
func dedupeEligible(frameType uint32) bool {
switch frameType {
case TypeText, TypeJSON, TypeBinary, TypeFile, TypeGoverned:
case TypeText, TypeJSON, TypeBinary, TypeFile:
return true
default:
return false
Expand Down
Loading
Loading