Skip to content

Vendored the stream additions to the public API protos. - #10

Closed
moedash wants to merge 10 commits into
mainfrom
moe/AI-198-py-01-protos
Closed

moedash wants to merge 10 commits into
mainfrom
moe/AI-198-py-01-protos

Conversation

@moedash

@moedash moedash commented Sep 25, 2026 •

Copy link
Copy Markdown
Owner

This PR vendors the stream and notification channel additions to the public API protos.

What changed?

  • temporalio/api/stream/v1/ is new. It holds StreamRecord, StreamRecordKind and StreamStartPosition. Every provider in the series moves that record, with the user's value in body as an ordinary payload, so a codec applies.
  • command/v1, enums/v1 and history/v1 gain the subscribe and append stream commands, their command types, the two workflow stream event types and the new failed causes. workflowservice/v1 gains the fields that travel with them.
  • temporalio/api/notification/v1/ is new. It holds Notification, ChannelKind, ChannelListener and WorkflowListener. workflowservice/v1 gains NotifyChannel, RegisterChannelListener, UnregisterChannelListener, PollChannel and DescribeChannel; command/v1 gains the SubscribeNotificationChannel and UnsubscribeNotificationChannel commands, history/v1 the subscribed and unsubscribed events and the notifications field on the scheduled Workflow Task event, and enums/v1 their command types, event types and failed causes. The bridge protos gain the matching commands and the NotificationsReceived activation job, and services_generated.py and client_rpc_generated.rs carry the five calls.
  • workflow/v1 gains ChannelSubscriptionInfo, and DescribeWorkflowExecutionResponse gains channel_subscriptions: the channels a run stands on, with the counter it last accepted, the pending and scheduled notifications and the linked channel's counts.
  • A channel comes in two kinds. The independent kind is its own execution, keyed by namespace and name. The linked kind lives in one execution's state, a workflow's or a standalone activity's: Notification.linked_to names the owner as a common.v1.Execution, the five channel requests take an optional execution that addresses it, and DescribeChannelResponse reports kind and linked_to.
  • temporalio/bridge/sdk-core pins a protos-only Core branch. It's the commit main already pins, with the api branch's stream and channel diff applied to its api_upstream and the two bridge protos.
  • scripts/gen_stream_api_protos.py stages the pinned API protos, applies the api branch's diff, and regenerates only the files that diff touches.

Nothing imports these modules yet. The three layers on top are written against them.

Part of AI-198 (epic AI-37).

Why?

The stream protos live on an api branch that upstream Core doesn't pin. Regenerating everything from that branch would pull in unrelated api churn and bury the stream diff. The Core repin lets poe gen-protos reproduce this tree byte for byte, which is what check-protos asserts. The script stays because the next api branch update needs it again.

Two kinds of channel because most listeners are one workflow. An independent channel pays its own writes per notification on top of the listener's; a channel kept in the listener's own state makes a notification one write on the owner, the same write that schedules its task, and needs no subscribe command.

How did you test it?

I ran uv run poe lint and the unit suite without the stream cases under uv run pytest, plus tests/test_service.py, which checks the generated service surface against the protos. A one-off script round-trips a StreamRecord through the vendored modules. The PR adds no tests of its own, since it's generated code only.

  • Unit Tests
  • Staging
  • End to End Tests

The generator stages the pinned API tree, applies the stream branch's diff
and regenerates only the files it touches. Core moves to
`moe/AI-198-api-protos-on-85b71d7`, which is the commit main pins plus that
same diff on its `api_upstream`, so the vendored tree regenerates
byte-for-byte from the pin and `check-protos` agrees.
The stream commands name a stream rather than an id, and an appended batch
records its end offset rather than a count. Core moves to the matching
protos-only branch so the vendored tree still regenerates from the pin.
Core moves to the protos-only branch that carries the new StreamStartPosition message and the start_position field, so the vendored tree regenerates from the pin.
Core moves to the protos-only branch that carries the Wake message, the
wakes field on the workflow task poll response and the WakeWorkflowExecution
call, so the vendored tree and the bridge client regenerate from the pin.
…the clients.

Core moves to the protos-only branch commit that carries the notification channel: the Notification message, the five channel calls, the subscribe command and event, the notifications on the scheduled Workflow Task event, and the bridge's command and activation job for them. The vendored tree, the payload visitor and the bridge client regenerate from the pin.
The protos pin carries ChannelSubscriptionInfo on the describe response and the UnsubscribeNotificationChannel command, event and failed cause. The pin refuses the lang command the way it refuses the subscribe, so the layers above pin the delivery Core before a workflow can unsubscribe.
A standalone activity can own a channel too, so the five channel requests
and both `linked_to` fields carry `common.v1.Execution` in place of
`WorkflowExecution`, with the old numbers reserved.
The previous shape was never released, so the five channel requests and both
`linked_to` fields reuse the numbers `workflow_execution` and the old
`linked_to` had, with nothing reserved. Names and types are unchanged.
@moedash

moedash commented Oct 3, 2026

Copy link
Copy Markdown
Owner Author

Replaced by #19, #20, #21, #22, #23, #24, #25, #26, #27, #28, #29, #30, #31, #32, #33, #34, #35, #36, #37, #38, #39, #40, #41, #42, #43, #44, #45, #46, #47, #48, #49, #50, #51, #52, #53, #54 and #55.

Same content, split into 37 PRs in the v3 series: the notification channel first, then the streaming interface, then native streams and the rest. The branch stays as a pin.

@moedash moedash closed this Oct 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

skip-changelog Changelog entry rides another PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant