Skip to content

Merged the current Core into the external stream branch. - #3

Closed
moedash wants to merge 106 commits into
moe/AI-198-external-core-repairsfrom
moe/AI-198-core-unified
Closed

moedash wants to merge 106 commits into
moe/AI-198-external-core-repairsfrom
moe/AI-198-core-unified

Conversation

@moedash

@moedash moedash commented Sep 16, 2026 •

Copy link
Copy Markdown
Owner

This PR merges current upstream main into the external stream branch and fixes the call sites it broke.

What changed?

  • The upstream commits in the list are upstream's own. The review here is the merge resolution and the few commits on top.
  • managed_run.rs follows upstream's WftFailureKind reporting and its renamed auto_reply_fail field, and keeps the external stream state.
  • Two seams in the external stream code: add_cmd_to_wf_task takes its annotations by value, for the lang-issued channel commands and the ones Core issues on its own, and the arm that rejects a stray external stream command names the command in its fatal!.
  • A note at the WorkflowStreamProgress ordering rule says why workflow command pagination can't move the marker. Pages carry the same command list in order, the server buffers only the intermediate pages, and the final page merges them into one commit.
  • Carried the external stream task and replay wake fixes. #2 is merged forward: the stream and notification protos, the channel calls on the client and in the C bridge, and Core resuming parked external stream waits from channel notifications.
  • From Carried the external stream task and replay wake fixes. #2 as well, the WorkflowStreamChannels report, so lang names the channels a run listens on and Core defers the subscribe to the completion that ends the Workflow Task.
  • From Carried the external stream task and replay wake fixes. #2 as well, the UnsubscribeNotificationChannel command and its machine, issued for a channel that left the reported set ahead of any subscribe on the same completion.
  • From Carried the external stream task and replay wake fixes. #2 as well, the vendored unsubscribe command, event and failed cause, and ChannelSubscriptionInfo with DescribeWorkflowExecutionResponse.channel_subscriptions.

This layer holds the external lineage plus upstream only. #4 adds the native stream commits.

Part of AI-198 (epic AI-37).

Why?

The two prototype Cores sat weeks apart on main, so combining them meant reconciling everything at once. After this merge, stream code is the only difference left between them. The Python bridge also needs WorkerDeploymentVersion::builder from the newer base to compile.

How did you test it?

The full cargo suite ran on the branch, and the sdk-core unit suite, cargo fmt --check and cargo test-lint ran again on the head that carries the deferred subscribe and the unsubscribe. The bridge compile check ran from sdk-python against the descendant branch. No live lane runs against this head on its own, since the Python SDK pins #4.

  • Unit Tests
  • Staging
  • End to End Tests

maciejdudko and others added 30 commits August 14, 2026 11:13
* Prefactor activity execution traits

* Prefactor ephemeral server APIs

* Add SDK test environments

* Address SDK testing review feedback

* self review

* self review: better heartbeating

* fix merge

* fixup merge conflict

* pr feedback

* rename

* add back error msg part

* pr feedback

* support saa
* chore(sdk): release 0.7.0

* Apply suggestions from code review

Co-authored-by: Chris Olszewski <chrisdolszewski@gmail.com>
* feat(client): allow setting memo on workflow start

`WorkflowStartOptions` had no memo field, so there was no way to attach a
memo when starting a workflow from the client. The read side and the
workflow side already exist - continue-as-new can set one, and
`WorkflowExecutionDescription::memo()` reads one back - it was only the
client start path that was missing.

Adds `memo: Option<ProtoMemo>` and wires it into both
`StartWorkflowExecutionRequest` and `SignalWithStartWorkflowExecutionRequest`.

Uses the proto type directly, matching the existing `header: Option<Header>`
field. Both are `map<string, Payload>` and the caller has to convert values
either way. Imported as `ProtoMemo` so it doesn't collide with the
re-exported `temporalio_common::Memo`, which is the read-side wrapper.

Two integ tests in workflow_client_tests.rs: one that a memo set at start
comes back through describe, one that it's empty when unset.

* feat(client): use MemoValues for memo on workflow start

Per review, the client shouldn't ask callers to build the proto Memo
themselves. It now takes `MemoValues`, the same type the workflow surface
already uses for continue-as-new and `upsert_memo`.

`MemoValue`/`MemoValues` move out of `temporalio_workflow` and into
common-wasm, next to the existing `Memo` read wrapper. That's where they
have to go for both sides to reach them... `temporalio_workflow` depends on
common-wasm, not common. Re-exported from `temporalio_common`,
`temporalio_workflow` and `temporalio_sdk`, so existing imports still work.

`MemoValue` holds its value in an `Arc` now instead of an `Rc`, and wants
`Send + Sync`. `WorkflowStartOptions` travels through the interceptor chain
inside a `Send` future and an `Rc` can't cross that. Same shape as
`DataConverter`, which already erases its converters as
`Arc<dyn ... + Send + Sync>`.

Also runs the memo through the payload codec, not just the payload
converter. The read side already codec-decodes it... describe and list both
run `decode_payloads` over the memo. So converter-only encoding broke the
round trip for anyone with a real codec. Caught this after wiring up the
tests below, the two client tests fail without it.

Client unit tests cover start and signal-with-start under a codec, plus the
serialization failure path. Integ tests moved over to `MemoValues`.

* Address review feedback on memo

`MemoValue` implements `TemporalSerializable` instead of exposing a doc
hidden `to_payload`, so callers serialize it through the normal converter
path. It also picks up the caller's serialization context now rather than
hardcoding `Workflow`.

`MemoValues::encode` is gone in favour of documented `get`/`iter`. The
three callers build the payload map themselves. That is a few duplicated
lines each, but nothing doc hidden in the public API.

Test cleanups from the review... dropped the `memo_with` helper, the
`assert_ne!` on the raw bytes that the following assertion already covers,
and the empty-memo integ test that the client unit test covers.

* fixup changelog

---------

Co-authored-by: Chris Olszewski <chrisdolszewski@gmail.com>
* fix(sdk): typed signal_with_start_workflow

* fix: rebase memo fixes
…oralio#1511)

* Stabilize legacy query activation order

* Clarify legacy query ordering comment

SDK-Sentinel-Request: temporalio#1511/comment-5398459129
* Move changelog release notes helper to its own crate

* Select the changelog for release notes
* Allow including LA arguments in marker

* Expose option in Rust SDK
…oralio#1483)

* Add envconfig integration test mode

* Add isolated Cloud namespace helper

* Test envconfig harness in isolated Cloud namespace

* Clarify envconfig snapshot comment

* Move Cloud namespace commands into integration runner
…1525)

Updates the requirements on [base64](https://github.com/marshallpierce/rust-base64) to permit the latest version.
- [Changelog](https://github.com/marshallpierce/rust-base64/blob/master/RELEASE-NOTES.md)
- [Commits](marshallpierce/rust-base64@v0.22.0...v0.23.1)

---
updated-dependencies:
- dependency-name: base64
  dependency-version: 0.23.1
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…io#1536)

* chore(sdk): mark incoming error as non_exhaustive

* chore(sdk/envconfig): add new SDK owned envconfig structs instead of reexporting from core

* chore(sdk): mark schedule aspects with nonexhaustive

* chore(sdk): remove legacy ActExtiValue

* chore(sdk): move FailOnNondeterminismInterceptor to tests

* chore(sdk): non_exhaust data converter types

* chore(sdk): mark SerializationContext as non_exhaustive

* chore(sdk): make TaskToken bytes private

* chore(sdk): make ActivityCloseTimeouts non exhaustive

* chore(sdk): mark workflow execution info with non_exhaustive

* chore(sdk): WorkerCallbacks is nonexhaustive

* chore(sdk): make Priority and WorkflowDeploymentVersion nonexhaustive

* changelog fixups

* chore(sdk): mark envconfig as non_exhaustive

* fix lint

* fix docs

* .
yuandrew and others added 15 commits September 16, 2026 18:52
Cancelled and deadline-exceeded polls already retain their slots during
backoff. Keep their targets unchanged so transient failures do not leave
polling concurrency reduced until a later server scale-up.
Picks up the per-runner `timeout-minutes` fix so `macos-intel` integ tests get the 40 minutes the matrix asks for.
The same fix already arrived here with the Core merge, so this only keeps the branch a descendant of the repairs branch.
The unified branch carries the same vendored WIT, so consumers that run the newer nexgen need the placeholder declarations here too.
The Python bridge builds against
temporalio_common::WorkerDeploymentVersion::builder, which only exists
in the newer upstream Core.
…o#1608)

`temporalio-common`'s build script emits `payload_visitor_impl.rs` by iterating
`PayloadModel::payload_containing`, a `HashSet<String>`, and orders each message's
fields by iterating `oneof_fields`, a `HashMap<i32, _>`. Both use the default
`RandomState`, which is seeded per process, so the generated file holds the same
implementations in a different order on every build.

The file is a compile input, so its content is part of what a compilation cache
keys on. sccache therefore misses `temporalio-common` on every build, and — because
each miss changes the crate's own output — misses every crate downstream of it too.
In a workspace using the Rust SDK that is `temporalio-client`, `temporalio-sdk-core`
and `temporalio-sdk`, plus the user's own crates that link them.

Collect both into a `Vec` and sort before emitting, matching what the payload-limits
generator in the same file already does for its roots, generated names and oneof
indices.

Validation: five consecutive build-script runs against the patched tree produce
byte-identical `payload_visitor_impl.rs` and `payload_limits_impl.rs`; three runs
against the unpatched tree produce three different files. The generated content is
unchanged — sorting the unpatched output's lines yields the patched output.
* Update API definitions

* Fix API definition update

* Classify callback context as a blob

* Order callback context limit classification
Pages carry the same command list in order, only the intermediate ones
are buffered, and the final page merges them, so the completion still
reaches History as one commit.
# Conflicts:
#	crates/sdk-core/src/worker/workflow/managed_run.rs
@moedash moedash added the skip-changelog Changelog entry rides another PR label Sep 26, 2026
# Conflicts:
#	crates/sdk-core/src/protosext/mod.rs
# Conflicts:
#	crates/sdk-core/src/protosext/mod.rs
# Conflicts:
#	crates/sdk-core/src/worker/workflow/machines/workflow_machines.rs
Carries the vendored api change c239d35..4304fd8: the five channel requests take a common.v1.Execution and both linked_to fields become Execution. Core-3 7a0d774 applies the same patch, so this merge brings #3 to the main lineage's head without merging core-3 itself.
moedash added a commit that referenced this pull request Oct 3, 2026
Carries #2 7006981 through #3 093c55e: the five channel requests take a common.v1.Execution and both linked_to fields become Execution. The native stream protos are untouched.
Carries the vendored api change 4304fd8..071feb8: the five channel requests' execution fields and both linked_to fields reuse the old numbers and nothing is reserved. Core-3 fbc064b applies the same patch.
moedash added a commit that referenced this pull request Oct 3, 2026
Carries #2 677d468 through #3 2a516e1: the execution and linked_to fields reuse the old numbers and nothing is reserved. The native stream protos are untouched.
@moedash

moedash commented Oct 3, 2026

Copy link
Copy Markdown
Owner Author

Replaced by #8, #9, #10, #11, #12, #13, #14, #15, #16, #17, #18, #19, #20, #21, #22 and #23.

Same content, split into 16 PRs in the v3 series: the notification channel first, then the streaming interface, then native streams and the rest. The branch stays as a pin.

@moedash moedash closed this Oct 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

skip-changelog Changelog entry rides another PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.