NexusSerializationContext for data/failure converters by JoshuaFrenchwood · Pull Request #1828 · temporalio/sdk-python

JoshuaFrenchwood · 2026-09-09T20:07:39Z

What was changed

Added NexusSerializationContext for nexus callers and sync handlers. This allows data and failure converters to use nexus endpoint, service and operation to be used for encoding and decoding.

Why?

This enables codecs to select serialization behavior or encryption keys by Nexus endpoint, service, or operation.

For example, workflows calling two Nexus endpoints can encrypt each endpoint’s payloads with a different key while ensuring that inputs, results, and failures are decoded with the converter selected for the corresponding operation.

Checklist

Closes
How was this tested:

Added unit/functional tests to verify that NexusSerializationContext works as expected.

Any docs updates needed?

tconley1428 · 2026-09-09T22:37:37Z

                self._workflow_context_failure_converter,
            )
+            summary_payload_converter = payload_converter
+            failure_converter = self._context_free_failure_converter


We should have a todo here. @VegetarianOrc already needs to extend this for worker callbacks.

Added a TODO for this

Worker callbacks are only delivered as Nexus tasks, and won't affect start nexus operation commands from a workflow.

Aside from that, is there a reason why we aren't using user_failure_converter here?

tconley1428 · 2026-09-11T18:47:08Z

    run_id: str | None
    rpc_metadata: Mapping[str, str | bytes]
    rpc_timeout: timedelta | None
-    result_type: type[Any] | None


This and the below are breaking changes that would need to be called out explicitly.

Added this back

tconley1428 · 2026-09-11T18:48:53Z

    async def get_nexus_operation_result(
        self, input: GetNexusOperationResultInput
-    ) -> Any:
+    ) -> temporalio.api.workflowservice.v1.PollNexusOperationExecutionResponse:


This appears to be the only proto being exposed in the whole interceptor, it probably shouldn't be.

It seems like the most similar thing to do like other ones would be a new object like WorkerTaskReachability or NexusOperationExecutionDescription, the latter of which similarly contains a raw proto and a dataconverter.

Created a new object called GetNexusOperationResultOutput for this. It includes raw_result, raw_failure, and data_converter.

VegetarianOrc · 2026-09-11T21:04:50Z

+        )
        return await NexusOperationExecutionDescription._from_execution_info(
            info=resp.info,
            data_converter=self._client.data_converter,


Is there a reason that this dataconverter doesn't get the serialization context?

Updated, before operation wasnt a field in GetNexusOperationResultInput, but since I added that it can create the serialization context

VegetarianOrc · 2026-09-11T21:12:10Z

        cls,
        info: temporalio.api.nexus.v1.NexusOperationExecutionInfo,
        data_converter: temporalio.converter.DataConverter,
+        failure_data_converter: temporalio.converter.DataConverter | None = None,


I'm not sure this is necessary. I think data_converter should get the context and be used for failure conversion and elsewhere.

Removed that, thanks

VegetarianOrc · 2026-09-11T21:12:43Z

        result_type: type | None = None,
        endpoint: str = "",
        service: str = "",
+        _nexus_serialization_context: (


Suggested change

_nexus_serialization_context: (

nexus_serialization_context: (

Removed this field

VegetarianOrc · 2026-09-11T21:47:17Z

+    _data_converter: DataConverter = field(repr=False, compare=False)
+
+
+@dataclass
+class GetNexusOperationResultOutput:
+    """Output for :py:meth:`OutboundInterceptor.get_nexus_operation_result`.
+
+    .. warning::
+        This API is experimental and unstable.
+    """
+
+    raw_result: temporalio.api.common.v1.Payload | None
+    """Raw result payload if the operation succeeded."""
+
+    raw_failure: temporalio.api.failure.v1.Failure | None
+    """Raw failure if the operation failed."""
+
+    data_converter: DataConverter = field(repr=False, compare=False)
+    """Data converter for decoding the result or failure."""


I think we should consider adding endpoint, service and operation to GetNexusOperationResultInput. Then the serialization context can be created in the terminal interceptor and decoding can happen as it did previously.

Threading the data converter or context itself through and having this output type feels a bit off to me. Happy to discuss though!

I like that approach, I updated it to have all three, endpoint/service/operation.

VegetarianOrc · 2026-09-11T21:51:04Z

+    The context is not propagated to the eventual result of an asynchronous operation. Standalone
+    operation handles use the context of their start request, including when an existing operation
+    is returned, while handles created without starting an operation do not receive it.


The first sentence here seems to belong with the section above and the part about standalone operation handles is a little hard to parse.

Update this doc, tried to copy over what we had in the go-sdk

VegetarianOrc · 2026-09-11T21:52:00Z

+    Callers and handlers receive this context on opposite sides of failure conversion. Contextual
+    encodings should therefore be self-describing and support legacy payloads without context.


Maybe I'm just not seeing it, but I'm not sure what "opposite sides of failure conversion" means exactly.

VegetarianOrc · 2026-09-11T21:56:43Z

+        data_converter = data_converter or self._data_converter_for_nexus_task(
+            endpoint, start_request.service, start_request.operation
+        )


IMO we should either make this always use self._data_converter_for_nexus_task and remove the argument or make the argument required.

Made it required.

VegetarianOrc · 2026-09-11T22:01:38Z

                self._workflow_context_failure_converter,
            )
+            summary_payload_converter = payload_converter
+            failure_converter = self._context_free_failure_converter


Worker callbacks are only delivered as Nexus tasks, and won't affect start nexus operation commands from a workflow.

Aside from that, is there a reason why we aren't using user_failure_converter here?

VegetarianOrc · 2026-09-11T22:16:19Z

+    HMAC_ENCODING = b"binary/nexus-context-hmac"
+    ZLIB_ENCODING = b"binary/nexus-context-zlib"
+    HMAC_KEY = b"nexus-context-test-key"


Smallish nit: Seems like we could probably prove the context usage w/o using these various encodings and using a simpler marker or something.

Updated this to be a simpler red/blue test for two different nexus endpoint

JoshuaFrenchwood requested review from Evanthx, Quinn-With-Two-Ns, VegetarianOrc and atol September 9, 2026 20:07

JoshuaFrenchwood force-pushed the adding-nexus-serialization-context branch from f36df7a to 0774bf3 Compare September 9, 2026 20:09

JoshuaFrenchwood marked this pull request as ready for review September 9, 2026 20:09

JoshuaFrenchwood requested a review from a team as a code owner September 9, 2026 20:09

JoshuaFrenchwood force-pushed the adding-nexus-serialization-context branch from 0774bf3 to 21ee938 Compare September 9, 2026 20:16

tconley1428 reviewed Sep 9, 2026

View reviewed changes

Comment thread temporalio/worker/_workflow_instance.py Outdated

tconley1428 reviewed Sep 9, 2026

View reviewed changes

Comment thread temporalio/worker/_nexus.py Outdated

tconley1428 reviewed Sep 9, 2026

View reviewed changes

Comment thread temporalio/client/_interceptor.py Outdated

tconley1428 reviewed Sep 9, 2026

View reviewed changes

Comment thread temporalio/client/_impl.py

JoshuaFrenchwood requested a review from tconley1428 September 10, 2026 21:50

JoshuaFrenchwood force-pushed the adding-nexus-serialization-context branch 3 times, most recently from c91cfa3 to b98ea57 Compare September 11, 2026 15:02

JoshuaFrenchwood added 3 commits September 11, 2026 13:30

NexusSerializationContext for data/failure converters

72e6f1e

Updating the nexus context for summary

4e8e392

Moving decoding logic to the nexus operation handle

117a7fd

JoshuaFrenchwood force-pushed the adding-nexus-serialization-context branch from b98ea57 to 117a7fd Compare September 11, 2026 18:30

tconley1428 reviewed Sep 11, 2026

View reviewed changes

Adding GetNexusOperationResultOutput for the interceptor

d756048

JoshuaFrenchwood requested a review from tconley1428 September 11, 2026 20:39

VegetarianOrc reviewed Sep 11, 2026

View reviewed changes

Refine Nexus serialization context propagation

15aad9a

JoshuaFrenchwood requested a review from VegetarianOrc September 11, 2026 23:48

	_nexus_serialization_context: (
	nexus_serialization_context: (

		Callers and handlers receive this context on opposite sides of failure conversion. Contextual
		encodings should therefore be self-describing and support legacy payloads without context.

Uh oh!

Conversation

JoshuaFrenchwood commented Sep 9, 2026

What was changed

Why?

Checklist

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants