Skip to content

fix(sdk): load state with unlisted subtypes - #762

Draft
ParidelPooya wants to merge 1 commit into
mainfrom
fix/accept-unknown-operation-subtype
Draft

ParidelPooya wants to merge 1 commit into
mainfrom
fix/accept-unknown-operation-subtype

Conversation

@ParidelPooya

Copy link
Copy Markdown

Part of #749. This is the prerequisite that the issue lists as step 2: the SDK must load a subtype that OperationSubType does not list, before any code records a custom subtype.

The defect

  1. The SDK parses every operation in the execution state on each invocation, with Operation.from_dict.
  2. Operation.from_dict and OperationUpdate.from_dict call OperationSubType(data["SubType"]).
  3. OperationSubType is a closed enum. An unlisted value raises ValueError.
  4. So one operation with an unlisted subtype fails the handler before user code runs. This happens even when the code never reads that operation.

Two sources can record such a subtype: another SDK version, and a library built on the SDK. Once #749 adds custom subtypes, a rollback to an SDK version without this fix would fail every execution that recorded one.

The change

OperationSubType._missing_ returns a member for any non-empty string:

  • A listed value returns its listed member, as before.
  • Any other string returns a member that the enum creates on the first lookup. The member is stored in _value2member_map_, so every later lookup returns the same object.
  • .value returns the recorded string. to_dict therefore writes the subtype back unchanged.
  • The created members do not appear in iteration or in __members__.
  • A value that is not a non-empty string still raises ValueError.

No other source file changes. Parsing, serialization, and the replay identity check work unchanged, because they already use OperationSubType(...), .value, and is.

Why a member, not a plain str

#749 proposed keeping an unlisted subtype as a str. This PR does not, for one reason:

  1. The otel plugin, the insight plugin, and the testing package all read sub_type.value.
  2. A plugin written by a customer can do the same, because sub_type is typed OperationSubType | None in the plugin info objects.
  3. A str has no .value. So a str would raise AttributeError in each of these places.
  4. A member keeps .value, is, and the declared type correct for every consumer.

The follow-up PR for #749 can convert a caller's sub_type="MySubtype" with OperationSubType("MySubtype"). So the public config can accept a str while the rest of the SDK keeps one type.

Tests

  • lambda_service_test.py: the lookup of an unlisted string, rejection of "", None, and 3, and round-trips through Operation.from_dict and OperationUpdate.from_dict.
  • execution_test.py: two tests that pass a raw invocation event.
    • State with an operation that has an unlisted subtype loads, and the execution succeeds.
    • A replayed child context whose checkpoint has an unlisted subtype fails with NonDeterministicExecutionError. The message names the checkpoint's subtype.
  • model_test.py in the testing package: test_events_to_operations_invalid_sub_type asserted that an unlisted subtype raises. It now asserts that the subtype loads. The testing package source is unchanged. Its try/except ValueError stays, because the testing package accepts core SDK versions from 1.0.0, and those versions still raise.

Without the fix, the new core tests fail with ValueError: 'PyTest...' is not a valid OperationSubType. In the execution tests, the handler itself raises.

Verified

  • The cached-member approach on Python 3.11.6, 3.12.5, 3.13.12, and 3.14.6: lookup identity, isinstance, iteration, __members__, pickle, deepcopy, repr, hashing, and 50 threads that look up the same string at once and all get one object.
  • Core SDK: 1,773 tests pass, coverage 98.27%, mypy and ruff clean.
  • Testing package: 1,596 tests pass, mypy and ruff clean.
  • The otel and insight suites were not run, because neither package changes.

- OperationSubType(value) returns a member for any non-empty string.
  A listed value returns its listed member. Any other string returns
  a member that the enum creates once and reuses, so .value returns
  the recorded string and `is` still compares equal lookups.
- Operation.from_dict and OperationUpdate.from_dict therefore load an
  operation whose subtype the enum does not list.

The SDK parses every operation in the state on each invocation. A
subtype recorded by another SDK version, or by a library on top of the
SDK, raised ValueError during that parse. The handler then failed
before user code ran, even when the code never read that operation.
This is a prerequisite for custom subtypes (#749).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant