Skip to content

fix(sdk): checkpoint the wrapped error for failed contexts - #816

Closed
ParidelPooya wants to merge 2 commits into
mainfrom
fix/checkpoint-wrapped-error
Closed

ParidelPooya wants to merge 2 commits into
mainfrom
fix/checkpoint-wrapped-error

Conversation

@ParidelPooya

@ParidelPooya ParidelPooya commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Problem

When an SDK operation exception escapes a child context, map item or parallel branch, ChildContextOperation.handleChildContextFailure checkpoints the inner operation's raw ErrorObject. That is right for a wrapper such as StepFailedException, whose object is the user's original error, but it loses identity for SDK errors that name the failure themselves:

Inside runInChildContext Invoke/callback throws Context checkpoint ErrorType before
invoke stopped with States.Stopped InvokeStoppedException States.Stopped
invoke stopped with com.acme.PaymentError InvokeStoppedException com.acme.PaymentError
invoke timed out InvokeTimedOutException no error at all
callback timed out CallbackTimeoutException no error at all
callback failed with no type CallbackFailedException null type

So a reader of the context checkpoint (telemetry exporters, MapResult consumers, replay) cannot tell what failed the context. Telemetry spec v3 ("Core SDK requirement") requires every failed context to checkpoint the wrapped error, not the wrapper.

Change

ChildContextOperation.wrappedError picks the error to checkpoint:

  • CallbackTimeoutException, InvokeTimedOutException, InvokeStoppedException, and a CallbackFailedException or InvokeFailedException whose error has no (or a blank) type: an ErrorObject with ErrorType = the exception's class name, keeping the original message, ErrorData and stack trace when present. For these the raw object is missing or is the stop error, so it does not say what failed the context.
  • Any other operation exception (a wrapper such as StepFailedException or a typed callback/invoke failure, a nested ChildContextFailedException, or a DurableOperationException thrown by user code, even with an empty or missing error object): its ErrorObject, unchanged, as before.
  • waitForCallback contexts keep the callback's own ErrorObject as before. Live, waitForCallback rethrows the callback's exception (whose error object is the callback's), and a replay of the completed, pruned context can read only the context checkpoint, so the two must hold the same object.

translateException no longer tries to rebuild an SDK operation exception from error data. It falls through to ChildContextFailedException / MapIterationFailedException / ParallelBranchFailedException as before.

User-visible changes

  • Context checkpoints (ContextDetails.error) of a context failed by a stopped or timed-out invoke, a timed-out callback, or a typeless callback/invoke failure now carry ErrorType = the SDK exception class name (for example software.amazon.lambda.durable.exception.InvokeStoppedException) instead of the stop error's type, null, or no error.
  • The ChildContextFailedException thrown for those contexts carries that ErrorObject, so its message is Child context failed with error of type ...InvokeStoppedException. Message: ... instead of ... of type States.Stopped ... or Child context failed without an error. Its class is unchanged, live and replayed.
  • The execution error of a handler that rethrows such a ChildContextFailedException carries that ErrorObject.
  • MapResult.getError(i) for such an item is unchanged (it still reports the thrown MapIterationFailedException). Only the iteration's checkpoint changes.
  • Unchanged: user errors, wrappers with a typed error, operation exceptions thrown by user code, every top-level operation, and waitForCallback.

Compatibility

  • Checkpoints written by older SDK versions still replay: their context errors are read as before.
  • Nothing is rebuilt from the new ErrorType values. They are SDK class names that translateException now skips, so replay does not try to deserialize an SDK exception.
  • Exception classes, MapResult and the public API are unchanged.

Tests

  • New ChildContextWrappedErrorIntegrationTest (LocalDurableTestRunner), each scenario live and replayed (the handler catches, waits for a new invocation, and the replay throws again):
    • child context checkpoint type and message for: step PaymentError; invoke failed with an external type; invoke stopped (States.Stopped, user type); invoke timed out; callback failed (typed, no type, "Error"); callback timed out. Live and replayed failures have the same class and error object.
    • nested contexts (outer > child > invoke) stopped and timed out: both contexts checkpoint the SDK error.
    • map item with a stopped invoke or timed-out callback: the iteration checkpoint carries the SDK error type.
    • waitForCallback: the context keeps the callback's own error object.
  • mvn -B install for sdk, sdk-testing, sdk-integration-tests and otel-plugin on JDK 17: 1589 / 53 / 741 / 916 tests passed, 0 failures (5 skipped, all already present). spotless:check passed. The existing PluginErrorContainerCompatibilityTest, WaitForCallbackPrunedReplayTest and otel VirtualOperationErrorStatusTest pass unchanged. The second commit narrows the first one so these pass.
  • The full reactor build is in fix(telemetry)!: align error names and log lines with JS #814, which builds on this branch.

Known, not changed here: after the backend prunes a completed waitForCallback context's children, its replay throws ChildContextFailedException while the live run throws CallbackFailedException / CallbackTimeoutException. This was already the case, and WaitForCallbackPrunedReplayTest asserts it. The error object is the same in both runs.

A context that failed because an SDK error escaped it checkpointed the
inner operation's raw ErrorObject. For a timed-out callback or invoke that
object is null, and for a stopped invoke it is the stop error (for example
States.Stopped), so the context checkpoint lost which error failed it.

The context now checkpoints the wrapped error: the original ErrorObject of
a wrapper such as StepFailedException when it has a type, otherwise an
ErrorObject typed with the SDK error's class name (CallbackTimeoutException,
InvokeTimedOutException, InvokeStoppedException, or the exception class
when the error has no type), keeping message, data and stack trace. SDK
operation exception types are not rebuilt from error data on translation.
waitForCallback keeps the callback's own ErrorObject, because it rethrows
the callback's exception live.
@ParidelPooya
ParidelPooya requested a review from a team October 22, 2026 16:13
@ParidelPooya
ParidelPooya deployed to ai-pr-review-runtime October 22, 2026 16:14 — with GitHub Actions Active
@ParidelPooya
ParidelPooya deployed to ai-pr-review-runtime October 22, 2026 16:14 — with GitHub Actions Active
Comment on lines +208 to +209
errorObject =
getSubType() == OperationSubType.WAIT_FOR_CALLBACK ? opEx.getErrorObject() : wrappedError(opEx);

This comment was marked as outdated.

@github-actions

This comment has been minimized.

Only an SDK error that names the failure itself (timed-out callback,
timed-out or stopped invoke) and a CallbackFailedException or
InvokeFailedException whose error has no type are checkpointed under
their class name. Any other DurableOperationException that escapes a
context, such as one a user throws with an empty or missing error
object, keeps that object, as the OpenTelemetry plugin's
VirtualOperationErrorStatusTest requires.
@ParidelPooya
ParidelPooya deployed to ai-pr-review-runtime October 22, 2026 16:24 — with GitHub Actions Active
Comment on lines +209 to +210
errorObject =
getSubType() == OperationSubType.WAIT_FOR_CALLBACK ? opEx.getErrorObject() : wrappedError(opEx);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Codex AI review · Finding arf_v1_kxt4ac5ayulxw4xmu4ktdps4xu

[P2] Preserve callback error identity through enclosing contexts

When a virtual child context calls waitForCallback and the callback times out, the enclosing context now throws ChildContextFailedException with an error object typed CallbackTimeoutException. If the handler catches this and waits, replay prunes the completed wait's children. The wait then throws ChildContextFailedException with a null error object, which wrappedError preserves, so the enclosing failure changes from a typed error to null. Previously, both runs exposed null. This can change error-dependent branches and durable operation sequences during replay. Extend the preservation rule to failures escaping waitForCallback into enclosing contexts, using the persisted wait context's error consistently, and add a virtual-child live/pruned-replay regression test.

To suppress repeats, reply /ai verdict deferred, /ai verdict false-positive, or /ai verdict out-of-scope on the first nonblank line. Add rationale on later lines. A new /ai verdict accepted allows future repeats again.

@github-actions

Copy link
Copy Markdown
Contributor

Codex AI review

One P2 replay-consistency regression remains at sdk/src/main/java/software/amazon/lambda/durable/operation/ChildContextOperation.java:210. Static review only; tests were not executed.

Reviewed commit 8dde1e75606b29601b642f6884d82794498757bb. Workflow run

@ParidelPooya

Copy link
Copy Markdown
Contributor Author

Closing in favour of a simpler approach. We no longer try to make error.name identical across SDKs, and the core SDKs stay unchanged. Each exporter will instead write error and its cause in one shared shape, with each language keeping its own type names; the READMEs will document the differences. A new, smaller exporter-only PR replaces this one. The branch is kept, so this can be reopened if needed.

This branch was successfully deployed

1 active deployment
ai-pr-review-runtime — 8dde1e75 Deployed Oct 22, 2026 by ParidelPooya via ai-pr-review / Codex review / Generate Codex review #2624
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant