Skip to content

test: relax child context replay time thresholds - #660

Merged
zhongkechen merged 1 commit into
mainfrom
codex/relax-child-context-replay-threshold
Aug 26, 2026
Merged

test: relax child context replay time thresholds#660
zhongkechen merged 1 commit into
mainfrom
codex/relax-child-context-replay-threshold

Conversation

@zhongkechen

@zhongkechen zhongkechen commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Issue Link, if available

Failing E2E job: https://github.com/aws/aws-durable-execution-sdk-java/actions/runs/33020522022/job/98349518389

Description

The Java 21 E2E job completed all 500 child contexts successfully, but the best replay time across three attempts was exactly 20 ms. The test required the value to be strictly less than 20 ms, so the performance assertion failed even though the result and operation-tracking assertions passed.

Raise the replay-time budgets for 100 and 500 child contexts from 10/20 ms to 20/30 ms. These values match the neighboring async-step performance tests and cover timing variation already observed across Java 21 and Java 25 runs.

Demo/Screenshots

Not applicable; this is a test-threshold adjustment.

Checklist

  • I have filled out every section of the PR template
  • I have thoroughly tested this change

Testing

  • mvn spotless:apply
  • mvn -pl examples -am test -Dtest=ManyAsyncChildContextExampleTest -Dsurefire.failIfNoSpecifiedTests=false
    • Tests run: 3, Failures: 0, Errors: 0, Skipped: 0

Unit Tests

No new unit tests were needed because the change only adjusts existing cloud performance-test inputs. The focused local example test suite passed.

Integration Tests

No new integration tests were added. The affected cloud E2E test will run in this PR's workflow matrix.

Examples

No new example was added; the existing ManyAsyncChildContextExample is unchanged.

@zhongkechen
zhongkechen requested a review from a team August 26, 2026 23:21
@zhongkechen
zhongkechen temporarily deployed to ai-pr-review-runtime August 26, 2026 23:21 — with GitHub Actions Inactive
@zhongkechen
zhongkechen temporarily deployed to ai-pr-review-runtime August 26, 2026 23:21 — with GitHub Actions Inactive
@zhongkechen
zhongkechen temporarily deployed to ai-pr-review-runtime August 26, 2026 23:21 — with GitHub Actions Inactive
@zhongkechen zhongkechen self-assigned this Aug 26, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Codex AI review

No actionable findings. The PR only relaxes cloud replay-time thresholds to match neighboring performance tests. Residual risk is limited to normal environment-dependent timing variability.

Reviewed commit e4b272a80006ee8e80150c2f3284073e1eafddfa. Workflow run

@github-actions

Copy link
Copy Markdown
Contributor

Claude AI review

This PR is a minimal, test-only change: it raises the replay-time budgets in testManyAsyncChildContextExample (a cloud-only, -Dtest.cloud.enabled=true-gated performance test in CloudBasedIntegrationTest) from 10/20 ms to 20/30 ms for the 100/500 child-context cases, matching thresholds already used by the neighboring testManyAsyncStepsVirtualThreadExample test. No production code, public API, checkpoint/replay logic, serialization, or concurrency behavior is touched.

Verified:

  • @CsvSource arity/types still match the method signature (int steps, long maxExecutionTime, long maxReplayTime).
  • The test class is gated behind @EnabledIf("isEnabled") (test.cloud.enabled=true), so this does not affect default mvn test runs, consistent with AGENTS.md guidance that cloud tests are opt-in.
  • The new thresholds match the PR's stated rationale (observed ~20ms best replay time for 500 child contexts across 3 attempts) and are consistent with sibling test thresholds in the same file.
  • Per AGENTS.md/CONTRIBUTING.md testing requirements, no new unit/integration tests are needed for a threshold-only tuning of an existing cloud E2E test.

No correctness, API compatibility, replay-semantics, concurrency, or logging concerns apply.

Residual test risk: widening the replay-time budget by 10 ms in both cases slightly reduces this test's sensitivity to small, incremental replay-performance regressions in child contexts, since regressions that stay under the new looser ceiling would no longer be caught. This is an acceptable, deliberate trade-off given the observed timing flakiness at the tighter bound (examples/src/test/java/software/amazon/lambda/durable/examples/CloudBasedIntegrationTest.java:635).

Reviewed commit e4b272a80006ee8e80150c2f3284073e1eafddfa. Workflow run

@zhongkechen
zhongkechen merged commit 39d11d9 into main Aug 26, 2026
28 checks passed
@zhongkechen
zhongkechen deleted the codex/relax-child-context-replay-threshold branch August 26, 2026 23:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants