Skip to content

fix(amber): enable checkpoint serialization - #7996

Open
anthonychengit wants to merge 1 commit into
apache:mainfrom
anthonychengit:fix/checkpoint-serialization
Open

fix(amber): enable checkpoint serialization#7996
anthonychengit wants to merge 1 commit into
apache:mainfrom
anthonychengit:fix/checkpoint-serialization

Conversation

@anthonychengit

Copy link
Copy Markdown
Contributor

What changes were proposed in this PR?

Make CheckpointState implement Serializable, so the existing Kryo binding can persist worker checkpoints.

Before: finalize checkpoint → serialize CheckpointState → no matching binding → NotSerializableException

After: finalize checkpoint → Kryo binding → flush/close → restore the original checkpoint payload

The regression test uses a real worker write path, stores a non-empty Unicode payload, reads the record back from sequential storage, and asserts the returned size and restored value. Two existing write-branch tests now await successful completion instead of discarding failures.

Any related issues, documentation, discussions?

Closes #7907

How was this PR tested?

Added positive persistence/round-trip coverage while retaining the existing estimate-only and checkpoint-isolation cases.

$env:STORAGE_JDBC_URL='jdbc:postgresql://localhost:15432/texera_codex_jooq?currentSchema=texera_db,public'
$env:STORAGE_JDBC_USERNAME='postgres'
$env:STORAGE_JDBC_PASSWORD=''
sbt "WorkflowExecutionService/testOnly org.apache.texera.amber.engine.architecture.worker.promisehandlers.FinalizeCheckpointHandlerSpec"
sbt "scalafixAll --check" "scalafmtCheckAll"

Result: 6 tests passed; Scalafix and Scalafmt checks passed. The focused suite used an isolated embedded PostgreSQL database and port.

Was this PR authored or co-authored using generative AI tooling?

Generated-by: OpenAI Codex (GPT-5)

@github-actions

Copy link
Copy Markdown
Contributor

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

  • Contributors with relevant context: @Yicong-Huang, @aglinxinyuan
    You can notify them by mentioning @Yicong-Huang, @aglinxinyuan in a comment.

@Yicong-Huang Yicong-Huang added the release/v1.2 back porting to release/v1.2 label Aug 26, 2026
@github-actions
github-actions Bot requested a review from xuang7 August 26, 2026 18:17
@github-actions

Copy link
Copy Markdown
Contributor

Backport auto-label report

This fix: PR was checked against each actively-supported release branch. release/* labels drive the post-merge backport, so add or remove one to change where this fix lands.

Release branch Analysis
release/v1.2 Change detected on this branch — label added; this fix is queued to backport here. Requested review from @xuang7.

Auto-label run.

@github-actions

Copy link
Copy Markdown
Contributor

⚠️ Benchmark changes need a look

🟢 2 better · 🔴 4 worse · ⚪ 9 noise (<±5%) · 0 without baseline

Compared against main 444fc58 benchmarked on this same runner, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.

Dashboard · Run

config throughput MB/s latency max Δ latest / 7d
🟢 bs=10 sw=10 sl=64 375 0.229 25,561/33,383/33,383 us 🟢 -9.0% / 🔴 +141.2%
🔴 bs=100 sw=10 sl=64 767 0.468 125,793/196,040/196,040 us 🔴 +28.8% / 🔴 +109.3%
bs=1000 sw=10 sl=64 912 0.557 1,084,833/1,207,128/1,207,128 us ⚪ within ±5% / 🔴 +33.7%
Baseline details

Latest main 444fc58 from same runner

config metric PR latest main 7d avg Δ latest Δ 7d
bs=10 sw=10 sl=64 throughput 375 tuples/sec 373 tuples/sec 894.86 tuples/sec +0.5% -58.1%
bs=10 sw=10 sl=64 MB/s 0.229 MB/s 0.228 MB/s 0.546 MB/s +0.4% -58.1%
bs=10 sw=10 sl=64 p50 25,561 us 24,494 us 11,139 us +4.4% +129.5%
bs=10 sw=10 sl=64 p95 33,383 us 36,683 us 13,843 us -9.0% +141.2%
bs=10 sw=10 sl=64 p99 33,383 us 36,683 us 16,839 us -9.0% +98.2%
bs=100 sw=10 sl=64 throughput 767 tuples/sec 815 tuples/sec 1,166 tuples/sec -5.9% -34.2%
bs=100 sw=10 sl=64 MB/s 0.468 MB/s 0.498 MB/s 0.711 MB/s -6.0% -34.2%
bs=100 sw=10 sl=64 p50 125,793 us 121,321 us 87,509 us +3.7% +43.7%
bs=100 sw=10 sl=64 p95 196,040 us 152,247 us 93,675 us +28.8% +109.3%
bs=100 sw=10 sl=64 p99 196,040 us 152,247 us 102,153 us +28.8% +91.9%
bs=1000 sw=10 sl=64 throughput 912 tuples/sec 897 tuples/sec 1,198 tuples/sec +1.7% -23.9%
bs=1000 sw=10 sl=64 MB/s 0.557 MB/s 0.547 MB/s 0.731 MB/s +1.8% -23.8%
bs=1000 sw=10 sl=64 p50 1,084,833 us 1,118,932 us 859,766 us -3.0% +26.2%
bs=1000 sw=10 sl=64 p95 1,207,128 us 1,198,616 us 902,897 us +0.7% +33.7%
bs=1000 sw=10 sl=64 p99 1,207,128 us 1,198,616 us 937,957 us +0.7% +28.7%
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,533.62,200,128000,375,0.229,25560.62,33383.28,33383.28
1,100,10,64,20,2607.38,2000,1280000,767,0.468,125793.43,196040.00,196040.00
2,1000,10,64,20,21921.87,20000,12800000,912,0.557,1084833.39,1207128.03,1207128.03

@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.25%. Comparing base (5de9bf9) to head (a15f1ee).
⚠️ Report is 1 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #7996      +/-   ##
============================================
+ Coverage     93.21%   93.25%   +0.04%     
- Complexity     4664     4678      +14     
============================================
  Files          1175     1177       +2     
  Lines         47646    47685      +39     
  Branches       5319     5312       -7     
============================================
+ Hits          44413    44469      +56     
+ Misses         1752     1740      -12     
+ Partials       1481     1476       -5     
Flag Coverage Δ *Carryforward flag
access-control-service 81.00% <ø> (ø) Carriedforward from 5de9bf9
agent-service 99.32% <ø> (ø) Carriedforward from 5de9bf9
amber 89.43% <100.00%> (+0.12%) ⬆️
computing-unit-managing-service 73.67% <ø> (ø) Carriedforward from 5de9bf9
config-service 86.73% <ø> (ø) Carriedforward from 5de9bf9
file-service 86.70% <ø> (ø) Carriedforward from 5de9bf9
frontend 95.67% <ø> (ø) Carriedforward from 5de9bf9
notebook-migration-service 79.31% <ø> (ø) Carriedforward from 5de9bf9
pyamber 97.74% <ø> (ø) Carriedforward from 5de9bf9
workflow-compiling-service 77.19% <ø> (ø) Carriedforward from 5de9bf9

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@anthonychengit
anthonychengit marked this pull request as ready for review August 26, 2026 19:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

engine fix release/v1.2 back porting to release/v1.2

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Worker checkpoints are never persisted: CheckpointState has no serialization binding

3 participants