Skip to content

feat(platform): verify Spark commit-failure recovery - #26

Open
zhouning wants to merge 1 commit into
feat/ar1-metadata-fabric-object-store-readiness-gatefrom
feat/ar1-metadata-fabric-spark-commit-failure-recovery
Open

feat(platform): verify Spark commit-failure recovery#26
zhouning wants to merge 1 commit into
feat/ar1-metadata-fabric-object-store-readiness-gatefrom
feat/ar1-metadata-fabric-spark-commit-failure-recovery

Conversation

@zhouning

Copy link
Copy Markdown
Owner

Summary

  • add a local Spark/Iceberg commit-failure rehearsal with a loopback REST proxy that injects HTTP 503 before provider forwarding
  • verify zero snapshot, row, and referenced-file drift after the failed call, then one exact visible commit after an explicit retry
  • verify direct MinIO inventory, Gravitino readback, cleanup, fail-closed evidence integrity, platform runtime registration, and CI coverage

Validation

  • M3-12 validator: passed with contract 6d8944ab80246dc65891aa81118cb8b73f7ecad699be9a2af5e62d8260c41002 and evidence 39571cdac1e4043bcfc2d03a73b2b12ff925210daf8ae36bc640b8cb14d89401
  • M3-12 tests: 23 passed
  • M3-10 regression: 17 passed
  • platform_truth tests: 11 passed
  • required platform suite: 764 passed
  • Kustomize, Python compile, shell syntax, JSON/YAML, and diff checks: passed
  • rehearsal namespace and its two persistent volumes: absent

Scope

This is local Docker Desktop evidence for a known pre-forward failure and one explicit retry. It does not prove uncertain commit reconciliation, network exactly-once behavior, production object storage, protected identity or TLS, cancel or lineage, Flink, full Spark conformance, or production readiness.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant