Skip to content

feat(platform): verify Spark Iceberg REST interoperability - #23

Open
zhouning wants to merge 1 commit into
feat/ar1-metadata-fabric-jdbc-catalog-restartfrom
feat/ar1-metadata-fabric-spark-iceberg-rest-interoperability
Open

feat(platform): verify Spark Iceberg REST interoperability#23
zhouning wants to merge 1 commit into
feat/ar1-metadata-fabric-jdbc-catalog-restartfrom
feat/ar1-metadata-fabric-spark-iceberg-rest-interoperability

Conversation

@zhouning

Copy link
Copy Markdown
Owner

Summary

  • run Spark 3.5 through the Gravitino Iceberg REST 1.11 server against the same JDBC catalog and local warehouse PVC used by the authenticated Gravitino API
  • verify deterministic read/write, schema evolution, two snapshots, time travel, Gravitino metadata readback, bounded 403 denial, image identity, same-node PVC scope, and complete cleanup
  • add a fail-closed profile, Kubernetes rehearsal, checked evidence, ADR-055, runtime inventory, roadmap/System-of-Record updates, and CI validation

Verification

  • live Docker Desktop rehearsal: local_spark_iceberg_rest_interoperability_verified=true
  • contract fingerprint: a78b95d36a6a5d5f4b5e303be21263d00fdd7102c3a70ce282ba69f2d8cdcd2e
  • evidence fingerprint: 50f9d0021db11e22364697d1ad8928ee068d28dc8046556bbca1a4e1c819f8e0
  • required suite: 692 passed
  • focused M3-9 and platform truth: 28 passed
  • evidence verify/validate and kubectl kustomize: passed

Claim boundary

This is local same-node interoperability using Basic-authenticated Gravitino, unauthenticated HTTP Iceberg REST, a file warehouse, and one shared RWO PVC. Spark cancellation/reconcile/lineage, Flink, protected identity, OIDC/TLS, object-store durability, production ingestion, spark_conformance_verified, and production_ready remain false.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant