Hardening: typed transient failures + required Deployer.exists()

[jogrogan]

Decouple Proteus DDL

Hoptimator's validation and deployment logic is currently reachable only by issuing SQL DDL through a Calcite/JDBC connection. That couples every caller to Calcite and forces schema definitions through a lossy  Avro → RelDataType → Avro  round-trip. Hoptimator via a CLI or over REST/gRPC may want to create/validate tables by handing us a table name + a schema directly — not by constructing DDL.

This stacked series decouples the validators and deployers from Calcite so the same validation and deployment engine can be driven either by SQL or by a direct, connection-free call — while keeping validators/deployers as a clean SPI. The payoff: an API call to "create table from a schema" path with identical guarantees to DDL (including dry-run), lossless Avro fidelity, and no Calcite dependency for callers that don't want one. It also keeps the door open for follow-ups (e.g. materializations, source→sink field mappings) without re-plumbing.

Stacked series (review/merge in order):

1.  decouple-pr1-spi  — SPI foundation:  java.sql.Connection  →  DeploymentContext  (pure decoupling, no behavior change)
2.  decouple-pr2-api  — SQL-free  TableService  + lossless Avro preservation
3.  decouple-pr3-hardening  — typed transient failures + required  Deployer.exists() 
   (This is PR 3 of 3.)

PR 3/3 — Hardening: typed transient failures + required Deployer.exists()

Base: jogrogan/decouple-pr2-api · Branch: jogrogan/decouple-pr3-hardening
Stacked series: 3 of 3 (PR1 SPI foundation → PR2 direct table API → this)

Review/merge after PR2. The diff below is against PR2.

Why

PR1 and PR2 deliver the SPI decoupling and the SQL-free TableService. This final PR
hardens that surface so it behaves correctly under real-world failure modes and so the
direct-path contract is enforced rather than assumed.

What changes

1. Typed transient failures across the SPI

• K8s failures are now classified by retryability instead of being blanket-treated as
  transient. K8sApi/K8sYamlApi normalize connectivity failures to a typed
  SQLTransientException, and genuinely non-retryable failures surface as
  SQLNonTransientException.
• These typed exceptions propagate through the deploy/validate path
  (ValidationService, DeploymentService, ConnectionService) so callers can
  distinguish "retry me" from "this will never succeed."
• Database-list failures are surfaced instead of being swallowed into an empty
  "no databases" result.

2. Deployer.exists() is now an abstract SPI method

• Previously an "assume-absent" default masked re-creates. exists() is now abstract, and
  the three real implementors (K8s, Venice, and the logical/materialized-view deployers)
  are backfilled with genuine existence checks.
• Consequence: a direct-path create against an incompatible existing table (e.g. a
  different schema, updateIfExists=false) is rejected with a clear
  SQLNonTransientException("... already exists") rather than silently masked.

Testing

• New/expanded unit tests: K8sApiTest, K8sYamlApiTest, K8sUtilsTest,
  K8sPipelineBundleTest, K8sMaterializedViewDeployerTest, ValidationServiceTest,
  DeploymentServiceTest, ConnectionServiceTest, and the per-deployer exists()
  backfills (Venice, MySQL, Logical, provider tests).
• Integration tests updated to assert the typed-exception contract (e.g. Venice
  createWithoutUpdateIfExistsFailsWhenStoreExists now expects SQLNonTransientException).

Review notes

• ~44 files. Two independent themes (typed transient failures; required exists()); both
  are additive/behavioral hardening on top of PR2's API.
• Does not re-touch PR2's TableService/DirectDeploymentContext core or the Venice
  test's resolveVenice helper — the only change to VeniceTableServiceIntegrationTest
  here is the exception-type assertion, a different hunk than PR2's resolution change.

[github-actions]

Code Coverage

Overall Project	84.73% -0.1%	🟢
Files changed	92.76%	🟢

File	Coverage
MySqlDeployerProvider.java	100%	🟢
ConnectionService.java	100%	🟢
ValidationService.java	100%	🟢
DirectDeploymentContext.java	100%	🟢
CalciteDeploymentContext.java	100%	🟢
K8sUtils.java	100%	🟢
K8sYamlApi.java	100%	🟢
VeniceDeployerProvider.java	100%	🟢
LogicalTableDeployerProvider.java	100%	🟢
KafkaDeployerProvider.java	100%	🟢
K8sMaterializedViewDeployer.java	96.03%	🟢
DeploymentService.java	94.67%	🟢
K8sDatabaseConfigResolver.java	92.15%	🟢
K8sApi.java	91.54% -5.81%	🟢
K8sPipelineBundle.java	88.15%	🟢
HoptimatorDdlUtils.java	87.21%	🟢
LogicalTableDeployer.java	74.47%	🟢