Skip to content

[views] Add /v1 views REST contract, controller, handler and validation - #694

Open
ruolin59 wants to merge 5 commits into
linkedin:mainfrom
ruolin59:rufan-linkedin-views-v2-api-contract
Open

[views] Add /v1 views REST contract, controller, handler and validation#694
ruolin59 wants to merge 5 commits into
linkedin:mainfrom
ruolin59:rufan-linkedin-views-v2-api-contract

Conversation

@ruolin59

@ruolin59 ruolin59 commented Aug 25, 2026

Copy link
Copy Markdown
Collaborator

Summary

Views: the whole /v2/databases/{databaseId}/views wire surface in one PR — models, routes, handler and structural validation.

Business logic is deliberately stubbed. No catalog, repository, HTS persistence, admission service or authorization enforcement lands here. ViewsService's only implementation returns 404 VIEWS_DISABLED, which matches the design's default-off posture: views ship disabled per database, and a route against a non-enabled database is specified to return 404 so Spark's ResolveViews fall-through to loadTable stays intact.

Views are a new resource, so mounting them under /v2 breaks no existing client. GET /v1/databases/{databaseId}/tables/{tableId} is unchanged.

What's here

Models (api/spec/v0)

  • CreateUpdateViewRequestBodyviewId, databaseId, clusterId, schema, representations, sourceDialect, defaultCatalog, defaultNamespace, viewProperties, baseViewVersion
  • ViewRepresentation component — {type, sql, dialect}, kept a list so Coral output can be added later without a wire change
  • GetViewResponseBody — pointer only; SQL, schema and history stay in the metadata file
  • GetAllViewsResponseBodyPage<GetViewResponseBody> populated with identifiers only, mirroring the existing table search convention
  • ViewDto + MapStruct mapper

RoutesViewsController: POST 201 · GET item 200 · PUT 200/201 · GET list 200 · DELETE 204. The controller holds no business logic; ViewsApiHandler delegates to ViewsService.

ValidationOpenHouseViewsApiValidator, structural only. No SQL is parsed, translated or validated against an engine. Covers identifier/cluster consistency, Iceberg SchemaParser round-trip, exactly one spark representation with a matching sourceDialect, reserved openhouse./policies key rejection, baseViewVersion shape, and SQL/schema UTF-8 payload limits (256 KiB / 512 KiB).

Error codes — the full 14-value ViewErrorCode enum ships now, including codes M1 never emits, so later milestones add behavior without an enum change.

Design decisions worth reviewer attention

Error codes select HTTP status only and are never serialized. ErrorResponseBody is untouched, so no other service's error shape moves. A new generic CodedApiException in services/common carries only HttpStatus — it has no view vocabulary, so services/common stays independent of services/tables.

Conventions were derived from the existing table flow rather than invented. baseViewVersion accepts absent-or-INITIAL_VERSION on POST because OpenHouseCatalog stamps INITIAL_TABLE_VERSION on create; the PUT token stays opaque because the table validator constrains baseTableVersion only to non-empty; reserved-key matching reuses HouseTableSerdeUtils.IS_OH_PREFIXED (case-sensitive) rather than reimplementing it; and list paging has no upper size cap because ApiValidatorUtil.validatePageable has none.

One deliberate deviation: path identifiers are length-checked at 128 chars, which OpenHouseTablesApiValidator does not do. Without it an over-long id returns a misleading 404 instead of 400. Flagged in the test javadoc; reverting is just the rule plus its test.

Tests

89 view tests; services/tables goes 523 → 585, services/common 9 → 18. Verified green across every module that depends on services/common: tables, housetables, jobs, internalcatalog, common — 865 tests, 0 failures.

ViewApiContractTest freezes the M1 wire surface, satisfying the acceptance criterion that adding the admission service, the polymorphic lookup or /versions later changes no field this ships. It pins declared field sets, Jackson-introspected property sets, and the exact serialized JSON key sets — including Spring's PageImpl shape. Both it and the error-status coverage were mutation-tested: adding a page-level field and swapping metadataLocation/viewVersion each turn the suite red.

Scope notes

  • GET /v1/.../tables/{tableId} is unchanged, and the "404 for a view name" criterion is satisfied vacuously. ViewsDisabledService is the only ViewsService implementation and every method throws, so no view can be created and no view row can exist — there is no input for which the v1 table route can encounter a view name. The included test covers the risk this PR does introduce: that ViewsController might shadow or collide with v1 table routing. It doesn't. The discriminator case only becomes reachable, and therefore testable, in M2 when entityType makes views persistable.
  • @Secured annotations declare the five view privileges, but AuthorizationInterceptor unconditionally allows today, so they are declarative only. The stable privilege names are the one-way door this PR closes.
  • Pre-existing and out of scope: ServiceAuditAspect records the full request payload into audit events, so view SQL can reach audit sinks even though exception messages are redacted. Worth tracking separately against the "SQL stored opaquely" one-way door.

@mkuchenbecker

Copy link
Copy Markdown
Contributor

Why is this V2 if we never had V1 support on these apis?

@ruolin59

Copy link
Copy Markdown
Collaborator Author

Why is this V2 if we never had V1 support on these apis?

We discussed this during the design meeting. Essentially we felt it was cleaner to make it something like "v1 doesn't support views, whereas v2 does".

@ruolin59
ruolin59 marked this pull request as ready for review August 26, 2026 16:35
@mkuchenbecker

Copy link
Copy Markdown
Contributor

We discussed this during the design meeting. Essentially we felt it was cleaner to make it something like "v1 doesn't support views, whereas v2 does".

I don't think that makes sense. V2 is not an API set but an indicator this is V2 of a particular API.

@ApiResponse(responseCode = "503", description = "View GET: SERVICE_UNAVAILABLE")
})
@GetMapping(
value = {"/v2/databases/{databaseId}/views/{viewId}"},

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All openhouse tables APIs are on V1 and and we kept V0 only for backward compatibility reasons. So the current version of OH APIs are on V1. We introduced V2 API for search to support pagination while keeping the existing url same. As we are introducing new APIs for view with separate controller layers and underlying layer we should start with V1 keeping views consistent with tables APIs. (cc: @mkuchenbecker )

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm, based on what you are saying, should the views API keep to V1 for the most part, but use V2 for the paginated search endpoint?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, and moved to /v1 in f31921a9.

Your framing is what settled it: /v2 on tables means version two of that specific endpoint, introduced so tables/search could add pagination while /v1/.../tables/search stayed live. Views has no /v1 list endpoint, so there was nothing for it to be version two of. We also only plan a paginated list for views, so there's no non-paginated /v1 variant to leave room for.

All five routes are now /v1/databases/{databaseId}/views[/{viewId}]. No collision with the existing tables, tables/search, aclPolicies or softDeletedTables segments.

Two things worth noting since tables and views now share the /v1/databases/{databaseId}/ prefix:

  • The audit redactor scopes by request URI, so its patterns moved with the routes. Its tests now also pin the old /v2 paths as non-matching so the redaction scope can't silently drift back.
  • The routing regression test was strengthened rather than renamed. It asserts on response bodies that a view-like table name reaches the tables handler, that a table literally named views still reaches the tables handler, and that both view routes reach the views handler.

909 tests, 0 failures.

ruolin59 and others added 2 commits August 26, 2026 10:59
Views M1 (BDP-108397): the whole /v2/databases/{databaseId}/views wire
surface -- models, routes, handler and structural validation. Business
logic is deliberately stubbed; no catalog, repository, HTS persistence
or admission work lands here.

Views are a new resource, so mounting them under /v2 breaks no existing
client, and GET /v1/databases/{databaseId}/tables/{tableId} is unchanged.

Models (api/spec/v0)
  - CreateUpdateViewRequestBody: viewId, databaseId, clusterId, schema,
    representations, sourceDialect, defaultCatalog, defaultNamespace,
    viewProperties, baseViewVersion.
  - ViewRepresentation component: {type, sql, dialect}.
  - GetViewResponseBody: pointer only -- SQL, schema and history stay in
    the metadata file.
  - GetAllViewsResponseBody: Page<GetViewResponseBody>, populated with
    identifiers only, mirroring the table search convention.
  - ViewDto and its MapStruct mapper.

Routes (ViewsController)
  POST 201 - GET item 200 - PUT 200/201 - GET list 200 - DELETE 204.
  The controller holds no business logic; ViewsApiHandler delegates to
  ViewsService, whose only implementation today returns 404
  VIEWS_DISABLED, matching the design's default-off posture.

Validation (OpenHouseViewsApiValidator), structural only -- no SQL is
parsed, translated or validated against an engine: identifier and
cluster consistency, Iceberg SchemaParser round-trip, exactly one
spark representation with a matching sourceDialect, reserved
openhouse./policies keys rejected, baseViewVersion shape, and SQL/schema
UTF-8 payload limits (256 KiB / 512 KiB).

Error codes: the full 14-value ViewErrorCode enum ships now, including
codes M1 never emits, so later milestones add behavior without an enum
change. Codes select HTTP status only and are never serialized;
ErrorResponseBody is unchanged, so no other service's error shape moves.

Tests: 89 view tests. ViewApiContractTest freezes the M1 wire surface so
adding admission, the polymorphic lookup or /versions later cannot
silently change a field this ships.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…torUtil

Review feedback on the views API asked why identifier validation was
duplicated rather than shared. It turned out to be duplicated three ways
already, before views existed, so consolidate rather than add a fourth
copy.

Two helpers move into the existing shared ApiValidatorUtil, which seven
validators across three services already use:

  collectViolations(validator, object, failures)
    Replaces nine byte-identical bean-validation loops in the tables,
    databases, snapshots, views, jobs and both housetables validators.

  validateIdentifier(fieldName, value, failures)
    Replaces five copies of the empty-then-regex identifier check. The
    single-message short-circuit is preserved: an absent identifier is
    not additionally reported as malformed.

One user-visible message changes. OpenHouseDatabasesApiValidator had
drifted to "databaseId provided: %s" while the tables and views
validators both emit "databaseId : provided %s"; it now matches. No test
asserted the drifted wording.

Views keeps its extra identifier length rule, layered after the shared
call and guarded so an empty or malformed identifier still yields
exactly one message. Its error-code precedence is untouched: bean
violations remain generic-category and never set the schema or dialect
flags.

Deliberately left alone: the jobs and housetables identifier checks.
They skip the empty check and jobs uses the hyphen-allowing regex, so
folding them in would mean parameterizing both the pattern and the
empty-check behaviour, which costs more than the duplication saves.

Also drops five now-dead OperatorWrap suppressions and eight unused
imports left behind by the extracted loops.

Verified across every module that depends on services/common:
872 tests, 0 failures.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
example =
"<fs>://<hostname>/<openhouse_namespace>/<database_name>/<viewUUID>/metadata/<uuid>.metadata.json")
@JsonProperty(access = JsonProperty.Access.READ_ONLY)
private String metadataLocation;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[iceberg-rest][blocking] Return the complete Iceberg REST load response

The pinned
Iceberg REST load-response schema
requires both metadata-location and a complete metadata object whose required fields are defined
by the
view-metadata schema.
This response returns only a pointer plus OpenHouse-specific fields such as clusterId, viewUri,
viewVersion, and creationTime. The API therefore does not carry enough state to load or refresh
a view without an out-of-band metadata read or plugin-owned behavior because the schemas, current
version, version log, representations, UUID, location, and properties are absent.

Please make the public load response match the linked Iceberg REST schema, including the complete
metadata object. If OpenHouse still needs a pointer-only internal response, keep it on a separate
internal contract rather than making it the view load API.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the clarification — agreed on avoiding gratuitous differences from Iceberg REST even while keeping the plugin architecture.

The pointer-only response is an explicit decision in the finalized M1 design, which records the parent-doc call to implement against the existing OpenHouse specs and defer Iceberg REST so tables and views keep similar query paths. The Spark 3.5 path follows metadataLocation and lets Iceberg's ViewMetadataParser load the complete metadata, so the definition isn't lost — it's just not duplicated on the wire.

This is a real REST-compatibility gap rather than a claim of compatibility, and I'd rather name it as follow-up work than paper over it. My reasoning for not closing it here is in the top-level comment: partial alignment doesn't buy the interop, and two response conventions in one service is the expensive part.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't an answer but an explanation. Revisit the decision.

Whether or not Clients use rest is orthogonal as to whether the API is rest compatible, and there is no blocker for compatibility that is outlined.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair — that was a rationale, not a blocker, and I should be straight that there isn't one. Returning the full metadata object is technically achievable here.

My objection is scope, not feasibility. If a view load returns complete metadata inline, a table load should too — otherwise the service has two response philosophies, and the next person has to know which resource follows which. GetTableResponseBody today returns a pointer plus schema, not TableMetadata. So the change I'd support is "OpenHouse load responses carry full metadata", applied to both resources with one client story — not "views does, tables doesn't."

If you want that scoped, I'll take it as its own piece of work. What I don't want is views unilaterally establishing a second convention inside the same service.

@Schema(
nullable = true,
description = "The version of the view that the current update is based upon")
private String baseViewVersion;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[iceberg-rest][blocking] Use the Iceberg create and commit request models

This body defines a separate write protocol. It requires OpenHouse-only values such as clusterId,
sourceDialect, and baseViewVersion, repeats path identity in the body, and PUT replaces the
whole definition. Iceberg REST instead uses the
create-view schema
and the
commit-view schema,
whose optimistic checks and mutations are expressed as requirements and updates.

Supporting arbitrary Iceberg commits would require code outside this API to load and rewrite
metadata, invent sourceDialect, and collapse update actions into a custom base token. That makes
the plugin the owner of commit semantics. Please make the wire model directly represent the
Iceberg REST request schemas so the API remains complete if the plugin is removed.
OpenHouse-specific identity can be derived from the route or stored as server-owned metadata.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same thread as the load-response comment.

The patterns here aren't view inventions — CreateUpdateTableRequestBody also requires clusterId, also carries tableId/databaseId in the body alongside the path, and also uses a baseTableVersion token rather than requirements/updates. Views is designed to use that same commit pattern. sourceDialect is the one view-specific addition: Iceberg records a dialect per representation but not which one was authored, so it has nowhere else to live.

If we move to the Iceberg REST request schemas it should happen for tables and views together — a requirements/updates commit model for views alone would leave the two resources with different write protocols.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pattern matching off existing code is the wrong paradigm. Please review this with respect to the data that is required and returned from the rest spec. Its an bad assumption that pattern matching on existing objects is the right metic of correctness.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right that "tables does it" isn't a correctness argument, and I leaned on it too hard. Let me put it properly.

Conformance to a spec we haven't adopted isn't self-justifying either — neither consistency nor conformance is the metric on its own. The metric is what each costs and buys. Partial conformance buys nothing today: no REST client works against these endpoints regardless, since the routes and envelope differ. Consistency buys something concrete and immediate: one set of conventions, one set of shared components, and one migration path if we adopt the spec later.

So I'm not arguing the current shape is correct because it matches tables. I'm arguing the divergence has a real cost and the conformance has no realised benefit until we adopt the spec properly — at which point both resources should move together.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So the actual issue here is twofold:
1 - Version ID is a catalog concept not actually tied to the metadata.
2 - We have had issues in the past because we pass (version,metadata) around causing incorrect handling of the data.

The metadata path is actually unique and used by OSS with respect to conflict resolution, and this has impact because e.g. drop+re-create will rotate the table but the version may remain the same (0) causing an incorrect modification.

My suggestion is we consider using the metadata path vs the version ID as the unique thing for transactions due to the issues associated with version ID.

// An absent or empty list is already reported by bean validation.
return;
}
if (representations.size() != 1) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[iceberg-view][blocking] Do not freeze out future Trino representations

The
Iceberg view representation rules
allow a view version to contain multiple SQL representations and require only that a dialect appear
at most once. The validator rejects every list whose size is not one and rejects every dialect
except lowercase spark.

Spark-only execution is the current scope, so the API does not need to execute Trino SQL in this
milestone. The contract is being frozen now, however, and these checks make adding a Trino
representation later require a validation and wire-behavior change despite the request already
using a list for future representations.

Please accept and persist representations with unique normalized dialects now. Admission and
read-time policy can continue to allow only Spark until Trino support is added.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The list shape is already frozen, so adding Trino later is a validation relaxation rather than a wire change — and relaxing a rule never breaks a request that works today, so I don't think this is a one-way door.

Holding it for M1 is deliberate: the ticket specifies exactly one entry with dialect spark, and UNSUPPORTED_VIEW_DIALECT exists to enforce that. The admission service is a no-op in this milestone, so if the validator accepts Trino nothing downstream rejects it and we'd persist representations no in-scope engine can execute.

Agreed admission is the right home for dialect policy once it's real — filing the relaxation against that milestone.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this != 1 is the problem as its conflating the domain logic and not asserting what you intend to actually protect. Make it a config on which dialects are supported and this problem is generic and future-proof for the half as you implement more plugins.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, and this is the better framing — size != 1 is a proxy for the rule I actually want, which is "dialects this deployment supports."

I'll make the supported set config-driven and drop the count check. Unique dialects already prevents duplicates, so with the set configured to spark for M1 the behaviour is unchanged — a Trino representation is still rejected — but the rule now says what it means, and adding a dialect later is config rather than a code change.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Config-driven dialect validation is in ed622f92. The supported set is now cluster.tables.views.supported-dialects, defaulting to spark, and the size != 1 check is gone — uniqueness already prevents duplicate dialects, so an unsupported dialect is now rejected on its own terms rather than via a count proxy.

Behaviour under the default is unchanged: a Trino representation is still rejected in M1. ViewsValidatorMultiDialectTest configures spark,trino and asserts a two-representation request is accepted, so the extension point is demonstrated rather than assumed. Adding an engine is now a config change.


@Schema(description = "Page of View objects in a database", example = "")
@JsonProperty(access = JsonProperty.Access.READ_ONLY)
private Page<GetViewResponseBody> pageResults;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[iceberg-rest][blocking] Return the Iceberg REST list shape

The pinned
Iceberg REST list-response schema
returns identifiers as {namespace: [...], name: ...} and uses an opaque next-page-token. This
response exposes a Spring Page<GetViewResponseBody> whose elements are intentionally incomplete,
including null pointer fields and creationTime: 0. It also freezes Spring pagination internals
and OpenHouse-only item fields into a public list response. The null fields and zero timestamp also
violate the boundary rule: absence is represented as Optional internally, and an external
response contains only fields that the response contract actually supplies.

Please return the Iceberg identifier list and continuation-token shape. Keep any page-number or
framework-specific state behind the API boundary.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The v2 paginated table endpoint puts a Page<GetTableResponseBody> in pageResults on GetAllTablesResponseBody, and views mirrors that.

The sparse elements match tables too: v2 table search returns identifier-only DTOs unless the caller opts into more via ?fields=, and GetTableResponseBody.creationTime is likewise a primitive long that serializes as 0 when unpopulated. So views is following the existing shape rather than introducing one, and ?fields= is the same extension point if we want richer view list results later.

That said, the Iceberg identifier structure ({namespace: [...], name: ...}) and an opaque continuation token are the cleanest of your REST asks to adopt, since they don't touch commit semantics. I'd still want it done for tables and views together rather than views diverging first.

ErrorResponseBody.builder()
.status(httpStatus)
.error(httpStatus.getReasonPhrase())
.message(codedApiException.getMessage())

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[iceberg-rest][blocking] Preserve the Iceberg REST error model

The pinned
Iceberg REST error model
requires message, type, and numeric code inside the
error-response wrapper.
CodedApiException reduces the view taxonomy to an HTTP status, and this handler emits the status
reason plus free text. NO_SUCH_VIEW, DATABASE_NOT_FOUND, and VIEWS_DISABLED, for example, all
become indistinguishable at the API boundary unless another component parses mutable message text.
If the plugin is removed, no typed failure contract remains.

Please preserve a stable public error type and numeric code in the REST envelope. The internal
ViewErrorCode names do not need to become the public vocabulary, but the boundary must map each
outcome once to a documented Iceberg-compatible type.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair point, and I want to be straight that this is a deliberate gap rather than an oversight: NO_SUCH_VIEW, DATABASE_NOT_FOUND and VIEWS_DISABLED do all surface as 404 with only the message distinguishing them.

ErrorResponseBody is shared across tables, jobs, housetables and optimizer, and no OpenHouse resource emits a machine-readable code today. Widening that envelope affects every service that consumes it, so I'd rather sequence it as its own change than fold it into this PR. Filing it so it isn't lost.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To put a point on this this, we get errors like:
"Table has invalid metadata: Failed to open input stream for file: "

IO errors reading table metadata are interpreted as raw 500 errors vs 503. I would preserve the errors rather than using the same RuntimeException pattern so we can cleanly delineate.

private String type;

@Schema(
description = "Opaque SQL text of the view representation. The server never parses it.",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[writing-review][blocking] Do not publish milestone behavior as a permanent promise

The generated schema says, "The server never parses it," while the PR description says admission
logic will arrive in a later milestone. This class does not parse SQL, but that does not mean the
service will never parse it. The same absolute framing appears in the pointer-response Javadoc.
These statements turn today's implementation into a public promise and will become false when the
admission logic or complete REST-compliant load response is added.

Please describe the current contract instead. For example, say that this endpoint accepts SQL as
opaque text and that this response currently omits definition fields.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in a033058. The absolute phrasing now describes the current contract instead — this one reads "accepts it as opaque text: it is stored as sent and is not parsed or rewritten here", and the same sweep covered GetViewResponseBody, ViewDto and ViewsController.

I left "never serialized" on ViewErrorCode and the two "never echoed" statements in the validator, since those are structural invariants and redaction requirements rather than milestone behaviour.

* <p>The status is the only thing that reaches the wire. The error body shape is unchanged, so no
* subclass taxonomy is serialized.
*/
public abstract class CodedApiException extends RuntimeException {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[code-smells][blocking] Make the new API failure hierarchy checked

CodedApiException extends RuntimeException, so every expected view failure is unchecked.
Validation throws ViewRequestValidationFailureException, the disabled service throws
ViewApiException, and none of the validator, service, or handler interfaces declares either
failure. ViewRequestValidationFailureException.requireBadRequest adds another unchecked
IllegalArgumentException, while ViewApiException accepts a nullable errorCode that is later
dereferenced during exception handling.

Make the domain hierarchy checked and declare the specific failures through the internal
interfaces. Replace the IllegalArgumentException guard with a type that can represent only valid
validation codes, and keep errorCode required. Translate the checked exception once in
OpenHouseExceptionHandler. The legacy table API's runtime-exception convention is not a defense;
this new /v2 surface is the strangler boundary.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both specific defects are real and are fixed in a033058:

  • ViewApiException accepted a nullable errorCode that getHttpStatus() dereferenced, so a null would have NPE'd inside the exception handler and surfaced as a 500 rather than the intended status. It's now Objects.requireNonNull in both constructors.
  • requireBadRequest policing the code range with IllegalArgumentException is gone. ViewRequestValidationFailureException now takes a type that can only hold the three 400-class codes, so the guard is unnecessary rather than relocated.

On making the whole hierarchy checked I'd push back for now. CodedApiException extends RuntimeException matches every existing exception in OpenHouseExceptionHandler, and making views the only checked one forks the shared handler and the validator utilities this PR just consolidated. There's also no domain layer behind ViewsService yet — it's a stub — so declaring checked failures through those interfaces would be declaring them through a seam with no implementation. Worth doing as a service-wide change with a sequencing story; happy to help scope that.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed, the error handling from openhouse routinely mishandles errors. OpenhouseExceptionHandler is an area we can improve.

@EqualsAndHashCode
@NoArgsConstructor(access = AccessLevel.PROTECTED)
@AllArgsConstructor(access = AccessLevel.PROTECTED)
public class ViewDto {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[code-smells][blocking] Remove nullable and zero-sentinel state from ViewDto

This internal DTO uses nullable fields for both pointer and definition state, and primitive
creationTime and lastModifiedTime values use zero when the mapper leaves them unset. The same
type is expected to represent a write command, a loaded view, and an identifier-only list item.
Each phase therefore relies on undocumented knowledge about which fields are null and whether zero
is a timestamp or an absence sentinel.

Split this into required domain types for create or commit input, loaded view state, and list
summary. Use Optional only for values that are legitimately absent in each type. Convert to and
from nullable JSON fields at the HTTP mapper boundary.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ViewDto mirrors TableDto here — that's also used as write input, loaded state and sparse list element, with primitive long timestamps showing the same zero-when-unset behaviour.

The tri-modal use is a fair criticism. I'd rather not split it while ViewsService is a stub, though: all three types would feed a method that throws VIEWS_DISABLED, so we'd be designing a domain model with nothing to validate it against. The natural point is when the catalog and commit path land and there's real state to model — the split will be better informed then, and I'd rather do it once with that knowledge than guess now.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think its a very simple change to avoid a null and the DTOs don't cross the wire and don't need ot match.

"databaseId : provided %s, doesn't match with the RequestBody %s",
databaseId, requestBody.getDatabaseId()));
}
validateSchema(requestBody.getSchema(), failures);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[code-smells][blocking] Normalize nullable request fields once

validateBody passes nullable request getters into separate internal helpers, and those helpers
repeat null checks for representations, catalog, namespace, properties, and base version. This
spreads the external JSON null contract throughout the validator and creates multiple places where
absence semantics can diverge.

At the validator entry point, convert each external nullable value to Optional or reject it as a
required field. Pass required values and explicit Optional values to the internal validation
functions. The raw JSON DTO should not cross the boundary-validation step.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed — five helpers each deciding what absence means is a real spread. Fixed in a033058: validateBody normalizes each optional field once and passes required values or explicit Optionals to the helpers, so absence is decided in one place. Behaviour is unchanged; all 27 validator tests pass unedited.

public static String schemaOfExactUtf8Size(int totalBytes) {
int padding = totalBytes - VIEW_SCHEMA_LITERAL.length();
if (padding < 0) {
throw new IllegalArgumentException(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[code-smells][blocking] Remove unchecked exceptions from test-data helpers

schemaOfExactUtf8Size and sqlOfExactUtf8Size throw IllegalArgumentException for invalid
helper input. These are new unchecked failure paths, even though they are confined to test
utilities.

Use a checked test-fixture exception and declare it at the helper boundary, or change the helper API
so an invalid size cannot be constructed. Do not introduce IllegalArgumentException as the
precondition mechanism.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in a033058. Rather than add a checked exception I made the invalid input unconstructible: schemaOfExactUtf8Size(int) / sqlOfExactUtf8Size(int) are replaced by four no-argument fixtures derived from the limits — schemaAtMaxUtf8Size(), schemaOneByteOverMaxUtf8Size(), and the SQL equivalents. There's no argument to get wrong, so there's no precondition to enforce.

@mkuchenbecker

Copy link
Copy Markdown
Contributor

The previous was a claude review that I reviewed the output before posting. To give more context with resepect to REST, we can still use the glue-style plugin for the implementation but ensuring the data we receive and return matches the rest spec or is as close as possible will make a future transition easier.

@ruolin59
ruolin59 force-pushed the rufan-linkedin-views-v2-api-contract branch from 59dfe0a to b08778d Compare August 26, 2026 18:10
@ruolin59
ruolin59 marked this pull request as draft August 26, 2026 18:10
@ruolin59
ruolin59 marked this pull request as ready for review August 26, 2026 18:17
…andling

Review feedback on the /v2 views wire surface. Six accepted items; the
Iceberg REST alignment and checked-exception migration comments are
answered on the PR rather than changed here.

Audit redaction. ServiceAuditAspect records the whole cached request
body, so a view create or replace wrote the caller's SQL and schema into
service audit events. A new ServiceAuditPayloadRedactor seam in common
lets a service register redaction without common knowing what a view is;
the tables-side implementation scopes by request URI, so table, database
and snapshot payloads are untouched, and replaces schema and every
representations[*].sql with a marker rather than dropping the keys. The
aspect now also fails closed: if a redactor throws, the payload is
dropped instead of emitted raw.

Failure handling. ViewApiException accepted a null error code that
getHttpStatus() then dereferenced, so a null would have surfaced as a 500
from inside the exception handler rather than the intended status; it is
now required. ViewRequestValidationFailureException policed its code
range with an IllegalArgumentException guard, which is replaced by a type
that can only hold the three 400-class codes, making the guard
unnecessary. The Iceberg schema parse no longer catches bare Exception:
it catches IllegalArgumentException and UncheckedIOException, which is
the actual split -- Iceberg wraps Jackson IO failures in the latter and
raises the former for structurally valid JSON that fails its own checks.
A parser defect now propagates as a server fault instead of being
reported as a client 400. ViewSchemaParseBoundaryTest pins that split so
an Iceberg upgrade that moves it fails loudly.

Validator null handling. validateBody normalizes each optional request
field once and passes required values or explicit Optionals to the
helpers, so absence is decided in one place rather than five.

Documentation. Schema and Javadoc text that described this milestone's
behaviour as permanent ("the server never parses it") now describes the
current contract, since those strings are published in the generated
OpenAPI document.

Test fixtures. The exact-size schema and SQL helpers took an int and
threw IllegalArgumentException on bad input; they are replaced by
no-argument fixtures derived from the limits, so invalid input cannot be
constructed.

901 tests, 0 failures across common, tables, jobs, housetables and
internalcatalog.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@ruolin59

Copy link
Copy Markdown
Collaborator Author

Thanks for the context, and understood on the review provenance.

On REST: I don't think "as close as possible" is a stable target. Partial alignment doesn't buy the interop — no client can program against close to the spec, so a REST-shaped-but-not-compliant response still needs OpenHouse-specific client code. And it front-loads the cheap part: the expensive work in adopting REST is the semantics (requirements/updates commits, namespace handling, error taxonomy, continuation tokens), not the response bodies. When we actually transition we'd rewrite both resources against the real spec anyway.

The cost, though, lands immediately. Views returning REST-shaped bodies while tables return OpenHouse-shaped ones means two response conventions in one service, and everything shared absorbs it — the exception handler, the validator utilities, the audit aspect, the generated client. The second commit here consolidates nine duplicated validation blocks into shared utilities; REST-shaping views alone forks that back apart. Two shapes is the maintenance problem, not writing a different API.

So I'd keep views consistent with tables now and treat Iceberg REST as a deliberate migration for both together. Happy to help scope it if you think that should be near-term.

Separately, several of your comments are independent of the REST question — the audit payload leak, the over-broad parser catch, the nullable errorCode, the IllegalArgumentException guard, the validator's null handling, and the test-helper preconditions. Those are all fixed in a033058; I've replied inline on each.

One correction worth surfacing from that batch: on the parser catch, IllegalArgumentException alone would have been wrong. Iceberg's JsonUtil.parse wraps Jackson IO failures, so malformed JSON arrives as UncheckedIOException and only structurally-valid-but-invalid schemas raise IllegalArgumentException. Narrowing to the latter would have turned every malformed schema into a 500. The catch covers both, and there's a test pinning the split so an Iceberg upgrade that moves it fails loudly.

@ruolin59
ruolin59 marked this pull request as draft August 26, 2026 19:36
@mkuchenbecker

Copy link
Copy Markdown
Contributor

On REST: I don't think "as close as possible" is a stable target. Partial alignment doesn't buy the interop — no client can program against close to the spec, so a REST-shaped-but-not-compliant response still needs OpenHouse-specific client code. And it front-loads the cheap part: the expensive work in adopting REST is the semantics (requirements/updates commits, namespace handling, error taxonomy, continuation tokens), not the response bodies. When we actually transition we'd rewrite both resources against the real spec anyway.

Ideally we just implemt the rest spec but use the plugin as glue. If you don't implement the rest spec, I want to do a deep dive into each api to understand the differences as to why we can't be rest compatible and what the blockers are.

Assuming we CANT implement rest:

  1. Don't require data that the rest spec does not require.
  2. Don't exclude data that the rest spec currently inlcudes.
  3. Make sure all information returned from the SPEC is included in the response.

@ruolin59

Copy link
Copy Markdown
Collaborator Author

Happy to do the deep dive. But working through the three rules concretely, I don't think they produce the compatibility they're aiming at.

The load-bearing problem is that none of them are one-way doors. Relaxing a required field later is backward-compatible. Adding a field to a response later is backward-compatible. So every change the rules ask for is equally available whenever we actually adopt the spec — applying them now doesn't reduce future work, it just moves views off the shape tables uses in the meantime.

Rule 1 also doesn't buy compatibility on its own. A client written against the REST spec fails against this API regardless of whether we require clusterId, because the routes, the request envelope and the response shape all differ. Dropping required fields doesn't make an incompatible API compatible — it makes it incompatible in a different way, and now also inconsistent with tables. clusterId in particular is required on every OpenHouse API and catches misrouted requests; views would be the only resource not validating it.

On rules 2 and 3 — the information isn't excluded from the API, it's at metadataLocation, and the Iceberg client reads it natively from there. Inlining it would mean this service defines and maintains its own complete wire model of ViewMetadata — schemas, version log, representations, properties, UUID — duplicating state that already has a canonical serialized form. If that model ever drifts from Iceberg's, we've made compatibility worse while looking more compatible.

What I'd optimise for instead is uniformity. Keeping views and tables on one shape means one set of conventions to learn, one set of shared components, and — if we adopt the spec — one migration with one client story, rather than two resources starting from different places and needing separately-designed transitions. Views conforming halfway now is the thing that makes that migration harder, not easier.

Still up for the deep dive if you want to walk each endpoint — I'd just want the outcome to apply to tables and views together.

@ruolin59
ruolin59 marked this pull request as ready for review August 26, 2026 20:27
Review feedback: the representations check asserted a proxy for the rule
it meant. Rejecting any list whose size was not one, and hardcoding
"spark" at the two dialect comparisons, conflated "what this deployment
can execute" with "how many representations were supplied".

The supported set is now a cluster property, cluster.tables.views
.supported-dialects, defaulting to spark. The size check is gone;
uniqueness already prevents duplicate dialects, and a dialect outside
the configured set is rejected on its own terms. Behaviour under the
default is unchanged -- a Trino representation is still rejected in M1 --
but the rule now states its intent, and supporting another engine
becomes a configuration change rather than a code change.

Comparison remains exact lowercase, and duplicate detection remains
case-insensitive. The configured values are resolved once into an
immutable set that tolerates a null lookup, since a representation can
reach the membership test with no dialect. Error messages name the
configured set rather than spark; they carry server configuration only,
never request payload.

ValidatorConstants.SPARK_VIEW_DIALECT is removed: it became unused, and
its documentation described a rule that no longer exists.

ViewsValidatorMultiDialectTest configures spark and trino together and
asserts a two-representation request is accepted, so the extension point
is demonstrated rather than assumed.

907 tests, 0 failures across common, tables, jobs, housetables and
internalcatalog.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
mkuchenbecker added a commit to mkuchenbecker/openhouse that referenced this pull request Aug 28, 2026
#42 squash-merged the upstream sync into main, so main's tree already
matches the sync branch this PR was stacked on. Merging main in re-bases
the diff on main by merge-base rather than by rewriting history, so the
PR can retarget to main and still show only upstream linkedin#694's 48 files.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DVsAKbhB8vYDa4YUqePvmd
*/
@Builder(toBuilder = true)
@Value
public class GetViewResponseBody {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Pattern(
regexp = ALPHA_NUM_UNDERSCORE_REGEX_HYPHEN_ALLOW,
message = ALPHA_NUM_UNDERSCORE_ERROR_MSG_HYPHEN_ALLOW)
private String clusterId;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is unclear why this is needed on the request, as discussed. Response makes more sense.

ruolin59 added a commit to ruolin59/openhouse that referenced this pull request Aug 28, 2026
…merge)

Extracted from linkedin#694 so the wire contract can be read on its own. This
branch is a reading aid: linkedin#694 remains the change under review and the one
that would merge. Tests, controller, handler, validator and service seam
are all in linkedin#694; only the models are reproduced here.

  CreateUpdateViewRequestBody carries viewId, databaseId, clusterId,
  schema, representations, sourceDialect, defaultCatalog,
  defaultNamespace, viewProperties and baseViewVersion. representations
  is a list of ViewRepresentation {type, sql, dialect} rather than a
  scalar sql field, so a second dialect can be added later without
  changing the shape.

  GetViewResponseBody carries the pointer and row-backed identity only.
  SQL, schema and history stay in the Iceberg metadata file, which the
  client already reads through its own FileIO.

  GetAllViewsResponseBody is paginated from the first release and reuses
  GetViewResponseBody for its elements, populated with identifiers only,
  matching how the v2 table search behaves.

  ViewErrorCode declares all fourteen failure modes now, including the
  ones this milestone never emits, so later work adds behaviour without
  changing the enum. It selects an HTTP status and is never serialized;
  the shared error body is untouched.

273 lines across five files.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Review feedback from both reviewers, and they are right. On tables, /v2
means version two of one specific endpoint: it was introduced solely so
POST /v2/databases/{databaseId}/tables/search could add pagination while
/v1/.../tables/search stayed live. It is not an API generation.

Views has no /v1 list endpoint, so there is nothing for it to be version
two of, and the prefix carried no meaning there. Views will only ever
expose a paginated list, so no non-paginated /v1 variant needs reserving.
All five routes move to /v1, which is where every other current
OpenHouse resource lives.

No collision: the existing /v1/databases/... routes occupy the tables,
tables/search, aclPolicies and softDeletedTables segments, and views
occupies its own.

The audit redactor scopes by request URI, so its two patterns move with
the routes. Had they been left behind, view SQL and schema would have
been written unredacted into service audit events. The redactor tests
cover the new paths and additionally pin the old /v2 paths as
non-matching, so the scope cannot silently drift back.

Tables and views now share the /v1/databases/{databaseId}/ prefix and are
separated only by their own segment, so the routing regression test is
stronger rather than merely renamed. It asserts on response bodies that a
view-like table name reaches the tables handler, that a table literally
named "views" still reaches the tables handler, and that both view routes
reach the views handler. A test asserting the old contract, that view
paths under /v1 must not resolve, is inverted to guard against a stale
/v2 mapping being reintroduced.

909 tests, 0 failures across common, tables, jobs, housetables and
internalcatalog.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
ruolin59 added a commit to ruolin59/openhouse that referenced this pull request Sep 1, 2026
…merge)

Extracted from linkedin#694 so the wire contract can be read on its own. This
branch is a reading aid: linkedin#694 remains the change under review and the one
that would merge. Tests, controller, handler, validator and service seam
are all in linkedin#694; only the models are reproduced here.

  CreateUpdateViewRequestBody carries viewId, databaseId, clusterId,
  schema, representations, sourceDialect, defaultCatalog,
  defaultNamespace, viewProperties and baseViewVersion. representations
  is a list of ViewRepresentation {type, sql, dialect} rather than a
  scalar sql field, so a second dialect can be added later without
  changing the shape.

  GetViewResponseBody carries the pointer and row-backed identity only.
  SQL, schema and history stay in the Iceberg metadata file, which the
  client already reads through its own FileIO.

  GetAllViewsResponseBody is paginated from the first release and reuses
  GetViewResponseBody for its elements, populated with identifiers only,
  matching how the v2 table search behaves.

  ViewErrorCode declares all fourteen failure modes now, including the
  ones this milestone never emits, so later work adds behaviour without
  changing the enum. It selects an HTTP status and is never serialized;
  the shared error body is untouched.

273 lines across five files.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@ruolin59 ruolin59 changed the title [views] Add /v2 views REST contract, controller, handler and validation [views] Add /v1 views REST contract, controller, handler and validation Sep 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants