Skip to content

Clarify query-backed Dataset.source representation and SQL dialect #298

Description

@mchades

Problem

The core specification
defines Dataset.source as a string that may identify a physical table/view or contain a query.

For a query-backed source, the specification provides neither:

  • a discriminator that distinguishes a relation identifier from SQL text; nor
  • a SQL dialect for parsing or validating the query.

Consumers therefore cannot interpret query-backed sources consistently.

Questions

  1. Is a query-backed Dataset.source intended to be portable core behavior?

  2. If so, should the specification provide a structured form such as:

    source:
      type: query
      dialect: ANSI_SQL
      query: SELECT ...
  3. Should the existing string form remain the relation-identifier form for compatibility?

  4. If query interpretation is intentionally consumer-specific, should the specification
    explicitly allow consumers to reject query-backed sources?

Context

Gravitino maps an OSI dataset source to a catalog Table or View. Because the query dialect is
currently unspecified, its proposed structured Metric View model requires a query to be registered
as a logical View before it can be referenced as a dataset source.

Related discussions: #52, #294, and #104.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions