Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 64 additions & 0 deletions superset-core/src/superset_core/semantic_layers/types.py
Original file line number Diff line number Diff line change
Expand Up @@ -358,6 +358,52 @@ class SemanticResult:

This includes any requests (SQL queries, HTTP requests) that were performed in order
to obtain the results, in order to help troubleshooting.

**Count results.** A row count is one row with one integer column holding
the size of the requested result window (see :class:`SemanticQuery`). The
column's name is not part of the contract (providers conventionally use
``COUNT``): a consumer reads the count from that single cell and does not
look it up by name. An empty window is the cell ``0``: not an empty table,
not ``None`` and not a missing column. A count that cannot be established
exactly is an error
(``superset_core.semantic_layers.errors.SemanticResultCompletenessError``,
as incomplete or unverified), never an estimate, the length of a fetched
page or a zero standing in for "unknown".

**Value lists.** A distinct-value result has one column, named after the
dimension. What such a list proves is its *completeness evidence*, one of
three states:

- *complete*: every distinct value matching the request was obtained, with
evidence that the request was fulfilled and no cap, visible or hidden, cut
it short;
- *known more*: at least one further distinct matching value exists outside
the returned values, proven in the same observation (a distinct value seen
beyond the bound, or a value removed by a cut), never by a later query;
- *unknown*: neither is proven. Unknown is a valid suggestion result, not a
defect.

This type carries no evidence field. The provider's obligation comes from
the entry point: ``SemanticView.get_values`` must return the whole domain
or raise. A consumer, though, may treat a list as complete only on explicit
evidence, never because of the entry point it came from; where evidence is
carried explicitly and is absent, the reading is unknown, and absence is
never read as complete. Whether the values are an *ordered prefix*, the
first values under a deterministic documented provider order whose NULL
placement and tie handling are stated, is a separate claim with its own
proof: the order must have been applied before every cut, and sorting an
already-cut subset afterwards cannot make it a prefix. No evidence may be
inferred from the list itself. A list exactly as long as a cap proves
nothing; a list shorter than the cap proves completeness only with evidence
that nothing was cut; an empty list of unknown completeness is not proof
that no value matches. A consumer whose own ceiling removes a value from a
proven-complete list turns it into known more, and may call the result an
ordered prefix only under an order it applied to the complete list before
cutting; a ceiling that removes nothing changes nothing.

Evidence belongs to one observation. A count and a page fetched separately
may see different source states; nothing here promises a snapshot across
requests, and ordering is not isolation.
"""

requests: list[SemanticRequest]
Expand Down Expand Up @@ -389,6 +435,24 @@ class SemanticQuery:
return rows in any order. ``get_row_count`` gets no default, since order
cannot change a count.

**What a count counts.** The *matching result* of a query is its rows at
the requested grain after ``filters``, grouping and ``group_limit``, before
``order``, ``offset`` and ``limit``. The *requested result window* is that
result, ordered, with ``offset`` rows skipped and at most ``limit`` rows
kept. A row count is the number of rows in the requested result window. For
a matching result of 123 rows: no limit and no offset counts 123;
``limit=50`` counts 50; ``limit=50, offset=100`` counts 23; ``limit=50,
offset=150`` counts 0. ``limit=None`` means no limit and an absent or ``0``
``offset`` means no offset, so a query with neither counts its whole
matching result; that unrestricted total is the count of no other query.
Rows are rows of the result, never underlying facts or metric values:
10,000 facts that aggregate to 12 groups count 12. A count validates the
same shape ``get_table`` would, the shape being members, ``filters``,
``group_limit``, ``limit`` and ``offset`` but not ``order``, so a query the
table would reject, such as an offset on a view without ``ROW_OFFSET``, is
rejected by the count as well, and a count is accepted whenever the table
query would be; a count never widens what may be asked.

``group_limit`` is a separate ranking, with its own metric, direction and
optional filters.

Expand Down
39 changes: 37 additions & 2 deletions superset-core/src/superset_core/semantic_layers/view.py
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,13 @@ class SemanticView(ABC):
before ``offset`` and ``limit``. The host gives an unordered ``get_table``
query that selects a temporal dimension a default chronological order. See
:class:`~superset_core.semantic_layers.types.SemanticQuery`.
- **Counts and value lists.** A row count is the size of the requested
result window, after ``offset`` and ``limit``, never the unrestricted
total of a windowed query or a count of underlying facts. A value list
proves one of complete, known more or unknown, and nothing is inferred
from its length. Exactness is never approximated: what cannot be verified
raises. See :class:`~superset_core.semantic_layers.types.SemanticQuery`
and :class:`~superset_core.semantic_layers.types.SemanticResult`.

Uniqueness works at two levels. In the **catalog**, dimensions are unique
per ``(name, grain)`` -- so several grain variants of one name are expected
Expand Down Expand Up @@ -157,6 +164,20 @@ def get_values(
never return a partial result instead.
Do not drop ``filters`` and retry when a filtered request is incomplete.

The result is the whole distinct domain of the dimension under
``filters``, so its completeness evidence is *complete* (see
:class:`~superset_core.semantic_layers.types.SemanticResult`). A ceiling
of the provider's own, whether a page size, a transport cap or a
hard-coded limit, is not a bound the caller asked for: a result such a
ceiling may have cut is ``"unverified"`` unless the provider has
evidence that nothing was cut. This signature carries no bound. A
bounded suggestion, where the caller names a ceiling and accepts fewer
values than exist together with explicit evidence of what the list
proves, is not expressible here and must not be improvised by
truncating silently. The values come in no promised order unless the
provider documents a deterministic order, NULL placement and ties
included, applied before any cut.

Raise SemanticQueryRejectedError for deliberate input rejection, such as a
search filter the dimension cannot apply; the host returns it to the user
as a client error with host-owned guidance. Leave operational and
Expand Down Expand Up @@ -186,11 +207,25 @@ def get_row_count(self, query: SemanticQuery) -> SemanticResult:
"""
Execute a query and return the number of rows the result would have.

Count the requested result window: the rows of the filtered, grouped
result that remain after ``query.offset`` and ``query.limit``, as
:class:`~superset_core.semantic_layers.types.SemanticQuery` defines it,
under the same validation of the query's shape, ``order`` aside, that
``get_table`` applies. Return one row with one integer column (see
:class:`~superset_core.semantic_layers.types.SemanticResult`); an empty
window is ``0``. The unrestricted total is the count only of a query with
neither limit nor offset; for any other query it may be a different
number and must not be returned as the count. A total may be turned into the
window count arithmetically only when it describes exactly this query's
matching result, with every filter, grouping and group limit applied.

Raise SemanticQueryRejectedError for deliberate input rejection.

Raise ``superset_core.semantic_layers.errors.SemanticResultCompletenessError``
when the result is incomplete or its completeness cannot be verified;
never return a partial result instead.
(``"incomplete"`` or ``"unverified"``) when the count cannot be established
exactly within the provider's budgets: never an estimate, the length of a
fetched page or a zero standing in for "unknown". Never return a partial
result instead.
"""

@abstractmethod
Expand Down
Loading