Repository navigation
docs(semantic-layers): state the count and value-list completeness contract - #45903
Draft
mikebridge wants to merge 2 commits into
Draft
mikebridge wants to merge 2 commits into
mikebridge wants to merge 2 commits into
Conversation
…ntract A row count is the size of the requested result window, after offset and limit, and never the unrestricted total of a windowed query or a count of underlying facts. A distinct-value list proves one of complete, known more or unknown, with an ordered prefix as a separate claim, and nothing is inferred from its length. Both rules are written into the SemanticView method docstrings and the SemanticQuery and SemanticResult types, with the exact-or-error rule unchanged. No provider or host behaviour changes.
Separate the provider's obligation on get_values (whole domain or raise) from what a consumer may conclude (complete only on explicit evidence). Make the count column's name non-normative, state that limit=None and offset 0 mean no window, define the validated shape as everything but order, require a deterministic order with NULL and tie handling for an ordered-prefix claim, and say a consumer ceiling that removes nothing changes nothing.
✅ Deploy Preview for superset-docs-preview ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #45903 +/- ##
==========================================
+ Coverage 84.73% 84.75% +0.02%
==========================================
Files 3069 3074 +5
Lines 205771 206418 +647
Branches 48396 48399 +3
==========================================
+ Hits 174354 174946 +592
- Misses 28123 28178 +55
Partials 3294 3294
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
SUMMARY
TL;DR. The semantic-layer SDK now says in plain terms what a row count and a list of filter suggestions mean. A row count is the number of rows in the requested window, after offset and limit, and never the total of the whole result or a count of the underlying facts. A list of distinct values proves one of three things, complete, known more, or unknown, and nothing is read into how long it is. Both rules are docstring text on the existing
SemanticViewmethods and theSemanticQueryandSemanticResulttypes. No provider or host behaviour changes in this PR.Background.
SemanticViewis the base class every semantic-layer provider implements. Its docstrings already fix the rules providers must agree on: member identity, grain variants, NULL handling in filters, and ordering before limit. They also require a provider to raiseSemanticResultCompletenessErrorinstead of returning a partial result. What they did not say is what a count counts and what a value list proves. Providers answered those questions differently: some count the rows the query would return after offset and limit, one reports the unrestricted total, and two apply a hard-coded ceiling to value lists without saying so. The host cannot tell these apart, so a paged table can show the wrong total and a capped suggestion list can look exhaustive.Problem. The SDK is about to be versioned. Tagging it with the meaning of counts and value lists left open would force a deprecation cycle once providers are aligned. The rules have been settled; this PR writes them into the SDK so the first tag carries them, while the provider alignment itself lands separately.
Changes.
SemanticQuerygains a "What a count counts" section: the matching result (rows at the requested grain after filters, grouping and group limit) and the requested result window (that result ordered, then offset and limit applied). A row count is the size of the window, with the worked example 123 rows / limit 50 / offset 100 → 23, and 10,000 facts aggregating to 12 groups → 12. A count validates the same query shape asget_tableand never widens what may be asked.SemanticResultgains "Count results" and "Value lists" sections. A count is one row with one integer column, read from that cell rather than by column name; an empty window is the cell0; an inexact count is an error rather than an estimate, a page length or a stand-in zero. A value list's completeness evidence is complete, known more or unknown. The type carries no evidence field: the provider's obligation comes from the entry point, while a consumer may treat a list as complete only on explicit evidence. An ordered prefix is a separate claim that needs a deterministic order, NULL placement and ties stated, applied before any cut. Nothing is inferred from the list's length, including an empty list of unknown completeness, and evidence belongs to one observation, promising no snapshot across requests.SemanticView.get_row_countsays to count the requested result window underget_table's validation of the query's shape (order aside), return the one-cell shape, never return the unrestricted total for a windowed query, and raise the existing completeness error (incomplete or unverified) when the count cannot be established exactly.SemanticView.get_valuessays the result is the whole distinct domain under the filters, that a ceiling of the provider's own is not a bound the caller asked for and makes a possibly-cut result unverified, that this signature carries no bound so a bounded suggestion must not be improvised by truncating silently, and that values come in no promised order unless the provider documents a deterministic one applied before any cut.SemanticViewclass docstring lists "Counts and value lists" among the rules the host relies on.BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
Not applicable: docstring text only.
TESTING INSTRUCTIONS
Read the four docstrings.
pytest superset-core/tests/semantic_layerspasses unchanged (81 passed, 4 skipped), and the changed-file pre-commit hooks (ruff, ruff-format, mypy, pylint) are green. No behaviour changes, so no new tests.ADDITIONAL INFORMATION