[hughhhh]

SUMMARY

Charts on Hive/Impala datasets were silently dropping rows when a partition filter mapping used the suggested transform unix_timestamp(:value). If the partition key stores hour- or day-start epoch seconds, that transform turns a filter bound like 10:47:23 into the raw instant rather than its bucket boundary, so the mirrored partition predicate excludes rows the original filter keeps. QA reproduced this on Hive 4.0.1 and Impala 4.4.1.

The fix:

• Engine default removed. Hive/Impala (and their subclasses: Spark, Databricks Hive, Ascend) no longer suggest a transform. A BIGINT key doesn't tell Superset its bucket size or timezone.
• Existing mappings protected. A temporal-to-numeric mapping whose transform is exactly the bare former default stops mirroring. Its filters still apply; only the extra partition predicate is dropped. The editor shows why. Stored expressions are not rewritten.
• Any raw-instant spelling caught. On Hive/Impala-family engines, the transform is also evaluated at two instants 2,843 seconds apart within one hour (a separate, cached probe). If the answers keep that distance (or 2,843,000 for milliseconds), the transform returns raw instants, so mirroring is declined. This catches CAST(unix_timestamp(:value) AS BIGINT), + 0, * 1, floor(unix_timestamp(:value)) and STRING timestamp columns. A key that really stores raw epoch seconds is declined too: full scan, correct rows. Other engines are unaffected, since a raw epoch column there is an ordinary index.
• Explicit data check. The editor's new Validate against data action takes one to three existing partition keys. It reads up to 50 rows from each, evaluates the transform fresh and compares each output with that same row's stored key. Results are matched, mismatch (with examples) or inconclusive. The read is a raw LIMIT 50 select (no GROUP BY) through QueryContext with raise_for_access() and RLS, with no COUNT(*) and no time filter. Probe caches and mirror verdicts are bypassed. A sampled match is reported as exactly that, not as proof for the whole dataset.
• Honest preview label. The expression-only preview reads "Expression evaluated — data not checked" instead of "Valid".
• Docs. The partition-filter-mapping guide and UPDATING.md give correct hourly/daily floor recipes and explain timezone and validation semantics.

Operator widening and the existing timezone normalization (adjust_time_bounds) are unchanged.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

• New Hive/Impala mapping: transform prefilled/suggested as unix_timestamp(:value) → no suggestion; placeholder asks for the expression that populates the key.
• Saved bare unix_timestamp(:value) on an hourly key: predicates mirrored, rows lost → not mirrored, with a warning explaining the missing bucket.
• Preview success: "Valid" → "Expression evaluated — data not checked", plus Validate against data.

TESTING INSTRUCTIONS

Proof: backend unit tests for sql/parse, connectors/sqla, datasets, partition mirroring and the Hive/Impala specs pass (3,938 total). Jest PartitionFilterMapping/ passes (119). Staged-file pre-commit passes, including mypy, pylint and the frontend type check. New tests execute generated SQL and compare mirrored and unmirrored row identities for hourly and daily keys across =, IN, ranges and inequalities. They also confirm that the sampling query keeps RLS and partition restrictions and issues no COUNT(*). A further executed-SQL test runs five raw-instant spellings on TIMESTAMP and STRING sources: they are declined on Hive and still mirror on SQLite. With the check disabled, 9 of the 10 cases fail. QA ran an earlier head (2b4db94) on real Hive 4.0.1 and Impala 4.4.1: all 8 story scenarios returned correct counts. QA's remaining High (the raw-instant spellings above) is addressed by the later commits, which have not yet been re-run on live engines.

With PARTITION_FILTER_MAPPING enabled on a Hive or Impala dataset with a TIMESTAMP column mapped to an hour-start BIGINT partition key:

1. Set the transform to unix_timestamp(:value), save, and reopen. Expect a "not mirrored" warning, and no partition predicate in View query.
2. Enter existing partition keys and click Validate against data. Expect "Partition mismatch found" with the source value, stored key and transform output.
3. Change the transform to CAST(floor(unix_timestamp(:value) / 3600) * 3600 AS BIGINT) and validate again. Expect "Matched N sampled rows across M partitions".
4. Compare chart counts for =, IN and partial-hour ranges with and without the mapping. They should be equal.

ADDITIONAL INFORMATION

• Has associated issue:
• Required feature flags: PARTITION_FILTER_MAPPING
• Changes UI
• Includes DB Migration
• Introduces new feature or API: optional validation_partition_values on the partition-mapping preview endpoint, data_validation in its response, safety_refusal in the dataset's partition summary
• Removes existing feature or API: Hive/Impala partition_value_transform_default (the response field stays, now null)

Reviewed with sol-loop: 4 rounds with Claude Opus (GPT Sol unavailable: Codex usage limit). Verdict: the last finding (reverting a failure cache that could have switched the check off for 24 hours) is fixed as requested.

🤖 Generated with Claude Code

[netlify]

✅ Deploy Preview for superset-docs-preview ready!

Name	Link
🔨 Latest commit	73ea97a
🔍 Latest deploy log	https://app.netlify.com/projects/superset-docs-preview/deploys/6ae506ce74cfe84a278be27a
😎 Deploy Preview	https://deploy-preview-45846--superset-docs-preview.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.
🤖 Make changes	Run an agent on this branch

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[codeant-ai-for-open-source]

  CodeAnt PR Risk

🟢  Low Risk

• The PR appears safe to merge; no actionable issues were identified in the review.

Assessed commit: 299eba59f361

[codecov]

Codecov Report

❌ Patch coverage is 90.56047% with 32 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.75%. Comparing base (aa7b2bc) to head (73ea97a).
⚠️ Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
superset/connectors/sqla/partition_validation.py	83.78%	7 Missing and 5 partials ⚠️
superset/connectors/sqla/partition_mapping.py	92.92%	4 Missing and 4 partials ⚠️
...PartitionFilterMapping/PartitionDataValidation.tsx	90.14%	7 Missing ⚠️
superset/datasets/api.py	44.44%	4 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #45846      +/-   ##
==========================================
+ Coverage   84.73%   84.75%   +0.01%     
==========================================
  Files        3069     3071       +2     
  Lines      205781   206128     +347     
  Branches    48391    48511     +120     
==========================================
+ Hits       174366   174696     +330     
- Misses      28122    28132      +10     
- Partials     3293     3300       +7     

Flag	Coverage Δ
hive	34.46% <14.76%> (-0.05%)	⬇️
javascript	81.01% <93.13%> (+0.02%)	⬆️
mysql	52.52% <14.76%> (-0.08%)	⬇️
postgres	52.52% <14.76%> (-0.09%)	⬇️
presto	36.18% <14.76%> (-0.05%)	⬇️
python	87.77% <89.45%> (+<0.01%)	⬆️
sqlite	52.29% <14.76%> (-0.08%)	⬇️
unit	82.54% <89.45%> (+0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.