Describe the bug
Spark accepts a leading + on a timestamp year, but Comet's native string-to-timestamp parser returns NULL. Widening the timestamp fuzz alphabet in PR #5682 exposed this on Spark 4.1.3. Observed inputs include +7528, +00463, +79821, and +2976.
To reproduce
Use Parquet-backed column input to avoid constant folding:
SET spark.sql.session.timeZone=UTC;
SET spark.sql.ansi.enabled=false;
CREATE TABLE positive_year_probe USING parquet AS SELECT '+7528' AS s;
SELECT s, CAST(s AS TIMESTAMP) FROM positive_year_probe;
With native execution disabled, Spark returns the start of year 7528. With Comet enabled, the cast returns NULL. The failing differential run on PR head 0a1fc76a0 plus the review follow-up used 10,000 seeded random strings and exposed nine such bare positive-year inputs. No other mismatch appeared in that run.
Expected behavior / proposed fix
Accept explicit positive years according to Spark's timestamp segment rules. Current patterns use -?[0-9]{4,6}, excluding +. Audit both timestamp and timestamp_ntz paths and numeric decoding before broadening the sign grammar; add column-input Legacy/TRY/ANSI coverage, including signed dates and malformed signs. Remove the narrow positive-bare-year fuzz exclusion in CometNativeCastSuite once fixed.
Describe the bug
Spark accepts a leading
+on a timestamp year, but Comet's native string-to-timestamp parser returns NULL. Widening the timestamp fuzz alphabet in PR #5682 exposed this on Spark 4.1.3. Observed inputs include+7528,+00463,+79821, and+2976.To reproduce
Use Parquet-backed column input to avoid constant folding:
With native execution disabled, Spark returns the start of year 7528. With Comet enabled, the cast returns NULL. The failing differential run on PR head
0a1fc76a0plus the review follow-up used 10,000 seeded random strings and exposed nine such bare positive-year inputs. No other mismatch appeared in that run.Expected behavior / proposed fix
Accept explicit positive years according to Spark's timestamp segment rules. Current patterns use
-?[0-9]{4,6}, excluding+. Audit both timestamp and timestamp_ntz paths and numeric decoding before broadening the sign grammar; add column-input Legacy/TRY/ANSI coverage, including signed dates and malformed signs. Remove the narrow positive-bare-year fuzz exclusion inCometNativeCastSuiteonce fixed.