GH-31374: [C++] Prevent strptime from normalizing invalid dates - #50747
fallintoplace wants to merge 1 commit into
Conversation
|
|
047180d to
9e2fca7
Compare
|
|
9e2fca7 to
d4ec27c
Compare
|
|
kita-renji
left a comment
There was a problem hiding this comment.
The fix looks correct to me. I built it locally (Linux, clang 22, Debug) and compared it with the merge-base 5d6cb93. The new test fails on the base and passes with the patch, and arrow-utility-test, arrow-csv-test and arrow-compute-scalar-temporal-test all pass.
With glibc, 1900-02-29, 2100-02-29, 2023-02-29 and the 31st of Apr/Jun/Sep/Nov are now rejected, while 2000-02-29, 2024-02-29 and Jan/Dec 31 still parse, also with %m/%d/%y, a time part and %z. Formats without %d (%Y-%m, %Y, time-only) behave as before thanks to the std::max(tm_mday, 1) clamp. The check runs after the strptime call, so it also covers the vendored musl path used on Windows (I simulated that on Linux, not on real Windows). As a side effect %Y-%j "2023-366" no longer parses as 2024-01-01. This also matches the ymd.ok() check the ISO8601 parser already has (value_parsing.h:657).
Some non-blocking points:
-
The user-facing change reaches a bit beyond
strptime, so it may be worth listing under user-facing changes: CSV with customtimestamp_parsers(a column containing2023-02-29used to infer astimestamp[s]holding 2023-03-01 and now infers asstring; default CSV options are unaffected), Gandivato_date(to_date_holder.cc:90) and Flight cookieExpiresparsing (cookie_internal.cc:127). All three are the right outcome. A small CSV test would pin it down. -
Day
00still gets through on Windows, which predates this PR. The vendored musl range check atcpp/src/arrow/vendored/musl/strptime.c:196isif (*dest - min >= range), while upstream musl has>= (unsigned)range, so values belowminpass and the clamp turns day 0 into day 1 ("2024-03-00" gives 2024-03-01). Probably a separate follow-up. -
Optional tests: one format without
%d(%Y-%m"2024-02" gives 2024-02-01), and computestrptimewitherror_is_null=truereturning null for 1999-02-30, which is the case from the issue.
The C++/Python/R workflow runs on this PR have 0 jobs and were closed after 30 days, so CI hasn't actually run yet and will need a re-trigger (e.g. a rebase).
Not checked: macOS libc, a real Windows build, pyarrow/R end to end, lint.
Reviewed with help from Claude Code; I ran the builds and checks above myself.
d4ec27c to
2bae38f
Compare
Rationale for this change
After
strptimepopulates atm, Arrow converts its calendar fields directly tosys_days. Sincesys_daysnormalizes invalid dates, an input such as1999-02-30becomes March 2 instead of failing to parse.What changes are included in this PR?
Validate the parsed
year_month_daybefore converting it tosys_days. Invalid dates now use the existing parse-failure path.The regression test covers invalid month lengths and leap days. It also verifies the exact timestamps produced for valid leap-year and month-end boundaries.
Are these changes tested?
Yes. The regression test fails against the original implementation and passes with this change. Both the focused
TimestampParser.StrptimeParser*tests and the completearrow-utility-testsuite pass locally.Are there any user-facing changes?
Invalid calendar dates passed to
strptimenow fail instead of producing normalized timestamps. Valid dates retain their existing results.This PR contains a "Critical Fix". Invalid input could previously produce an incorrect timestamp.