Skip to content

GH-31374: [C++] Prevent strptime from normalizing invalid dates - #50747

Open
fallintoplace wants to merge 1 commit into
apache:mainfrom
fallintoplace:fix/strptime-invalid-calendar-dates
Open

fallintoplace wants to merge 1 commit into
apache:mainfrom
fallintoplace:fix/strptime-invalid-calendar-dates

Conversation

@fallintoplace

@fallintoplace fallintoplace commented Jul 30, 2026 •

Copy link
Copy Markdown

Rationale for this change

After strptime populates a tm, Arrow converts its calendar fields directly to sys_days. Since sys_days normalizes invalid dates, an input such as 1999-02-30 becomes March 2 instead of failing to parse.

What changes are included in this PR?

Validate the parsed year_month_day before converting it to sys_days. Invalid dates now use the existing parse-failure path.

The regression test covers invalid month lengths and leap days. It also verifies the exact timestamps produced for valid leap-year and month-end boundaries.

Are these changes tested?

Yes. The regression test fails against the original implementation and passes with this change. Both the focused TimestampParser.StrptimeParser* tests and the complete arrow-utility-test suite pass locally.

Are there any user-facing changes?

Invalid calendar dates passed to strptime now fail instead of producing normalized timestamps. Valid dates retain their existing results.

This PR contains a "Critical Fix". Invalid input could previously produce an incorrect timestamp.

@fallintoplace
fallintoplace requested a review from pitrou as a code owner July 30, 2026 17:51
@github-actions

Copy link
Copy Markdown

⚠️ GitHub issue #31374 has been automatically assigned in GitHub to PR creator.

@fallintoplace
fallintoplace force-pushed the fix/strptime-invalid-calendar-dates branch from 047180d to 9e2fca7 Compare July 30, 2026 17:55
@fallintoplace fallintoplace changed the title GH-31374: [C++] Reject invalid calendar dates in strptime GH-31374: [C++] Reject invalid calendar dates parsed by strptime Jul 30, 2026
@github-actions

Copy link
Copy Markdown

⚠️ GitHub issue #31374 has been automatically assigned in GitHub to PR creator.

@fallintoplace
fallintoplace force-pushed the fix/strptime-invalid-calendar-dates branch from 9e2fca7 to d4ec27c Compare July 30, 2026 17:59
@fallintoplace fallintoplace changed the title GH-31374: [C++] Reject invalid calendar dates parsed by strptime GH-31374: [C++] Prevent strptime from normalizing invalid dates Jul 30, 2026
@github-actions

Copy link
Copy Markdown

⚠️ GitHub issue #31374 has been automatically assigned in GitHub to PR creator.

@kita-renji kita-renji left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fix looks correct to me. I built it locally (Linux, clang 22, Debug) and compared it with the merge-base 5d6cb93. The new test fails on the base and passes with the patch, and arrow-utility-test, arrow-csv-test and arrow-compute-scalar-temporal-test all pass.

With glibc, 1900-02-29, 2100-02-29, 2023-02-29 and the 31st of Apr/Jun/Sep/Nov are now rejected, while 2000-02-29, 2024-02-29 and Jan/Dec 31 still parse, also with %m/%d/%y, a time part and %z. Formats without %d (%Y-%m, %Y, time-only) behave as before thanks to the std::max(tm_mday, 1) clamp. The check runs after the strptime call, so it also covers the vendored musl path used on Windows (I simulated that on Linux, not on real Windows). As a side effect %Y-%j "2023-366" no longer parses as 2024-01-01. This also matches the ymd.ok() check the ISO8601 parser already has (value_parsing.h:657).

Some non-blocking points:

  1. The user-facing change reaches a bit beyond strptime, so it may be worth listing under user-facing changes: CSV with custom timestamp_parsers (a column containing 2023-02-29 used to infer as timestamp[s] holding 2023-03-01 and now infers as string; default CSV options are unaffected), Gandiva to_date (to_date_holder.cc:90) and Flight cookie Expires parsing (cookie_internal.cc:127). All three are the right outcome. A small CSV test would pin it down.

  2. Day 00 still gets through on Windows, which predates this PR. The vendored musl range check at cpp/src/arrow/vendored/musl/strptime.c:196 is if (*dest - min >= range), while upstream musl has >= (unsigned)range, so values below min pass and the clamp turns day 0 into day 1 ("2024-03-00" gives 2024-03-01). Probably a separate follow-up.

  3. Optional tests: one format without %d (%Y-%m "2024-02" gives 2024-02-01), and compute strptime with error_is_null=true returning null for 1999-02-30, which is the case from the issue.

The C++/Python/R workflow runs on this PR have 0 jobs and were closed after 30 days, so CI hasn't actually run yet and will need a re-trigger (e.g. a rebase).

Not checked: macOS libc, a real Windows build, pyarrow/R end to end, lint.

Reviewed with help from Claude Code; I ran the builds and checks above myself.

@fallintoplace
fallintoplace force-pushed the fix/strptime-invalid-calendar-dates branch from d4ec27c to 2bae38f Compare September 27, 2026 17:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants