Skip to content

[R] read_csv_arrow()'s timestamp_parsers parameter is a bit light on documentation and doesn't appear to do anything #33708

Closed
Closed
[R] read_csv_arrow()'s timestamp_parsers parameter is a bit light on documentation and doesn't appear to do anything#33708
Assignees
thisisnic
Labels
Component: RPriority: MediumStatus: needs championHigh impact issues which aren't being worked on but require a volunteer to move the task forward.Type: enhancement
Milestone
26.0.0

Description

@thisisnic
thisisnic
opened on Jan 16, 2023

Describe the bug, including details regarding any error messages, version, and platform.

library(arrow)
df <- data.frame(time = "2023-01-16 19:47:57")
dst_dir <- tempfile()
dir.create(dst_dir)
dst_file <- file.path(dst_dir, "data.csv")
write.table(df, sep = ",", dst_file, row.names = FALSE, quote = FALSE)

read_csv_arrow(dst_file, timestamp_parsers = c(TimestampParser$create(format = "%m-%d-%y")))
#>                  time
#> 1 2023-01-16 19:47:57

I'm not sure how exactly it's supposed to work, but it appears not to do anything, and neither the docs nor error messages are enough to tell me what I'm supposed to be doing/seeing here.

Component(s)

R

Activity

  1. thisisnic
    added
    Type: bug
    on Jan 16, 2023
  2. github-actions
    added
    Component: R
    on Jan 16, 2023

  3. eitsupi commented on Feb 24, 2023

    @eitsupi
    eitsupi
    on Feb 24, 2023
    Contributor
    library(arrow)
df <- data.frame(time = "2023-01-16 19:47:57")
dst_dir <- tempfile()
dir.create(dst_dir)
dst_file <- file.path(dst_dir, "data.csv")
write.table(df, sep = ",", dst_file, row.names = FALSE, quote = FALSE)

read_csv_arrow(dst_file, timestamp_parsers = c(TimestampParser$create(format = "%m-%d-%y")))
#>                  time
#> 1 2023-01-16 19:47:57

    In this example, the time column does not seem read as a timestamp type, so the timestamp_parsers option does nothing.

    The behavior of this option can be verified by specifying the column type.

    > read_csv_arrow(dst_file, col_types = schema(time = arrow::timestamp()), timestamp_parsers = list("%m-%d-%y"))
Error:
! Invalid: In CSV column #0: CSV conversion error to timestamp[s]: invalid value '2023-01-16 19:47:57'
Run `rlang::last_error()` to see where the error occurred.
    > read_csv_arrow(dst_file, col_types = schema(time = arrow::timestamp()), timestamp_parsers = list("%Y-%m-%d %H:%M:%S"))
# A data frame: 1 × 1
  time
  <dttm>
1 2023-01-16 19:47:57
    > read_csv_arrow(dst_file, col_types = schema(time = arrow::timestamp()), timestamp_parsers = list("%m-%d-%y", TimestampParser$create()))
# A data frame: 1 × 1
  time
  <dttm>
1 2023-01-16 19:47:57

    Regarding the ability to specify multiple parsers to fall back on, as in the last example, I was impressed to learn that pyarrow has this feature and it would be great to have examples in the R documentation.

  4. thisisnic
    changed the title [-][R] `read_csv_arrow()`'s `timestemp_parsers` parameter is a bit light on documentation and doesn't appear to do anything[/-] [+][R] `read_csv_arrow()`'s `timestamp_parsers` parameter is a bit light on documentation and doesn't appear to do anything[/+] on Mar 2, 2023
  5. thisisnic
    added
    Priority: Medium
    on May 30, 2023
  6. thisisnic
    added
    Type: enhancement
    Status: needs championHigh impact issues which aren't being worked on but require a volunteer to move the task forward.
    and removed
    Type: bug
    on Jun 22, 2026
  7. github-actions
    assigned
    thisisnic
    on Sep 6, 2026
  8. thisisnic
    added a commit that references this issue on Sep 16, 2026

    GH-33708: [R] read_csv_arrow()'s timestamp_parsers parameter is a bit…

    6d3331f
  9. thisisnic
    added this to the 26.0.0 milestone on Sep 16, 2026

  10. thisisnic commented on Sep 16, 2026

    @thisisnic
    thisisnic
    on Sep 16, 2026
    MemberAuthor

    Issue resolved by pull request 51166
    #51166

  11. thisisnic
    closed this as completedon Sep 16, 2026
  12. pitrou
    added a commit that references this issue on Sep 17, 2026

    apacheGH-33708: [R] read_csv_arrow()'s timestamp_parsers parameter is…

    1ef70a5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Component: RPriority: MediumStatus: needs championHigh impact issues which aren't being worked on but require a volunteer to move the task forward.Type: enhancement

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions