Repository navigation
GH-39793: [R] Export the C Data Interface allocators and document $export_to_c() / $import_from_c() - #51236
GH-39793: [R] Export the C Data Interface allocators and document $export_to_c() / $import_from_c()#51236singhpratech wants to merge 2 commits into
Conversation
…nt $export_to_c() / $import_from_c() allocate_arrow_schema(), delete_arrow_schema(), allocate_arrow_array(), delete_arrow_array(), allocate_arrow_array_stream() and delete_arrow_array_stream() are now exported, with a help page (?c-data-interface) that shows the round trip, and the Array methods list names $export_to_c() and $import_from_c(). Two tests cover the exports and an Array round trip through the exported functions. Closes apache#39793.
…Data Interface exports Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WQYE6hSNYZur5fRZC3mqGT
|
|
| test_that("the C Data Interface allocators are exported", { | ||
| exported <- getNamespaceExports("arrow") | ||
| for (f in c( | ||
| "allocate_arrow_schema", | ||
| "delete_arrow_schema", | ||
| "allocate_arrow_array", | ||
| "delete_arrow_array", | ||
| "allocate_arrow_array_stream", | ||
| "delete_arrow_array_stream" | ||
| )) { | ||
| expect_true(f %in% exported, label = paste(f, "is exported")) | ||
| } | ||
| }) |
There was a problem hiding this comment.
What's the reason for this test?
There was a problem hiding this comment.
It pins the six names as part of the exported interface, so a later roxygen run that loses an @export fails here, naming the function, rather than in a downstream package's check. The round-trip test below reaches them through arrow:: and would fail too, so if one test is enough I'll drop this one and keep the round trip.
| #' - `$export_to_c(array_ptr, schema_ptr)`: Fill an `ArrowArray` and an `ArrowSchema` struct | ||
| #' from this array for the Arrow C Data Interface; the pointers come from | ||
| #' [allocate_arrow_array()] and [allocate_arrow_schema()]. | ||
| #' - `Array$import_from_c(array_ptr, schema_ptr)`: Build an `Array` from structs |
There was a problem hiding this comment.
This last one looks different
|
Before we consider merging this, I think it'd be good to understand why we would want to do this instead of recommending folk use nanoarrow for this? |
|
The methods are the public part already: The case for nanoarrow instead is that it is the project's answer for the C Data Interface in R, one route is better than two, and the use case on the issue (r-polars) has since gone through nanoarrow. If that is where you would rather keep it, I will cut this to the documentation: the two methods on |
…wered, scope awaiting the maintainer
|
I think it makes more sense to keep nanoarrow as the canonical way to do this, yeah, thanks. Pinging @paleolimbot here too - what are your thoughts Dewey? |
|
Thanks for the ping! I'm an obvious fan of nanoarrow...the pointer functions there are hopefully well documented and help you get/work with the addresses in various ways that aren't exposed here. If for some reason you don't want that as a runtime dependency, you can also inline the allocators and deleters (by copying this header or adding nanoarrow to That said, documenting |
|
Thank you both for taking the time, and Dewey, thanks for the I am glad to go whichever way the project prefers. I would just like to put a few facts next to the
Where I agree completely: for working with the C structures, nanoarrow is the right tool and the Two ways to close this out, and I am happy with either:
Tell me which, and I will have the branch updated the same day. Thanks again for the reviews. |
Rationale for this change
Follows up on #39793.
Array$export_to_c()andArray$import_from_c()work, but the functions thatproduce the pointers they take,
allocate_arrow_schema()and friends, are not exported, and none ofit has a help page, so a package that hands a column to another library over the C Data Interface
has to use
arrow:::, whichR CMD checkflags.What changes are included in this PR?
The six allocate/delete functions from the issue are exported and documented on one help page
(
?c-data-interface) with a round-trip example; theArraymethods list names$export_to_c()and
$import_from_c(); a NEWS entry; two tests in test-bridge.R. NAMESPACE and the man pages wereregenerated with roxygen2 8.0.0, the version in
Config/roxygen2/version, so the diff is only theadditions. Per @thisisnic's note on #51227, the alignment behaviour of
Array$create()is a codecomment in the new file, not user documentation.
Are these changes tested?
Yes:
test-bridge.Rgains a test that the six names are exported and a test that anArraywitha null round-trips through the exported functions. The example on the help page runs under
R CMD checkas well.Are there any user-facing changes?
Yes: six newly exported functions and one new help page. No existing behaviour changes.
Thank you for looking at this. Happy to adjust the wording of the help page or move the
functions' documentation wherever fits the package's conventions better.
Closes #39793.