Skip to content

GH-39793: [R] Export the C Data Interface allocators and document $export_to_c() / $import_from_c() - #51236

Open
singhpratech wants to merge 2 commits into
apache:mainfrom
singhpratech:GH-39793-r-export-c-data-interface
Open

singhpratech wants to merge 2 commits into
apache:mainfrom
singhpratech:GH-39793-r-export-c-data-interface

Conversation

@singhpratech

@singhpratech singhpratech commented Sep 8, 2026 •

Copy link
Copy Markdown
Contributor

Rationale for this change

Follows up on #39793. Array$export_to_c() and Array$import_from_c() work, but the functions that
produce the pointers they take, allocate_arrow_schema() and friends, are not exported, and none of
it has a help page, so a package that hands a column to another library over the C Data Interface
has to use arrow:::, which R CMD check flags.

What changes are included in this PR?

The six allocate/delete functions from the issue are exported and documented on one help page
(?c-data-interface) with a round-trip example; the Array methods list names $export_to_c()
and $import_from_c(); a NEWS entry; two tests in test-bridge.R. NAMESPACE and the man pages were
regenerated with roxygen2 8.0.0, the version in Config/roxygen2/version, so the diff is only the
additions. Per @thisisnic's note on #51227, the alignment behaviour of Array$create() is a code
comment in the new file, not user documentation.

Are these changes tested?

Yes: test-bridge.R gains a test that the six names are exported and a test that an Array with
a null round-trips through the exported functions. The example on the help page runs under
R CMD check as well.

Are there any user-facing changes?

Yes: six newly exported functions and one new help page. No existing behaviour changes.

Thank you for looking at this. Happy to adjust the wording of the help page or move the
functions' documentation wherever fits the package's conventions better.

Closes #39793.

…nt $export_to_c() / $import_from_c()

allocate_arrow_schema(), delete_arrow_schema(), allocate_arrow_array(), delete_arrow_array(),
allocate_arrow_array_stream() and delete_arrow_array_stream() are now exported, with a help page
(?c-data-interface) that shows the round trip, and the Array methods list names
$export_to_c() and $import_from_c(). Two tests cover the exports and an Array round trip through
the exported functions. Closes apache#39793.
singhpratech added a commit to singhpratech/ArrowMetal that referenced this pull request Sep 8, 2026
…Data Interface exports

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQYE6hSNYZur5fRZC3mqGT
@github-actions

github-actions Bot commented Sep 8, 2026

Copy link
Copy Markdown

⚠️ GitHub issue #39793 has been automatically assigned in GitHub to PR creator.

Comment on lines +85 to +97
test_that("the C Data Interface allocators are exported", {
exported <- getNamespaceExports("arrow")
for (f in c(
"allocate_arrow_schema",
"delete_arrow_schema",
"allocate_arrow_array",
"delete_arrow_array",
"allocate_arrow_array_stream",
"delete_arrow_array_stream"
)) {
expect_true(f %in% exported, label = paste(f, "is exported"))
}
})

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the reason for this test?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It pins the six names as part of the exported interface, so a later roxygen run that loses an @export fails here, naming the function, rather than in a downstream package's check. The round-trip test below reaches them through arrow:: and would fail too, so if one test is enough I'll drop this one and keep the round trip.

Comment thread r/R/array.R
#' - `$export_to_c(array_ptr, schema_ptr)`: Fill an `ArrowArray` and an `ArrowSchema` struct
#' from this array for the Arrow C Data Interface; the pointers come from
#' [allocate_arrow_array()] and [allocate_arrow_schema()].
#' - `Array$import_from_c(array_ptr, schema_ptr)`: Build an `Array` from structs

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This last one looks different

@github-actions github-actions Bot added awaiting changes Awaiting changes and removed awaiting review Awaiting review labels Sep 16, 2026
@thisisnic

Copy link
Copy Markdown
Member

Before we consider merging this, I think it'd be good to understand why we would want to do this instead of recommending folk use nanoarrow for this?

@singhpratech

Copy link
Copy Markdown
Contributor Author

The methods are the public part already: $export_to_c() and $import_from_c() are what nanoarrow itself calls (as_nanoarrow_array.Array hands x$export_to_c() a nanoarrow_allocate_array(), and as_arrow_array.nanoarrow_array calls arrow::Array$import_from_c()), and 25.0.0 has no help page for either. The six allocators already exist in the package for the Python bridge. The change exports them and documents the methods so a package that already depends on arrow can call arrow's own methods without arrow::: (which R CMD check flags) or a second dependency for a struct allocation. That is the whole case, and it is a small one.

The case for nanoarrow instead is that it is the project's answer for the C Data Interface in R, one route is better than two, and the use case on the issue (r-polars) has since gone through nanoarrow. If that is where you would rather keep it, I will cut this to the documentation: the two methods on ?Array, with the example allocating through nanoarrow, and drop the exports, the new test and the NEWS entry. Your call.

singhpratech added a commit to singhpratech/ArrowMetal that referenced this pull request Sep 16, 2026
@thisisnic

thisisnic commented Sep 30, 2026 •

Copy link
Copy Markdown
Member

I think it makes more sense to keep nanoarrow as the canonical way to do this, yeah, thanks. Pinging @paleolimbot here too - what are your thoughts Dewey?

@paleolimbot

Copy link
Copy Markdown
Member

Thanks for the ping!

I'm an obvious fan of nanoarrow...the pointer functions there are hopefully well documented and help you get/work with the addresses in various ways that aren't exposed here. If for some reason you don't want that as a runtime dependency, you can also inline the allocators and deleters (by copying this header or adding nanoarrow to LinkingTo and using it directly): https://github.com/apache/arrow-nanoarrow/blob/main/r/inst/include/nanoarrow/r.h ).

That said, documenting import_from_c() and export_to_c() I think is reasonable. They have been around for a long time and aren't likely to change as far as I know.

@singhpratech

Copy link
Copy Markdown
Contributor Author

Thank you both for taking the time, and Dewey, thanks for the nanoarrow/r.h pointer — inlining the
allocators that way is a clean answer for a package that cannot take a dependency, and I will use it.

I am glad to go whichever way the project prefers. I would just like to put a few facts next to the
decision, since they shaped this PR and may be useful whoever reads #39793 next:

  • The project's own Python ↔ R integration guide (docs/source/python/integration/python_r.rst)
    shows R code calling Array$import_from_c(array_ptr, schema_ptr) and Array$export_to_c(...).
    Today the R package has no help page for either method, and ?Array does not list them.
  • The six allocators are already in the package: r/R/python.R uses them for the reticulate bridge.
    Exporting them is +170 / −0 across eight files and changes no behaviour.
  • [R] Export functions for low-level pointer operations #39793 has been open since January 2024 and was un-staled in November 2025, when a maintainer asked
    on the issue whether the reporter would open a PR; this PR is that follow-through.
  • nanoarrow's own R package calls Array$import_from_c() and $export_to_c(), so those two methods
    are public API in practice already.

Where I agree completely: for working with the C structures, nanoarrow is the right tool and the
right recommendation. The case this PR was aimed at is narrower — a package that already depends on
arrow and only needs to pass one array across the interface — and for that case the choice today is
arrow::: (which R CMD check flags) or a second dependency to allocate two small structs.

Two ways to close this out, and I am happy with either:

  1. Documentation only. I trim the PR to documenting $export_to_c() and $import_from_c() on
    ?Array, with the example allocating through nanoarrow and a line pointing readers to nanoarrow
    for everything beyond passing a pointer; the exports, the test and the NEWS entry go. If that is
    the decision, closing [R] Export functions for low-level pointer operations #39793 with that reason would spare the next person the same loop.
  2. As filed. Export the six functions as the issue asks.

Tell me which, and I will have the branch updated the same day. Thanks again for the reviews.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[R] Export functions for low-level pointer operations

3 participants