Render Arrow record batches progressively

[e-kotov]

Arrow.tableFromIPC(response) waits for the complete HTTP response before exposing any record batches, so smaller server-side batches currently create more layers without showing pixels earlier.

This change reads Arrow IPC with RecordBatchReader.from(response.body) and appends one stable Deck.gl layer as each batch arrives. The shared loader keeps the current parse2ArrowTable() path for GeoParquet and FlatGeobuf, including the new upstream FlatGeobuf support.

Verification

• node tests-js/progressive-arrow-batches.test.mjs
• JavaScript syntax checks for every file in inst/htmlwidgets/
• R CMD build .
• R CMD check --no-manual deckglgeoarrow_0.1.0.9001.tar.gz

The JavaScript regression test verifies that the first layer is installed before the second Arrow batch is read and that the non-Arrow parser path remains available. The package check completes with the warnings already present on current main for the removed point-cloud methods and the development geoarrowWidget FlatGeobuf helper.

[e-kotov]

@tim-salabim this is in relation to the batching question, but ignore for now

[tim-salabim]

how did the hackathon go?

[e-kotov]

Hi @tim-salabim ! I switched to other things (mostly GPU related), see SDSL discord for more details, or perhaps it will be on https://spatial-data-science.github.io/2026/ as some point.

For deckglgeoarrow I would ignore zeroserve and just go with your way with temporary arrow files. These files are written crazy fast because there is no reencoding/reformatting/etc process going on on these writes and these are essentially RAM-cached/paged, there is no way to beat it with the kind of serving I do in zeroserve.

However because these files may be quite large and accumulate quickly even in a single R session, I would maybe think of a mechanism to auto-prune them up somehow. If a user 'mapviews' the same huge dataset a few times with deckglgeoarrow and mapgl, they might end up with a 100-400mb duplicates of the same data. 5x400 is already 2GB of temp files... and if R crashes on Linux and macOS these might stay in temp for a few days, and on Windows I am not even sure it cleans up these R temp files on reboots. So a lot of junk may accumulate very quickly and this is not as innocent as a few html files from mapview/leaflet.

Also if the same data is displayed in different layers, but with different display customizations, it would also be duplicated on disk, as every layer of deckglgeoarrow would be dumped to disk. zeroserve could help here, as you could reuse the same url, but in deckglgeoarrow you would then need a 'mode' of some sort (zeroserve vs your file approach) and a mechanism to identify if the user is passing the same dataset in different deckglgeoarrow layers and deduplicate these for them behind the scenes.