Repository navigation
_decode_cf_datetime_dtype triggers 2 store reads per time variable — expensive on remote stores #11303
Description
Problem
_decode_cf_datetime_dtype (xarray/coding/times.py:339-371) reads the first and last element of every time-encoded variable during open_dataset / open_datatree to infer the decoded dtype. On local backends these reads are memory-mapped and free; on remote stores (zarr + S3, zarr + icechunk) each read is a round-trip through zarr.sync() → asyncio, even when chunks are already cached in the backend.
Cost scales as 2 × N_time_variables reads per open.
Downstream impact: this hits any workflow that opens many time-encoded groups from remote zarr — radar (CfRadial2: 1 time per sweep, 100+ sweeps per volume), CMIP model archives, satellite time-series, Pangeo-style DataTrees. Interactive notebooks, dashboards, and tile servers pay the cost on every open.
Impact — cProfile on main, 107-group DataTree (nexrad-arco/KLOT icechunk, 106 time variables)
total open_datatree time: 77.56s
| Function | cumtime | share |
|---|---|---|
_decode_cf_datetime_dtype |
35.17s | 45% |
└─ first_n_items |
25.16s | 32% |
└─ last_item |
9.51s | 12% |
└─ decode_cf_datetime (actual decode) |
0.09s | 0.1% |
The CPU cost of CF decoding is negligible. The 35 seconds are 100% I/O round-trip overhead.
Reproducer (public icechunk store, anonymous S3)
import icechunk, xarray as xr
from time import time
storage = icechunk.s3_storage(
bucket="nexrad-arco", prefix="KLOT", region="us-east-1", anonymous=True,
)
repo = icechunk.Repository.open(storage)
session = repo.readonly_session("main")
start = time()
dtree = xr.open_datatree(session.store, engine="zarr", zarr_format=3,
consolidated=False, chunks={})
print(f"open_datatree: {time() - start:.1f}s")Why the reads exist
The comment at times.py:346 (2018) explains: "Verify that at least the first and last date can be decoded successfully. Otherwise, tracebacks end up swallowed by Dataset.__repr__."
Empirically, no longer reproducible on current xarray — modern repr returns "..." for lazy arrays without triggering decode. But there is a second, undocumented purpose: with use_cftime=None and chunked data, decode_cf_datetime may return either datetime64[ns] or object (cftime) depending on whether values overflow the pandas ns range, and dask's map_blocks(func, array, dtype=dtype) casts output to the declared dtype — so a wrong declaration silently corrupts overflow values.
Activity
Prefer metadata over store reads in _decode_cf_datetime_dtype
kmuehlbauer commentedon Apr 17, 2026
@aladinor I'm wondering why times from weather radar would overflow the
ns-range? They shouldn't, at least not for the next 236 years.aladinor commentedon Apr 17, 2026
Hi @kmuehlbauer
Thanks for taking the time to read it, and you are right. Radar data is nowhere near the
datetime64[ns]range limits, and overflow can't happen for it in practice. However, the fix I'm proposing in #11304 reflects that: for backend-backed data (zarr,netCDF4,DAPwrapped inLazilyIndexedArray), it takes the metadata fast path with zero reads, which is the cause of the overflow I am describing in the issue.The only case where the fix still reads
first/lastis a narrow edge: in-memoryint64/float64arrays, novalid_max/time_coverage_endattrs,use_cftime=None,time_unit="ns". That's the combination where the decoder genuinely can't tell from metadata whether values fall in range, and dask'smap_blocks(func, array, dtype=...)would silently cast anobject/cftimeoutput to garbagedatetime64values. Keeping the fallback reads there preserves the existingdecode_cf_datetime_transition_to_invalidbehavior.kmuehlbauer commentedon May 8, 2026
@aladinor Since we can do
decode_times=CFDatetimeCoder()in the call toopen_datasetit would be possible to add a parameter toCFDatetimeCoder.__init__()requesting inference by metadata or by peaking into actual data.Something along the lines of:
That way we would keep fully backwards compatibility without behaviour change. Would this work?
Prefer metadata over store reads in _decode_cf_datetime_dtype
Prefer metadata over store reads in `_decode_cf_datetime_dtype` (#11304)