Skip to content

perf: share compiled read paths across r/z-variables and CDF v2 files - #76

Merged
Beforerr merged 1 commit into
mainfrom
perf-ttfx
Oct 6, 2026
Merged

Beforerr merged 1 commit into
mainfrom
perf-ttfx

Conversation

@Beforerr

@Beforerr Beforerr commented Oct 6, 2026 •

Copy link
Copy Markdown
Member

Lower time to first read for r-variable files (ACE, OMNI2) and CDF v2 files. Measured in fresh sessions on Julia 1.13.

first use before after
CDF v2 ds[name] (ac_h2_sis) 487 ms 3.6 ms
CDF v2 first attribute access (FILLVAL) 46 ms 0.04 ms
CDF v2 Array (Float32 2-D) 63 ms 4 ms
r-var Float32 1-D: attrib + Array + v[1] + v[:,1:1] ~45 ms ~0.4 ms
r-var Epoch: attrib + Array + v[1] + v[:,1:1] ~100 ms ~17 ms
  • rVDR is merged into VDR, with a new zvar flag. CDFVariable no longer specializes on the variable kind, so r-variables reuse the z-variable specializations.
  • LABL_PTR_* resolution uses Array, which skips DiskArrays indexing. That indexing compiles again for every string width (~60 ms each).
  • The precompile workload now also covers a v2 file (ac_h2_sis), and it compiles Dict(var.attrib) instead of a single get.

Cost: precompile takes about 1 s longer (~3.5 s → 4.5 s), and the package image grows from 8.7 MB to 10.1 MB. Load time is unchanged at ~66 ms.

Remaining cost, not addressed here: every new StaticString{N} width or unseeded element type (e.g. Int32 1-D) still costs ~30–100 ms for DiskArrays indexing (v[1], v[:,1:1]). Array(var) avoids it.

@codecov

codecov Bot commented Oct 6, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@github-actions

github-actions Bot commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Benchmark Results (Julia v1)

Time benchmarks
main b5f40f2... main / b5f40f2...
elx/full_load 23.4 ± 1.2 μs 17.4 ± 1.2 μs 1.34 ± 0.11
elx/sum_array 0.852 ± 0.47 μs 1.09 ± 0.61 μs 0.78 ± 0.61
elx/sum_lazy 1.15 ± 0.24 μs 1.13 ± 0.31 μs 1.02 ± 0.35
elx/sum_var_access 0.912 ± 0.49 μs 0.961 ± 0.6 μs 0.949 ± 0.78
elx/var_access 0.13 ± 0.01 μs 0.13 ± 0.01 μs 1 ± 0.11
epochs/DateTime_to_TT2000 29.2 ± 1.6 μs 29.6 ± 1.5 μs 0.986 ± 0.071
epochs/Epoch16_parse 0.166 ± 0.0058 ms 0.168 ± 0.0042 ms 0.991 ± 0.042
epochs/Epoch16_vs_DateTime 0.772 ± 0.011 μs 0.781 ± 0.01 μs 0.988 ± 0.019
epochs/TT2000_parse 0.189 ± 0.0068 ms 0.192 ± 0.0075 ms 0.987 ± 0.052
epochs/TT2000_string 3.44 ± 1.7 ms 3.36 ± 1.6 ms 1.02 ± 0.69
epochs/TT2000_to_DateTime 0.0566 ± 0.00065 ms 0.0572 ± 0.00053 ms 0.989 ± 0.015
epochs/TT2000_vs_DateTime 25.5 ± 0.12 μs 27.9 ± 0.5 μs 0.915 ± 0.017
mms/full_load 15.2 ± 1.5 μs 13.1 ± 1.4 μs 1.16 ± 0.16
mms/sum_array 0.0617 ± 0.0018 s 0.0608 ± 0.0015 s 1.02 ± 0.039
mms/sum_slice 2.23 ± 0.12 ms 2.22 ± 0.31 ms 1.01 ± 0.15
mms/sum_var_access 2.23 ± 0.13 ms 2.2 ± 0.32 ms 1.02 ± 0.16
mms/var_access 0.06 ± 0.01 μs 0.05 ± 0.01 μs 1.2 ± 0.31
time_to_load 0.116 ± 0.0017 s 0.117 ± 0.01 s 0.989 ± 0.086
Memory benchmarks
main b5f40f2... main / b5f40f2...
elx/full_load 0.217 k allocs: 24.8 kB 0.217 k allocs: 24.8 kB 1
elx/sum_array 12 allocs: 28 kB 12 allocs: 28 kB 1
elx/sum_lazy 12 allocs: 28 kB 12 allocs: 28 kB 1
elx/sum_var_access 14 allocs: 28.3 kB 14 allocs: 28.3 kB 1
elx/var_access 1 allocs: 0.234 kB 1 allocs: 0.234 kB 1
epochs/DateTime_to_TT2000 3 allocs: 0.0764 MB 3 allocs: 0.0764 MB 1
epochs/Epoch16_parse 3 allocs: 0.153 MB 3 allocs: 0.153 MB 1
epochs/Epoch16_vs_DateTime 0 allocs: 0 B 0 allocs: 0 B
epochs/TT2000_parse 3 allocs: 0.0764 MB 3 allocs: 0.0764 MB 1
epochs/TT2000_string 0.2 M allocs: 7.02 MB 0.2 M allocs: 7.02 MB 1
epochs/TT2000_to_DateTime 3 allocs: 0.0764 MB 3 allocs: 0.0764 MB 1
epochs/TT2000_vs_DateTime 0 allocs: 0 B 0 allocs: 0 B
mms/full_load 0.065 k allocs: 6.45 kB 0.065 k allocs: 6.45 kB 1
mms/sum_array 1.04 k allocs: 31.7 MB 1.04 k allocs: 31.7 MB 1
mms/sum_slice 0.069 k allocs: 1.29 MB 0.069 k allocs: 1.29 MB 1
mms/sum_var_access 0.069 k allocs: 1.29 MB 0.069 k allocs: 1.29 MB 1
mms/var_access 1 allocs: 0.219 kB 1 allocs: 0.219 kB 1
time_to_load 0.2 k allocs: 11.8 kB 0.2 k allocs: 11.8 kB 1

@Beforerr
Beforerr merged commit b5f40f2 into main Oct 6, 2026
7 checks passed
@Beforerr
Beforerr deleted the perf-ttfx branch October 6, 2026 13:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant