Skip to content

feat[cartesian]: Loops over data dimensions - #2686

Open
stubbiali wants to merge 62 commits into
GridTools:mainfrom
stubbiali:loops-over-data-dims
Open

feat[cartesian]: Loops over data dimensions#2686
stubbiali wants to merge 62 commits into
GridTools:mainfrom
stubbiali:loops-over-data-dims

Conversation

@stubbiali

@stubbiali stubbiali commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

This PR prototypes for-loops over data (i.e. static-sized) dimensions, easing the implementation of compute patterns ubiquitous in radiation and land surface parametrizations.

  • Support is implemented for all backends.
  • Allow calling functions from withing a for-loop and writing a for-loop within a function.
  • Because fields with data dimensions must be allocated upfront, functions can now modify fields in-place and not return anything, so avoiding aliasing/copying larger high-dimensional arrays.

TODO:

  • Unit tests

Ideas:

  • Add new concept for functions without any return statement
  • Allow defining higher-dimensional temporaries inside stencils

stubbiali and others added 30 commits October 3, 2024 18:14
@stubbiali
Add prototype support for for loops over data dims.
c247d41
@stubbiali
Merge remote-tracking branch 'upstream/main' into data-dim-loops
2529e02
@stubbiali
Add support for lists and tuples.
cc2def3
@stubbiali
Cosmetics.
e79917a
@stubbiali
Allow field data indices.
0d321ec
@stubbiali
Call CallInliner before DataDimLoopUnroller.
0856861
@FlorianDeconinck @stubbiali
Casting to INT. Add v_in_int = int(v_in_float) op as a base unitary…
4847fb4 
… op. Deactivate upcaster for thi cast call.
Florian Deconinck @stubbiali
Lint
8f5fe95
@FlorianDeconinck @stubbiali
Native function: f32, f64, round
a70a46c
@stubbiali
Enable for-loops in functions.
1d7cbad
@stubbiali
Enable functions with no return statements.
511222a
@stubbiali
Allow assigning call arguments inside functions.
45363a3
@stubbiali
Add inlining of constant function arguments.
7015268
@stubbiali
Inline constant function arguments recursively.
d151a78
@stubbiali
Properly support for-loops around functions.
10c1bfb
@stubbiali
Merge remote-tracking branch 'upstream/main' into ecrad
90743af
@stubbiali
Improve support for global tables in numpy generated code.
c811ba5
@stubbiali
Merge remote-tracking branch 'upstream/main' into ecrad
5bd2ec2
@stubbiali
Fully inline constant arguments.
878fd35
@stubbiali
Merge remote-tracking branch 'upstream/main' into ecrad
631ed51
@stubbiali
Prototype implementation of reductions.
4f25fa3
@stubbiali
Fix numpy codegen.
a75a0de
@stubbiali
Fix "TypeError: : cannot pickle 'PyCapsule' object".
01e7555
@stubbiali
Add custom AST nodes.
1230920
@stubbiali
Add For and ForIndex defir nodes.
a5398fb
@stubbiali
Add custom For and ForIndex gtir nodes.
1e65bd5
@stubbiali
Refactor common ForIndex.
d13a8ab
@stubbiali
Add For and ForIndex oir nodes.
c8dd831
@stubbiali
Add For and ForIndex gtcpp nodes.
6ed7ddd
@stubbiali
Support For and ForIndex in codegen.
8570ce1
stubbiali and others added 30 commits May 23, 2025 13:49
@stubbiali
Add support for for-loops in dace.
9af5a5c
@stubbiali
Fix single precision.
011e3d4
@stubbiali
Fix cuda extra compile args
57d7f5f
@stubbiali
Add support for for-loops in numpy backend
36c24fa
@stubbiali
Merge remote-tracking branch 'upstream/main' into loops-over-data-dim…
a1de48f 
…s-on-main

# Conflicts:
#	src/gt4py/cartesian/config.py
#	src/gt4py/cartesian/frontend/defir_to_gtir.py
#	src/gt4py/cartesian/frontend/gtscript_frontend.py
#	src/gt4py/cartesian/frontend/nodes.py
#	src/gt4py/cartesian/gtc/common.py
#	src/gt4py/cartesian/gtc/cuir/cuir_codegen.py
#	src/gt4py/cartesian/gtc/dace/daceir.py
#	src/gt4py/cartesian/gtc/dace/expansion/daceir_builder.py
#	src/gt4py/cartesian/gtc/dace/expansion/tasklet_codegen.py
#	src/gt4py/cartesian/gtc/dace/utils.py
#	src/gt4py/cartesian/gtc/gtcpp/gtcpp_codegen.py
#	src/gt4py/cartesian/gtc/numpy/npir.py
#	src/gt4py/cartesian/gtc/numpy/npir_codegen.py
#	src/gt4py/cartesian/gtc/numpy/oir_to_npir.py
#	src/gt4py/cartesian/gtc/passes/gtir_upcaster.py
#	src/gt4py/cartesian/gtc/ufuncs.py
#	src/gt4py/cartesian/gtscript.py
@gvollenweider
avoid int overflow for arrays with many gridpoints
cfa85f4
@stubbiali
Merge remote-tracking branch 'upstream/main' into loops-over-data-dims
d52cb9a
@stubbiali
Merge remote-tracking branch 'upstream/main' into loops-over-data-dims
3520e9e
@stubbiali
Merge remote-tracking branch 'upstream/main' into loops-over-data-dims
ef219b6
@stubbiali
Merge remote-tracking branch 'upstream/main' into loops-over-data-dims
a61087d 
# Conflicts:
#	src/gt4py/cartesian/frontend/gtscript_frontend.py
@stubbiali
Pass extra cxx compile args to cuda/rocm compiler
41300dc
@stubbiali
Revert "feat[cartesian]: DaCe optimal for/map schedule (GridTools#2628)"
e5805fd 
This reverts commit 20d1bea.

# Conflicts:
#	src/gt4py/cartesian/gtc/dace/oir_to_treeir.py
#	tests/cartesian_tests/integration_tests/multi_feature_tests/test_code_generation.py
@stubbiali
Revert "refactor[cartesian]: update treeir representation of the vert…
b04e352 
…ical loop (GridTools#2688)"

This reverts commit bd1e156.

# Conflicts:
#	src/gt4py/cartesian/gtc/dace/oir_to_treeir.py
#	src/gt4py/cartesian/gtc/dace/treeir.py
#	tests/cartesian_tests/integration_tests/multi_feature_tests/test_code_generation.py
@stubbiali
Add gt4py and dace version to stencil fingerprint
62b0896
@stubbiali
Merge remote-tracking branch 'upstream/main' into loops-over-data-dims
f6d0c10
@stubbiali
Fix vertical loop index symbol
06160a4
@stubbiali
Try moving vertical loop inside the kernel
25f7d66
@stubbiali
Revert "Try moving vertical loop inside the kernel"
29d4201 
This reverts commit 25f7d66.
@stubbiali
Reapply "refactor[cartesian]: update treeir representation of the ver…
bd91ef7 
…tical loop (GridTools#2688)"

This reverts commit b04e352.
@stubbiali
Reapply "feat[cartesian]: DaCe optimal for/map schedule (GridTools#2628…
00e9999 
…)"

This reverts commit e5805fd.

# Conflicts:
#	src/gt4py/cartesian/gtc/dace/oir_to_treeir.py
@stubbiali
Parametrize parallel vertical loop schedule
5ad3321
@stubbiali
Apply map collapse transformation to sdgf
e031aa7
@stubbiali
Fix typo
b41448f
@stubbiali
Remove debug code
92923b1
@stubbiali
Merge remote-tracking branch 'origin/fix-dace-perf' into loops-over-d…
c02a484 
…ata-dims
@stubbiali
Merge remote-tracking branch 'upstream/main' into fix-dace-perf
9d6749f
@stubbiali
Do not validate map-collapse pass
b9926f4
@stubbiali
Fix unit tests
19686d1
@stubbiali
Merge remote-tracking branch 'origin/fix-dace-perf' into loops-over-d…
82796c8 
…ata-dims
@stubbiali
Merge remote-tracking branch 'upstream/main' into loops-over-data-dims
dca1458

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@stubbiali @FlorianDeconinck @gvollenweider