feat[next]!: connectivities declared as classes, class-keyed offset providers, table_types; remove FieldOffset - #2910
Draft
egparedes wants to merge 5 commits into
Conversation
egparedes
added this pull request to stack #2900
September 22, 2026 07:00
egparedes
force-pushed
the
connectivities-as-types-6-class-keyed-providers
branch
from
September 23, 2026 16:10
2ec01e1 to
6a5a914
Compare
egparedes
force-pushed
the
connectivities-as-types-6-class-keyed-providers
branch
2 times, most recently
from
September 24, 2026 10:12
0619aca to
4ea2cfd
Compare
egparedes
removed this pull request from stack #2900
September 24, 2026 10:14
egparedes
changed the base branch from
connectivities-as-types-5-shared-locals-in-backends
to
connectivities-as-types-3-neighbor-connectivity
September 24, 2026 10:14
egparedes
added this pull request to stack #2917
September 24, 2026 10:14
…oviders; remove FieldOffset
The tree, the docs and the examples declare their connectivities as
NeighborConnectivity classes, and offset providers are keyed by those
declarations. Every program entry point normalizes a provider to the tag-keyed
form the IR and the backends use (as_tag_keyed_offset_provider), and checks the
tables against their declarations (check_offset_provider). A bare string key is
rejected as the removed FieldOffset spelling.
FieldOffset is removed: unstructured connectivities are NeighborConnectivity
declarations, Cartesian shifts are 'Dim + i', and as_offset takes the dimension
to shift along. Iterator-embedded shift and neighbors accept declarations, and
'DimensionIndex(kind=LOCAL)' is rejected in favour of LocalDimensionIndex.
The types of an offset provider's tables are called 'table_types' everywhere
('offset_provider_type' before): CompileTimeArgs, the compile entry points, the
DaCe orchestration, the IR passes and the tests. Like offset providers, they
are keyed by declarations at the frontend and by tags at the IR level.
scripts/python/migrate_connectivities.py migrates user code, including the
'offset_provider_type' -> 'table_types' rename.
… mesh locations `as_offset(dim, field)` needs index arithmetic like `dim + 1`, so `dim` must be an `AnyCartesianAxisIndex`: a mesh location or a local dimension is a `DSLError` in a field operator and a `TypeError` in embedded execution. The builtin's signature says so, and builtin signatures accept a `type[...]` narrowed to a level of the dimension hierarchy. The migration script decides per declaration whether a dimension is a `CartesianAxisIndex` (kind=VERTICAL, a Cartesian `FieldOffset`, `as_offset`, index arithmetic, or a staggered counterpart) or a mesh location (a dimension of a neighbor `FieldOffset`), and reports the dimensions with no evidence or with conflicting evidence. It also drops module-level aliases of the removed `DimensionKind.LOCAL` and reports its other uses.
…an no longer express
egparedes
force-pushed
the
connectivities-as-types-6-class-keyed-providers
branch
from
October 2, 2026 02:14
4ea2cfd to
750dc83
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #2907. Part 3 of 4 of the connectivities as types stack, and the only breaking PR (ADR 0030, updated here; the
FieldOffsetpart of ADR 0019 is superseded).What
toy_connectivity.py,cases_utils.py,fvm_nabla_setup.py, the unit tests that declared their own offsets, the Quickstart, the workshop and the examples. A local dimension subclassesLocalDimensionIndex(feat[next]: NeighborConnectivity[Domain, Codomain] declarations, NeighborTableType, and shared local dimensions on every backend #2907 removedDimensionKind.LOCAL).common.as_tag_keyed_offset_provider:Program.__call__,FieldOperator.__call__,FieldOperatorFromFoast.__call__,compileandCompilationOptions.connectivitiesstrictly, andembedded.context.update, the iteratorfendefand DaCe'sget_sdfg_conn_argsnon-strictly. Each declaration becomes itsoffset_tag, so lowering, the backends and compiled-program caching keep seeing tag-keyed providers, like hand-written IR. A bare string key such as"V2E"is rejected at the strict entry points as the removed spelling; the IR-level hooks accept any string, since the IR names offsets by string.common.check_offset_provider) at every entry point. The result is memoized per set of bound tables (hashed by theirid, as the compiled-program cache keys its variants), so a repeated call costs one hash. Reading the tables -- comparing the skip positions of tables over one shared local dimension -- happens only where a program is compiled (deep=True). A key that names the connectivity instead of the declaration ({V2E.tag: table}) is rejected with a pointer to{V2E: table}.FieldOffsetis removed, along withis_cartesian_offsetand theFieldOffsetspecial cases in grid-type deduction, gtfn and embeddedpremap. A declaration's__gt_type__(ats.ShiftType) andbound_table()are implemented directly. Iterator-embeddedshift/neighborsand tracing accept a declaration.as_offset(dim, field)takes the Cartesian axis to shift along (type[AnyCartesianAxisIndex]): likedim + 1it needs index arithmetic, so a mesh location or a local dimension is aDSLErrorin a field operator and aTypeErrorin embedded execution.offset_provider_typebecomestable_typeseverywhere:CompileTimeArgs.table_types, the gtfn and DaCe compile entry points, the DaCe orchestration, the IR passes and type inference, and the tests. Table types are keyed and normalized like an offset provider: by declarations (strictly) at the frontend, by tags at the IR level.compile(offset_provider=...)acceptsTableTypesLikeas well as tables, e.g.{V2E: NeighborTableType(connectivity=V2E, dtype=..., skip_value=None, max_neighbors=6)}for ahead-of-time compilation.OffsetProvider/TableTypesstay tag-keyed;OffsetProviderLike/TableTypesLikeare what users pass.Breaking changes
FieldOffsetis removed: declare aNeighborConnectivity; a Cartesian shift isDim + i.{V2E: table}); a bare string key raises.as_offsettakes a Cartesian axis, not a CartesianFieldOffset.offset_provider_typeis renamedtable_types(keyword arguments,CompileTimeArgs.table_types).commonand the type system:NeighborConnectivityType->NeighborTableType(with another shape),OffsetProviderType->TableTypes,ts.OffsetType(source, target)->ts.ShiftType(codomain, domain);DimensionKind.LOCALis removed (subclassLocalDimensionIndex; testcommon.is_local_dimension(dim)).CartesianAxisIndex; index arithmetic (D + 1) andStaggered[D]on any other dimension raise.Migration
scripts/python/migrate_connectivities.py(./scripts/run migrate-connectivities <paths> [--write]):Dimension(...)declarations to classes, deciding per declaration between a Cartesian axis (CartesianAxisIndex:kind=VERTICAL, a CartesianFieldOffset,as_offset, index arithmetic, or a staggered counterpart) and a mesh location (DimensionIndex: a dimension of a neighborFieldOffset). A dimension with no evidence, or with evidence for both, is declaredDimensionIndexand reported.DimensionKind.LOCALand reports its other uses.FieldOffsets toNeighborConnectivityclasses that adopt their existing local dimension (Local: typing.TypeAlias = E2CDim). That keeps local-dimension names working, and covers offsets that share a local dimension (C2CE).FieldOffsets and rewrites their uses (Koff[1]->KDim + 1,as_offset(Koff, …),dims.Koff, and the imports).offset_provider_type=keywords and.offset_provider_typeattributes totable_types, andOffsetProviderType/is_offset_provider_typetoTableTypes/is_table_types. A keyword is left alone, and reported, in a module that defines its own parameter of that name..valueon dimensions, andisinstance(…, Dimension).The counts depend on the ICON4Py revision, so here is a pinned one: a dry run at ICON4Py
89b4967(2026-09-21), by which point ICON4Py had already droppedKoff[1], rewrites 4 files (dimension.pyand 3 stencil modules). It declaresKDimaCartesianAxisIndexandEdgeDim/CellDim/VertexDimDimensionIndex, with no dimension left undecided, and reports 46 connectivity provider keys, 2 removable Cartesian provider entries, 5isinstance(…, Dimension)sites in 2 modules and 6 uses ofDimensionKind.LOCALin 5 modules to do by hand.Tests
cases_utilsmeshes) are class-keyed; theirtable_typesare tag-keyed, the form the IR-level APIs take.FieldOffsetregression tests become tests of reaching a declaration through other Python names.table_typesrename.Verification
Run locally (CPU) on this head; 0 failures in every row that ran:
test_next(internal, cpu, nomesh): embedded, roundtrip, gtfntest_next(dace, cpu, nomesh)src/gt4py/nextdoctests (in each session)test_typing_exports(mypy cases, pyright probes)pre-commit run --all-filesGitHub CI on the pushed head passes every check:
test_next(internal and dace; nomesh and atlas) on 3.12 and 3.14,test_cartesian,test_eve,test_storage, notebooks, package,typing-exportsand code quality. The local 3.13/3.14 runs were stopped in favour of CI.