Skip to content

feat[next]!: connectivities declared as classes, class-keyed offset providers, table_types; remove FieldOffset - #2910

Draft
egparedes wants to merge 5 commits into
connectivities-as-types-3-neighbor-connectivityfrom
connectivities-as-types-6-class-keyed-providers
Draft

egparedes wants to merge 5 commits into
connectivities-as-types-3-neighbor-connectivityfrom
connectivities-as-types-6-class-keyed-providers

Conversation

@egparedes

@egparedes egparedes commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #2907. Part 3 of 4 of the connectivities as types stack, and the only breaking PR (ADR 0030, updated here; the FieldOffset part of ADR 0019 is superseded).

What

class V2E(gtx.NeighborConnectivity[Vertex, Edge]):
    class Local(gtx.LocalDimensionIndex): ...

program(..., offset_provider={V2E: v2e_table})  # was {"V2E": v2e_table}
a(KDim + 1)                                     # was a(Koff[1])
a(as_offset(KDim, k_offsets))                   # was as_offset(Koff, k_offsets)
  • The tree declares its connectivities as classes: toy_connectivity.py, cases_utils.py, fvm_nabla_setup.py, the unit tests that declared their own offsets, the Quickstart, the workshop and the examples. A local dimension subclasses LocalDimensionIndex (feat[next]: NeighborConnectivity[Domain, Codomain] declarations, NeighborTableType, and shared local dimensions on every backend #2907 removed DimensionKind.LOCAL).
  • Offset providers are keyed by the declaration. Every entry point normalizes the provider with common.as_tag_keyed_offset_provider: Program.__call__, FieldOperator.__call__, FieldOperatorFromFoast.__call__, compile and CompilationOptions.connectivities strictly, and embedded.context.update, the iterator fendef and DaCe's get_sdfg_conn_args non-strictly. Each declaration becomes its offset_tag, so lowering, the backends and compiled-program caching keep seeing tag-keyed providers, like hand-written IR. A bare string key such as "V2E" is rejected at the strict entry points as the removed spelling; the IR-level hooks accept any string, since the IR names offsets by string.
  • Tables are checked against their declarations (common.check_offset_provider) at every entry point. The result is memoized per set of bound tables (hashed by their id, as the compiled-program cache keys its variants), so a repeated call costs one hash. Reading the tables -- comparing the skip positions of tables over one shared local dimension -- happens only where a program is compiled (deep=True). A key that names the connectivity instead of the declaration ({V2E.tag: table}) is rejected with a pointer to {V2E: table}.
  • FieldOffset is removed, along with is_cartesian_offset and the FieldOffset special cases in grid-type deduction, gtfn and embedded premap. A declaration's __gt_type__ (a ts.ShiftType) and bound_table() are implemented directly. Iterator-embedded shift/neighbors and tracing accept a declaration.
  • as_offset(dim, field) takes the Cartesian axis to shift along (type[AnyCartesianAxisIndex]): like dim + 1 it needs index arithmetic, so a mesh location or a local dimension is a DSLError in a field operator and a TypeError in embedded execution.
  • offset_provider_type becomes table_types everywhere: CompileTimeArgs.table_types, the gtfn and DaCe compile entry points, the DaCe orchestration, the IR passes and type inference, and the tests. Table types are keyed and normalized like an offset provider: by declarations (strictly) at the frontend, by tags at the IR level. compile(offset_provider=...) accepts TableTypesLike as well as tables, e.g. {V2E: NeighborTableType(connectivity=V2E, dtype=..., skip_value=None, max_neighbors=6)} for ahead-of-time compilation.
  • Types: the internal OffsetProvider / TableTypes stay tag-keyed; OffsetProviderLike / TableTypesLike are what users pass.

Breaking changes

Migration

scripts/python/migrate_connectivities.py (./scripts/run migrate-connectivities <paths> [--write]):

  • Rewrites Dimension(...) declarations to classes, deciding per declaration between a Cartesian axis (CartesianAxisIndex: kind=VERTICAL, a Cartesian FieldOffset, as_offset, index arithmetic, or a staggered counterpart) and a mesh location (DimensionIndex: a dimension of a neighbor FieldOffset). A dimension with no evidence, or with evidence for both, is declared DimensionIndex and reported.
  • Drops aliases of the removed DimensionKind.LOCAL and reports its other uses.
  • Rewrites FieldOffsets to NeighborConnectivity classes that adopt their existing local dimension (Local: typing.TypeAlias = E2CDim). That keeps local-dimension names working, and covers offsets that share a local dimension (C2CE).
  • Removes Cartesian FieldOffsets and rewrites their uses (Koff[1] -> KDim + 1, as_offset(Koff, …), dims.Koff, and the imports).
  • Renames offset_provider_type= keywords and .offset_provider_type attributes to table_types, and OffsetProviderType / is_offset_provider_type to TableTypes / is_table_types. A keyword is left alone, and reported, in a module that defines its own parameter of that name.
  • Reports what it cannot rewrite from the source: string provider keys, .value on dimensions, and isinstance(…, Dimension).

The counts depend on the ICON4Py revision, so here is a pinned one: a dry run at ICON4Py 89b4967 (2026-09-21), by which point ICON4Py had already dropped Koff[1], rewrites 4 files (dimension.py and 3 stencil modules). It declares KDim a CartesianAxisIndex and EdgeDim/CellDim/VertexDim DimensionIndex, with no dimension left undecided, and reports 46 connectivity provider keys, 2 removable Cartesian provider entries, 5 isinstance(…, Dimension) sites in 2 modules and 6 uses of DimensionKind.LOCAL in 5 modules to do by hand.

Tests

  • Shared fixtures (cases_utils meshes) are class-keyed; their table_types are tag-keyed, the form the IR-level APIs take.
  • The FieldOffset regression tests become tests of reaching a declaration through other Python names.
  • Unit tests cover normalization, bare-key and own-tag-key rejection, double binding, table checks (sharers included), deep vs shallow checking, memoization, the error when a provider is missing an entry, and checking table types given without tables.
  • Migration script tests include ICON4Py's actual shapes and the table_types rename.

Verification

Run locally (CPU) on this head; 0 failures in every row that ran:

Check 3.12 3.13 3.14
test_next (internal, cpu, nomesh): embedded, roundtrip, gtfn 5066 passed not run passed (CI)
test_next (dace, cpu, nomesh) 1511 passed not run passed (CI)
src/gt4py/next doctests (in each session) 119 passed
test_typing_exports (mypy cases, pyright probes) 20 passed, pyright 0 errors 20 passed, pyright 0 errors 20 passed, pyright 0 errors
migration-script tests 9 passed
pre-commit run --all-files clean

GitHub CI on the pushed head passes every check: test_next (internal and dace; nomesh and atlas) on 3.12 and 3.14, test_cartesian, test_eve, test_storage, notebooks, package, typing-exports and code quality. The local 3.13/3.14 runs were stopped in favour of CI.

@egparedes
egparedes added this pull request to stack #2900 September 22, 2026 07:00
@egparedes
egparedes force-pushed the connectivities-as-types-6-class-keyed-providers branch from 2ec01e1 to 6a5a914 Compare September 23, 2026 16:10
@egparedes
egparedes force-pushed the connectivities-as-types-6-class-keyed-providers branch 2 times, most recently from 0619aca to 4ea2cfd Compare September 24, 2026 10:12
@egparedes
egparedes removed this pull request from stack #2900 September 24, 2026 10:14
@egparedes
egparedes changed the base branch from connectivities-as-types-5-shared-locals-in-backends to connectivities-as-types-3-neighbor-connectivity September 24, 2026 10:14
@egparedes
egparedes added this pull request to stack #2917 September 24, 2026 10:14
@egparedes egparedes changed the title feat[next]!: class-keyed offset providers; remove FieldOffset feat[next]!: connectivities declared as classes, class-keyed offset providers, table_types; remove FieldOffset Sep 24, 2026
…oviders; remove FieldOffset

The tree, the docs and the examples declare their connectivities as
NeighborConnectivity classes, and offset providers are keyed by those
declarations. Every program entry point normalizes a provider to the tag-keyed
form the IR and the backends use (as_tag_keyed_offset_provider), and checks the
tables against their declarations (check_offset_provider). A bare string key is
rejected as the removed FieldOffset spelling.

FieldOffset is removed: unstructured connectivities are NeighborConnectivity
declarations, Cartesian shifts are 'Dim + i', and as_offset takes the dimension
to shift along. Iterator-embedded shift and neighbors accept declarations, and
'DimensionIndex(kind=LOCAL)' is rejected in favour of LocalDimensionIndex.

The types of an offset provider's tables are called 'table_types' everywhere
('offset_provider_type' before): CompileTimeArgs, the compile entry points, the
DaCe orchestration, the IR passes and the tests. Like offset providers, they
are keyed by declarations at the frontend and by tags at the IR level.

scripts/python/migrate_connectivities.py migrates user code, including the
'offset_provider_type' -> 'table_types' rename.
… mesh locations

`as_offset(dim, field)` needs index arithmetic like `dim + 1`, so `dim` must be
an `AnyCartesianAxisIndex`: a mesh location or a local dimension is a `DSLError`
in a field operator and a `TypeError` in embedded execution. The builtin's
signature says so, and builtin signatures accept a `type[...]` narrowed to a
level of the dimension hierarchy.

The migration script decides per declaration whether a dimension is a
`CartesianAxisIndex` (kind=VERTICAL, a Cartesian `FieldOffset`, `as_offset`,
index arithmetic, or a staggered counterpart) or a mesh location (a dimension
of a neighbor `FieldOffset`), and reports the dimensions with no evidence or
with conflicting evidence. It also drops module-level aliases of the removed
`DimensionKind.LOCAL` and reports its other uses.
@egparedes
egparedes force-pushed the connectivities-as-types-6-class-keyed-providers branch from 4ea2cfd to 750dc83 Compare October 2, 2026 02:14

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant