Upgrade Python to 3.14 and all dependent libraries by jomey · Pull Request #87 · iSnobal/smrf

jomey · 2026-10-22T14:08:27Z

This upgrades SMRF to run with Pyton 3.14, Numpy 2.5, and Pandas 3. The changes are small and all tests pass without updating gold files. All files that were touched also got a little syntax polish with the Ruff linter.

This should be merged after we cut a last release with the current Python 3.9 version. Then give this a good spin and see what broke that tests did not show.

Code behavior changes

The only logic that was changed was to twostream that now enforces to have the tau parameter always as an array. Before it also accepted a float, which made the code more complicated than needed.

Reduce the external package dependecies and copy the few methods over. The ARS package is no longer maintained and it is one less library that was only tested with old Python versions. Closes #12

Explictly set method argument values when combining datasets to the current defaults so there are no changes with future library upgrades.

np.Inf was changed to np.inf

Instead of returning an array with one element, directly grab the first index element.

Pandas no longer silently casts these values and raises an error. This ensures current behavior going forward.

`iteritems` was deprecated and is now either `items` or just the index.

Averaging float32 coordinates loses precision and varies with numpy version. Cast to float64 in get_center and update the expected test values (center x, basin longitude) to the exact results. Analyzed with the help of Copilot.

Backslashes cause conflicts.

These are now all listed as of 202610 with Python 3.14

Copilot

🟡 Changes recommended

Critical compatibility issues remain in the documentation environment and nighttime solar handling.

2 open issues

What changed in this PR

This pull request upgrades SMRF for Python 3.14 and newer NumPy, Pandas, SciPy, and Xarray releases while modernizing compatibility-related code.

Changes:

Updates Python and dependency requirements in pyproject.toml.
Replaces deprecated or private scientific-Python APIs.
Adds local NetCDF projection utilities.
Updates solar, topology, data-loading, output, and formatting code.

File	Description	Review notes
`smrf/utils/utils.py`	Uses public SciPy interpolation imports.	—
`smrf/utils/spatialnc.py`	Adds local NetCDF projection utilities.	—
`smrf/tests/data/test_topo.py`	Updates topology expectations and formatting.	—
`smrf/spatial/grid.py`	Replaces a private SciPy coordinate helper.	—
`smrf/output/output_netcdf.py`	Uses local projection utilities and modernized syntax.	—
`smrf/framework/ascii_art.py`	Cleans up ASCII-art string literals.	—
`smrf/envphys/storms.py`	Updates Pandas iteration and syntax.	—
`smrf/envphys/solar/model.py`	Adjusts solar model result handling.	Critical (4 votes): nighttime scalar-zero results can cause indexing errors.
`smrf/envphys/solar/cloud.py`	Adapts cloud calculations to solar output changes.	—
`smrf/distribute/wind/winstral.py`	Modernizes NumPy, Pandas, and exception usage.	—
`smrf/distribute/variable_base.py`	Replaces deprecated NumPy aliases and updates assignments.	—
`smrf/data/load_topo.py`	Updates coordinate precision and exception handling.	—
`smrf/data/hrrr/grib_file_xarray.py`	Updates Xarray combination arguments.	—
`pyproject.toml`	Raises the minimum Python version and updates dependencies.	Critical (1 vote): the Read the Docs Python environment remains incompatible with the new package requirement.

_{Review effort: Lite}

No logic change.

The twostream method now consistently returns arrays so the change to model.py can index the return results no matter if the sun is up or down.

Reduce the complexity of the method signature and always expect an array. The only place where this was called as a float was in the model.py This also consolidates the tests written for twostream.

jomey · 2026-10-22T16:46:01Z

        try:
            variable_data = xr.combine_by_coords(
-                variable_data, combine_attrs='drop'
+                variable_data, combine_attrs="drop", compat="no_conflicts", join="outer"


Addresses warning and enforces same behavior going forward.

jomey · 2026-10-22T16:46:44Z

-        x = ds.variables["x"][:]
-        y = ds.variables["y"][:]
+        x = ds.variables["x"][:].astype(np.float64)
+        y = ds.variables["y"][:].astype(np.float64)


Increase to float64 so the mean calculation is more precise. Made the tests fail.

jomey · 2026-10-22T16:47:21Z

                    flatwind.loc[m] = data_speed[m] / expVal
            else:
-                flatwind.loc[m] = np.NaN
+                flatwind.loc[m] = np.nan


Main change with outdated syntax.

jomey · 2026-10-22T16:47:35Z

-            self.min = self.config.get("min", -np.Inf)
-            self.max = self.config.get("max", np.Inf)
+            self.min = self.config.get("min", -np.inf)
+            self.max = self.config.get("max", np.inf)


Update to latest syntax

jomey · 2026-10-22T16:48:52Z

        # get solar using twostream
        dtt = pd.to_datetime(dt)
-        basin_sol.iloc[idt, :] = model_solar(dtt, lat, lon)
+        basin_sol.iloc[idt, :] = np.float32(model_solar(dtt, lat, lon))


Pandas now enforces casting on the user side

jomey · 2026-10-22T16:49:26Z

+def twostream(
+    cos_z: float,
+    S0: float,
+    tau: npt.NDArray[np.float64],


Enforce tau as arrays going forward to simplify method signature

jomey · 2026-10-22T16:49:55Z

+            "direct_transmittance": zeros.copy(),
+            "upwelling_irradiance": zeros.copy(),
+            "irradiance_at_bottom": zeros.copy(),
+            "irradiance_normal_to_beam": zeros.copy(),


Always return the same values independent if sun is down or up

jomey · 2026-10-22T16:50:41Z

-                    new_storm[sta] = 0
+                new_storm["start"] = time
+                for station in row.index:
+                    new_storm[station] = 0


New way to iterate over Pandas Series

jomey · 2026-10-22T16:51:14Z

            # Accumulate precip for storm total
-            for sta, mass in row.iteritems():
-                new_storm[sta] += mass
+            for station, mass in row.items():


Alternative way to iterate as dictionary

jomey · 2026-10-22T16:52:05Z

-    " "]
+    r"  .----------------.  .----------------.  .----------------.  .----------------.",
+    r" | .--------------. || .--------------. || .--------------. || .--------------. |",
+    r" | |    _______   | || | ____    ____ | || |  _______     | || |  _________   | |",


/ in strings need a raw string going forward to not indicate an escape character.

jomey · 2026-10-22T16:52:36Z

        # get triangulation
        if self.tri is None:
-            xy = _ndim_coords_from_arrays((self.metadata.utm_x, self.metadata.utm_y))
+            xy = np.column_stack((self.metadata.utm_x, self.metadata.utm_y))


Scipy syntax update

Hedrick-ARS

Well, looks like @jomey was busy here. Thanks for making this push to update all the new syntax for 3.14.

Does this entirely remove the need for the spatialnc package?

Hedrick-ARS

Well it looks like @jomey was busy. Thank you for updating all the syntax here for 3.14.

Does this remove the final dependency on spatialnc package?

jomey · 2026-10-22T19:39:54Z

Does this remove the final dependency on spatialnc package?

I build this PR based on the PR #86. That's why you see the changes to spatialnc again. Once we merge this first PR, then this will be gone from here.

jomey · 2026-10-25T22:15:54Z

To get the tests passing we first must work on an updated conda environment.
This is started here:
iSnobal/model_setup#40

jomey added 14 commits October 21, 2026 17:16

Utils - Integrate needed methods from ARS spatialn library

21b92d7

Reduce the external package dependecies and copy the few methods over. The ARS package is no longer maintained and it is one less library that was only tested with old Python versions. Closes #12

Output - NetCDf - Run Ruff on OutputNetCDF class

3cf21ee

Project TOML - Run Ruff

69eae16

HRRR - Grib Xarray - Address warning where future defaults might change

2e0aaf7

Explictly set method argument values when combining datasets to the current defaults so there are no changes with future library upgrades.

HRRR - Grib Xarray - Run Ruff linter

555da61

VariableBase - Update outdated numpy syntax

b659663

np.Inf was changed to np.inf

VariableBase = Run Ruff linter

8c74461

Envphus - solar model - Return first array value

ab81d4b

Instead of returning an array with one element, directly grab the first index element.

Envphys - solar cloud - Cast modelled solar as float32

e564d8f

Pandas no longer silently casts these values and raises an error. This ensures current behavior going forward.

Envphys - Storms - Update to Pandas 3 Series iteration

c3d0c13

`iteritems` was deprecated and is now either `items` or just the index.

Envphus - Storms - Run Ruff linter

f23b9a5

Distribute - Wind - Windstral - Update numpy nan sytnax and Ruff lint

c124c34

Data - Topo - Use float64 for coordinate calculation

57b621d

Averaging float32 coordinates loses precision and varies with numpy version. Cast to float64 in get_center and update the expected test values (center x, basin longitude) to the exact results. Analyzed with the help of Copilot.

ASCII art - Use raw strings to suppress Python warnings

c14dc1d

Backslashes cause conflicts.

jomey requested review from a team and a lite review from Copilot October 22, 2026 14:08

jomey added the maintenance Technical updates to code base, dependent libraries, package install, or software architecture label Oct 22, 2026

PyProject - Update dependencies

a547b7d

These are now all listed as of 202610 with Python 3.14

jomey force-pushed the python_314 branch from 18bdc2e to a547b7d Compare October 22, 2026 14:22

Copilot started reviewing on behalf of jomey October 22, 2026 14:27

View session

Copilot AI reviewed Oct 22, 2026

View reviewed changes

Comment thread pyproject.toml Resolved

Comment thread smrf/envphys/solar/model.py Resolved

jomey added 4 commits October 22, 2026 09:36

Envphys - Solar model - Annotate method and use better variable names

015c873

No logic change.

Envphys - Solar - Fix regression introduced with python upgrade

b04a72c

The twostream method now consistently returns arrays so the change to model.py can index the return results no matter if the sun is up or down.

Envphys - Solar twostream - Lint and annotate code, no logic change

6c64e2e

Envphys - Solar twostream - Enforce always passing tau as an array

0882c58

Reduce the complexity of the method signature and always expect an array. The only place where this was called as a float was in the model.py This also consolidates the tests written for twostream.

jomey commented Oct 22, 2026

View reviewed changes

Hedrick-ARS approved these changes Oct 22, 2026

View reviewed changes

jomey mentioned this pull request Oct 23, 2026

Pandas 3.1 release - Address deprecation warning #89

Open

brentwilder mentioned this pull request Oct 26, 2026

Project dependencies - Drop spatialnc iSnobal/topocalc#19

Open

Conversation

jomey commented Oct 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code behavior changes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Hedrick-ARS left a comment

Choose a reason for hiding this comment

Uh oh!

Hedrick-ARS left a comment

Choose a reason for hiding this comment

Uh oh!

jomey commented Oct 22, 2026

Uh oh!

jomey commented Oct 25, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jomey commented Oct 22, 2026 •

edited

Loading