Repository navigation
Conversation
Explictly set method argument values when combining datasets to the current defaults so there are no changes with future library upgrades.
np.Inf was changed to np.inf
Instead of returning an array with one element, directly grab the first index element.
Pandas no longer silently casts these values and raises an error. This ensures current behavior going forward.
`iteritems` was deprecated and is now either `items` or just the index.
Averaging float32 coordinates loses precision and varies with numpy version. Cast to float64 in get_center and update the expected test values (center x, basin longitude) to the exact results. Analyzed with the help of Copilot.
Backslashes cause conflicts.
These are now all listed as of 202610 with Python 3.14
The twostream method now consistently returns arrays so the change to model.py can index the return results no matter if the sun is up or down.
Reduce the complexity of the method signature and always expect an array. The only place where this was called as a float was in the model.py This also consolidates the tests written for twostream.
Pandas will soon deprecate the `inplace` option. Using this opportunity to review the Xarray to Dataframe conversion and reduce number dataframe manipulation. Assisted with: Haiku 5.5
Also switch the doc helper to a `sed` call
Similar update to the file loader and simplify the data frame creation while removing the `inplace` call.
New Pandas version that gets values from DatetimeIndexes no longer return a timezone naive index. Used this opportunity to reduce method complexity. Assisted with: Haiku 5.5
Speeds up execution.
Change the detrending of the data to use pure numpy matrix math.
The UTM library accepts numpy arrays and we don't neet to loop over the dataframe via `apply`
jomey
added
maintenance
There was a problem hiding this comment.
🟡 Changes recommended
The grid refactor introduces incorrect mask selection, measurement misalignment, and a single-neighbor interpolation failure.
3 open issues
What changed in this PR
This PR updates SMRF’s meteorological data processing for newer scientific Python libraries, alongside the prerequisite Python upgrade.
Changes:
- Replaces pandas-heavy HRRR loading and local grid detrending with NumPy operations.
- Updates solar model interfaces and adds focused radiation tests.
- Refreshes dependencies, build targets, and deprecated API usage.
| File | Description |
|---|---|
| smrf/utils/utils.py | Uses public SciPy interpolation imports. |
| smrf/tests/envphys/test_radiation.py | Removes tests moved into dedicated coverage. |
| smrf/tests/envphys/solar/test_twostream.py | Tests reference results, array shapes, and nighttime outputs. |
| smrf/tests/envphys/solar/test_model.py | Adds daytime and nighttime solar model tests. |
| smrf/tests/data/test_topo.py | Updates center-coordinate expectations and tolerances. |
| smrf/spatial/grid.py | Vectorizes local detrending and mask lookup. |
| smrf/framework/ascii_art.py | Uses raw strings for backslash-containing artwork. |
| smrf/envphys/storms.py | Replaces deprecated pandas iteration methods. |
| smrf/envphys/solar/twostream.py | Requires array optical depths and returns nighttime arrays. |
| smrf/envphys/solar/model.py | Adapts to array inputs and returns scalar irradiance. |
| smrf/envphys/solar/cloud.py | Streamlines clear-sky calculations and cloud-factor filling. |
| smrf/distribute/wind/winstral.py | Updates NumPy constants and modernizes syntax. |
| smrf/distribute/variable_base.py | Updates infinity constants and class attribute assignment. |
| smrf/data/load_topo.py | Calculates basin centers using float64 coordinates. |
| smrf/data/input/hrrr_grib.py | Updates the cloud-factor function call. |
| smrf/data/hrrr/grib_file_xarray.py | Makes dataset combination options explicit. |
| smrf/data/hrrr/file_loader.py | Uses NumPy grid flattening and vectorized coordinate conversion. |
| pyproject.toml | Updates Python requirements and scientific dependencies. |
| Makefile | Simplifies build, cleanup, and test targets. |
Review effort: Balanced
Only fixing the newly introduced regreassions and bring back support for descending grid coordinates and ensure order of incoming pandas data.
Assisted by Haiku 5.5
Having less than 2 points to fit a regression is not possible.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pandas just came out yesterday with a new 3.1 release that deprecates the
inplaceoption that is extensively used in SMRF.This branch addresses all warnings and replaces pandas dataframe use with pure numpy where appropriate.
All tests pass locally and no changes to precision.
Assisted with Haiku 5.5
NOTE
This branch should be reviewed after the Python upgrade is merged.