Repository navigation
Rewrite the Validation pages for students, with plots - #23
Merged
Merged
Conversation
Each Validation page opens with what it checks, and each case says why the check exists, what it tests, how the right answer is known and why it matters, with the reference, tolerance, test id and command in a collapsed box. The pages name the outputs PROTEUS reads from SPIDER and state that PROTEUS does not use the volatile and reaction parts, and Known limit lines name shared constants and untested paths.
The right-hand-side, rheological-front, boundary, mesh, material, reaction, composite and lookup pages name only the faults their tests catch and state the shared or same-rule comparisons as known limits. The rheological-front page names the PROTEUS field it feeds, the copied constants are described as copies, the unsourced reference lists are removed, and the observed flux ranges are marked as observed.
The mesh, material, IC and reaction pages keep caught faults in their first lead-in and state the rest as known limits. The interpolation page names all its callers, the Spaargaren limit states what the 2 literals catch, the rheological-front page says which PROTEUS fields come from other blocks, and the observed flux ranges and copied constants are labelled as such.
The mesh, reaction and material pages state the bounds that do catch a wrong core-mantle radius, equilibrium constant or blend, the IC limit names the entropy gradient option, and the rheological-front page defines the convection regime flag.
The committed validation plots are written and compared with the proteus-mpl 1.5.0 fixture.
The adiabat, abundance and ocean-moles cases draw their asserted profile, inventories and element ratios, written with the PETSc build of the CI jobs.
Run output moves by up to 5e-6 between PETSc builds, more than the stale-plot check allows. The atmosphere, energy, reaction, front and melt-fraction plots now draw only asserted values, rounded to 2 or 3 significant digits with at least 100 times the measured spread to the next rounding step, and closed forms on fixed grids. Every figure reads the same names, functions and arrays as its asserts, so a changed reference or tolerance moves the plot. The remaining run plots are written with the PETSc build of the CI jobs.
The finite-difference plot is written again with its data summary, so the stale-plot check covers it. The read texts keep each sentence under 25 words, define their symbols, use the cell-boundary and cell-centre names of the mesh nodes, and state that the asserts check the unrounded values.
The disabled-transport note separates the 3 maxima from the top-node gap, the front note states the 1-node allowance of the assert, and the rhs note gives its pass condition.
The dS/dt profiles of the run differ by about 7e-4 between the macOS and Linux builds, which the stale-plot check rejects. The plot now draws the largest |dS/dt| at 1 significant digit against the asserted window from 1e-12 to 1e-4; the page names the finite-difference comparison that the plot does not show.
Each How to read the plot note names the tolerance its test asserts when the plot cannot show it, and the energy, eos and rheological front notes stay within 6 sentences. The disabled-transport test asserts and draws the same named maxima, and the affine interpolation plot reads its nodes from the table fixture. The radiogenic heating sum is drawn as a wide pale line under the al26 line, the ocean-moles bars are grey so the stored values stand out, and 2 clipped axis labels are shorter.
The validation index defines blackbody50, PROTEUS, the basic and staggered nodes, IC and BC, and each page defines its symbols with units before the formula that uses them. Axis units are named in the plot notes, the energy page lists the 4 heat fluxes, and the bc cases follow the order of the index. The Adams-Williamson page states the 100 kg/m3 gap the test asserts, the rheological front page calls its check self-consistent, the Spaargaren prefactor is attributed as in eos.c, and the abundance note marks the reservoir directions as not asserted.
There was a problem hiding this comment.
🔵 Needs a closer look
The broad test-and-plot changes warrant human review, and repository verification was unavailable.
No open issues 🎉
What changed in this PR
This PR makes SPIDER’s validation documentation more accessible to students, linking physics checks to test-generated plots without changing the C solver.
Changes:
- Reorganizes validation pages around purpose, expected results, limitations, and plot interpretation.
- Adds plots to validation tests and display rounding for build-dependent values.
- Raises the CI requirement to
proteus-mpl>=1.5.0for committed-plot checks.
| File | Description |
|---|---|
| tests/test_twophase.py | Plots global melt-fraction identities. |
| tests/test_rhs.py | Replaces profile plots with a cooling-rate scale plot. |
| tests/test_rheologicalfront.py | Plots front location against neighboring nodes. |
| tests/test_reaction.py | Plots hydrogen and carbon transfer balances. |
| tests/test_mesh.py | Plots analytical and computed pressure profiles. |
| tests/test_matprop.py | Plots viscosity end-member checks. |
| tests/test_interp.py | Plots affine interpolation and query results. |
| tests/test_ic.py | Adds entropy and volatile-inventory plots. |
| tests/test_eos.py | Adds phase-boundary and compositional-viscosity plots. |
| tests/test_eos_lookup.py | Plots independent density comparisons. |
| tests/test_eos_composite.py | Plots the melt-fraction relation. |
| tests/test_eos_adamswilliamson.py | Plots analytical density comparisons. |
| tests/test_energy.py | Adds transport, radioactive decay, and steady-state plots. |
| tests/test_bc.py | Adds prescribed-flux and surface-entropy plots. |
| tests/test_atmosphere.py | Plots the grey-body radiation check. |
| tests/_plot_utils.py | Adds significant-digit display rounding. |
| docs/Validation/twophase.md | Explains melt bookkeeping checks and plots. |
| docs/Validation/rhs.md | Clarifies derivative checks and visualization limits. |
| docs/Validation/rheologicalfront.md | Explains front tracking and its limitations. |
| docs/Validation/reaction.md | Explains elemental transfer conservation. |
| docs/Validation/mesh.md | Explains pressure-profile validation. |
| docs/Validation/matprop.md | Explains viscosity end-member validation. |
| docs/Validation/interp.md | Explains interpolation checks and boundaries. |
| docs/Validation/index.md | Introduces validation concepts and terminology. |
| docs/Validation/ic.md | Explains initial entropy and inventory checks. |
| docs/Validation/eos.md | Explains phase-boundary and viscosity checks. |
| docs/Validation/eos_lookup.md | Explains independent table-read comparisons. |
| docs/Validation/eos_composite.md | Explains fusion entropy and melt fraction. |
| docs/Validation/eos_adamswilliamson.md | Explains analytical density validation. |
| docs/Validation/energy.md | Explains transport, heating, and energy balance. |
| docs/Validation/bc.md | Explains entropy and prescribed-flux boundaries. |
| docs/Validation/atmosphere.md | Explains grey-body surface radiation validation. |
| docs/assets/validation/rheologicalfront_location_dark.svg | Adds the dark-theme front-location plot. |
| .github/workflows/nightly.yml | Updates the plotting dependency minimum. |
| .github/workflows/ci.yml | Updates plotting requirements for test and installer jobs. |
Review effort: Balanced
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The Validation pages follow one layout for students who know physics but not SPIDER, and every case has a plot pair drawn inside its own test. Each page opens with what it checks; each case says why the check exists, what it tests, how we know the right answer and why it matters, gives a known limit where there is one, and shows a light and dark plot with a note on how to read it, above a box with the reference, tolerance and test id.
Changes
docs/Validation/).docs/assets/validation/, each at most 60 kB), drawn by thevalidation_plotfixture of proteus-mpl from the variables the tests assert on. Every test run compares the committed pair with the plotted data, so a changed assert or a changed result fails the test until the plot is made again.shown()helper to the precision the note states. The finite-difference check ofrhsdraws its asserted scale value; its relative comparison stays asserted and is named in the note.proteus-mpl>=1.5.0(ci.yml,nightly.yml), the version with the committed-plot check.No change to the C code, the Makefile or any result.
Validation
ad6b7a1572b369df7d8b019641abb6d735fbba86: https://github.com/FormingWorlds/SPIDER/actions/runs/40420490279 (70 passed on Ubuntu 24.04; every plot passes its committed-plot check on Linux).