Skip to content

Initialize fuel moisture independently of update cadence - #62

Merged
mefrediani merged 6 commits into
developfrom
fix/42-fmc-initialization
Oct 22, 2026
Merged

mefrediani merged 6 commits into
developfrom
fix/42-fmc-initialization

Conversation

@mefrediani

@mefrediani mefrediani commented Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator

With fmoist_freq > 1, the first scheduled moisture update could miss the step-1 initialization, leaving class moisture and forcing history undefined. Initialize once from the first valid atmospheric state independently of cadence, apply zero elapsed moisture time during initialization, and restore that state on restart.

For example, with dt=4 and fmoist_freq=3, initialization is at 4 s, the update at 12 s uses 8 s elapsed time, and later updates use 12 s intervals. Time-based updates retain their existing phase. Invalid or unavailable current/previous forcing fails explicitly. The NUOPC rainfall accumulator is initialized before its first import.

Checkpoints now record initialization status/time and moisture activation/cadence. Loading restores the lifecycle and validates class state, forcing history, and matching settings. Prescribed legacy checkpoints remain readable. PR #39 time-based checkpoints are accepted when their metadata establishes compatible initialized state; ambiguous legacy step-based checkpoints fail with a specific diagnostic. Configuration documentation and twelve focused lifecycle scenarios cover the correction. No new namelist options, ordinary diagnostic fields, or humidity-formula changes are included.

Refs #42. Its remaining extinction, controls, and diagnostic work remains open.

Review base and dependencies

  • This draft targets develop at ead119eca6dfb576d181a5560273d70035f780a3, which includes the squash merge of Add restart capability to the fire behavior model #39. The original local restart merge bc205bc and upstream ead119e have identical file trees but different ancestry.
  • Head a87b23dae1c8b4a282ed503202cbb34579f0536f contains six commits and six files, without historical restart commits in its commit range. The final atmospheric defaults use the original patch bounds (ifps:ifpe, jfps:jfpe) in Set_vars_to_default; state/state_mod.F90 now matches develop exactly and drops out of the feature diff.
  • The review follow-up sets the four atmospheric fill values in Set_vars_to_default, initializes the precipitation-rate accumulator in NUOPC Realize while preserving restored rainfall, and groups external/intrinsic imports before CFBM modules. The component member remains initialized; its NetCDF attribute remains fmc_initialized.
  • P00 / Add focused restart regression coverage #61 at ed18f91 remains an explicit validation dependency absent from this feature's ancestry. The remaining foundation merge order is this PR, then Add focused restart regression coverage #61. Update and validate P00 separately before its merge.
  • The earlier replayed implementation dec7eb2 was byte-for-byte identical to validated 3e472d8. Initialization-placement validation used separate local integration d144a898a436efb7ef411c781368e6853c0415b3, combining P00 and P02 at 439474f; all seven feature files matched that revision exactly. The subsequent patch-bound restoration changes only the four atmospheric default assignments. Original branches and validation evidence remain preserved.

Validation

Earlier validation at combined revision 554d150 used GNU Strict builds on Derecho with GNU 12.2.0, Cray MPICH 8.1.25, NetCDF 4.9.2, and ESMF 8.5.0. Serial, OpenMP, MPI, hybrid, and ordinary NUOPC/ESMX execution passed.

  • Twelve focused initialization/forcing/timing scenarios per build; 45 Python tests; formatting and diff checks passed.
  • All 15 restart cases passed, including the five delayed-moisture cases previously failing in P00. There were 65 continuous/segmented model runs and 30 loader probes.
  • All 320 file comparisons and 12 cross-layout comparisons had zero maximum absolute error; 450 checkpoint checks passed. Loader probes comprised five accepted schema-compatible resumes and 25 expected rejections.
  • Existing PR suite: 36 model runs, 40 execution comparisons, and 36 approved-reference comparisons passed. Largest cross-driver absolute difference was fgrnhfx = 0.125 W m-2, consistent with the existing baseline. No reference or tolerance changed.
  • Original PR Add restart capability to the fire behavior model #39-era checkpoints generated with P00 were also checked: prescribed/time-based resumes agreed exactly with corresponding P02 checkpoints, and ambiguous step-based state was rejected.

PBS jobs: 7890398.desched1, 7890541.desched1, 7890596.desched1. Full retained evidence: /glade/derecho/scratch/frediani/tmp/cfbm-P02-20261021T034953Z/, including attempt-02/results/restart-z8yehdcl/summary.json, attempt-02/results/pr-hw75p1af/summary.json, and legacy-originals/summary.json.

P01/P02/P04 combined acceptance remains pending, including shared clock precision and fractional/long-duration timing. Direct WRF-host execution/restart, NUOPC/ESMX restart, and transfer of checkpoints between MPI layouts remain untested here. Ordinary NUOPC/ESMX execution and comparisons among independent MPI layouts do not establish those restart paths. GitHub CI must pass on the final PR revision before merge; its normal PR suite does not run P00's optional restart suite.

Initialization-placement follow-up validation

Feature 439474f matches tested P00/P02 integration d144a898a436efb7ef411c781368e6853c0415b3 exactly. Four fresh GNU Strict builds (serial, OpenMP, MPI/NUOPC/ESMX, hybrid/NUOPC/ESMX) passed with the environment above.

  • All 15 restart cases pass: 65 continuous/segmented runs, 30 loading probes, 320 zero-error file comparisons (315 continuity and five accepted legacy resumes), 12 zero-error cross-layout comparisons, and 450 checkpoint checks. The restart suite's 67 CTest invocations include the twelve focused P02 lifecycle scenarios in each build.
  • The existing 36-run PR matrix, 40 execution comparisons, and 36 approved-reference comparisons pass. Maximum cross-driver difference remains fgrnhfx = 0.125 W m-2 (maximum percentage error 0.0006368705741347137%), within unchanged tolerances.
  • A connected, nonzero precipitation-rate ESMX_Data check passes with one MPI rank, four MPI ranks, and four ranks by four OpenMP threads. All 33 output-file comparisons against the previously validated executable are exact. Thirty accumulated-rain checks have maximum absolute error 1.862645149230957e-09 in model accumulation units.
  • Formatting and regression both passed in GitHub workflow 39690399727 on 439474f. Local formatting and whitespace checks pass. No approved reference or regression tolerance changed.

PBS job 7900048.desched1 completed the builds and both suites successfully, then exited 1 because the scratch rate-check script looked for its completion marker in stdout instead of the ESMF log. After correcting only that checker, job 7900164.desched1 reran the six rate-check model executions and exited 0. Failed-attempt evidence and original scripts remain preserved.

Evidence: /glade/derecho/scratch/frediani/tmp/cfbm-P02-init-review-20261021T225746Z/, including attempt-01/results/restart-hl_i2yr_/summary.json, attempt-01/results/pr-jpmyi5r2/summary.json, and attempt-01/rate-probe-02/summary.json. The combined P01/P04, coupled-restart, and direct WRF-host validation gaps above remain.

Patch-bound restoration

The four atmospheric defaults retain their original ifps:ifpe, jfps:jfpe patch sections. The temporary explicit memory-bound spelling in f6a6206 is superseded by a87b23d. The final assignments match the original expressions, and state/state_mod.F90 matches develop exactly. Defaults remain in Set_vars_to_default; the NUOPC cold-start rainfall initialization in Realize is unchanged.

Whitespace checks pass. The detailed HPC results above are from 439474f; the optional restart suite has not been rerun after restoring patch bounds. GitHub CI on a87b23d is pending.

@mefrediani
mefrediani changed the base branch from review/restart-pr39-base to develop October 21, 2026 20:27
@mefrediani
mefrediani requested a review from pedro-jm October 21, 2026 20:28
@mefrediani
mefrediani force-pushed the fix/42-fmc-initialization branch from 3e472d8 to dec7eb2 Compare October 21, 2026 20:46
@mefrediani
mefrediani marked this pull request as ready for review October 21, 2026 23:57
@mefrediani
mefrediani merged commit 622e191 into develop Oct 22, 2026
2 checks passed
@mefrediani
mefrediani deleted the fix/42-fmc-initialization branch October 23, 2026 05:04
mefrediani added a commit that referenced this pull request Oct 23, 2026
The unrestricted Rothermel moisture cubic produced nonzero reaction at
extinction and negative reaction above it, affecting `iboros`, base
spread and flame diagnostics before the later spread clamp. This change
sets moisture damping to exactly zero at or above extinction, preserves
the polynomial below extinction, and leaves physical moisture uncapped.
An explicit moisture threshold is retained because a lower clamp on the
polynomial alone can leave positive roundoff at exact extinction. The
moisture-damping comment identifies `etam`. The independent empirical
chaparral spread law remains unchanged. Refs #42; this addresses its
extinction scope only.

Validation follows the source and lifetime of each quantity:

- `Check_nml` requires finite, nonnegative prescribed `fuelmc_g`. Zero
is valid; units are water mass per dry fuel mass.
- `fuel_t%Check_fuels`, called once during Anderson initialization,
requires finite, positive `fuelmce` for burnable categories.
- Moisture initialization validates the fuel loads and fixed averaging
weights. The component then checks each evolving class after its update,
before averaging. Valid classes with valid weights do not require a
repeated check of the computed mean. Empty categories retain zero
weights.
- The restart reader checks effective and class moisture separately
through the FMC component. `Check_fmc` is a common type-bound procedure
of `fmc_t`, inherited by the moisture implementation. A source comment
explains why `Check_fmc` remains: `fmc_g` is read independently from the
checkpoint and used directly, and prescribed-moisture restarts have no
class state. This is an input check, including for supported legacy
files; an unchanged checkpoint from the same validated revision would
already satisfy it.

Computed-class checks execute after the OpenMP loop on the initial
thread, avoiding concurrent fatal diagnostics across tiles. Spatial
checks cover owned burnable cells. ROS owns the extinction calculation.
No namelist option or checkpoint schema is added; P02 initialization and
restart compatibility checks are retained. Full live/dead SB40
combustion and gridded-input readers remain separate work.

The review base is `develop` at `622e191`, including merged #39 and #62.
Head `af2ca03` contains a 12-file P03 diff. P00/#61 is a test dependency
only in separate integration `3b56260`; its commits are absent from this
feature branch. Production files, focused Fortran tests and
configuration documentation in that integration match the PR head
exactly.


Validation of `af2ca03` / integration `3b56260`:

- [GitHub workflow
39997883832](https://github.com/NCAR/fire_behavior/actions/runs/39997883832)
passes formatting and regression on the published head.
- Derecho PBS job `7922406` passes four GNU Strict builds: serial,
OpenMP, MPI and hybrid, including ordinary NUOPC/ESMX execution. All 35
focused scenarios pass in every build, covering extinction boundaries
and diagnostics, namelist/table inputs, class state, initialization
loads, and independently restored effective moisture. The bound checker
is also exercised without initialized class arrays, matching
prescribed-moisture restarts.
- All 378 PR CTests, 36 model runs, 40 execution comparisons and 36
approved-reference comparisons pass. Maximum reference error is zero;
maximum execution difference is 0.125 W m-2 in `fgrnhfx`, within the
existing tolerance.
- All 15 restart cases and 30 loading probes pass. The 315 continuity
file comparisons have zero error; 450 checkpoint checks and 12
independent layout comparisons also pass.
- An additional 22 checkpoint mutation/control probes pass in serial and
hybrid configurations. These include invalid `fmc_g` with unchanged
valid classes, invalid classes, and accepted zero/wet states.
- A separate Fortran probe with the GNU Strict arithmetic flags confirms
that, at a moisture ratio of one, the polynomial and `max(etam, 0.0)`
both return `2.384185791015625e-7` in single precision. This small
positive residual explains retaining the explicit extinction condition.
The below-extinction arithmetic is unchanged.

No reference or tolerance was changed. Direct WRF-host
execution/restart, NUOPC/ESMX restart, cross-layout checkpoint transfer
and combined P01/P04 acceptance remain untested. Legacy readability does
not establish numerical equivalence across the extinction correction.
Evidence, exact commands, source snapshots and PBS logs are retained
under
`/glade/derecho/scratch/frediani/tmp/cfbm-P03-type-bound-review-20261023T174535Z`.
mefrediani added a commit that referenced this pull request Oct 23, 2026
Adds continuous-versus-restarted regression coverage to the existing
harness and runs it in GitHub CI alongside the existing PR suite. The
cases exercise prescribed and evolving moisture, delayed moisture
updates, checkpoints around observed-perimeter activation, and
preservation of REAL64 fuel state. All 15 restart cases pass on the
merged P02 initialization correction, including the five cases that
failed on the original restart foundation.

## Scope and review base

- Base: `develop` at `622e191fac104e1a73ae99a769e5bd3dfee2946c`,
including merged #39 and #62.
- Head: `56b832037a8e73d198d395e1c194afbdb46ed205`.
- The 23-file diff contains the restart harness, tests/documentation,
suite help and CI selection. Production code and P02's focused unit
tests are identical to develop. No scientific tolerance or reference
changes.
- Merge `ce551ee` incorporates develop into the published P00 history.
The moisture component and restart-reader conflicts retain develop's
merged P02 implementation. This preserves the original P00 commits and
the common ancestor used by stacked #65.
- The harness includes the initialization metadata checks and 30
restart-loading probes previously validated in the P00/P02 integration.
These check supported legacy inference and explicit rejection of
ambiguous or damaged state.

CI builds serial, OpenMP, MPI and hybrid configurations once, then runs
both `pr` and `restart`. Both suites execute even if one fails; either
failure fails the regression check. The existing test-evidence artifact
retains both result directories. The approved `github-pr` baseline
remains unchanged, and optional candidate generation still uses only the
`pr` results. The configuration guide includes a restart YAML example
and explains suite selection, segment timing and comparison policy.

## Validation

Derecho PBS `7915414.desched1` exits zero after fresh serial, OpenMP,
MPI and hybrid GNU Strict builds. MPI/hybrid builds include NUOPC and
ESMX. Environment: GNU 12.2, Cray MPICH 8.1.25, NetCDF 4.9.2, ESMF 8.5,
shared NPL Python.

- All 67 restart CTests pass: 15 restart cases, 48 P02 lifecycle checks,
and four Python harness registrations.
- All 65 continuity model invocations and 30 restart-loading probes
complete as expected. The 315 continuity file comparisons plus five
accepted-legacy checkpoint comparisons are bitwise equal, with zero
maximum error.
- All 450 checkpoint validity checks and 12 cross-layout comparisons
pass; cross-layout error is zero.
- All 238 existing PR CTests pass, including 36 model runs. All 40
execution comparisons and 36 approved-reference comparisons pass.
Reference error is zero; maximum cross-driver error is 0.125 W m-2 in
`fgrnhfx`, within the unchanged tolerance.
- All 45 Python tests pass; YAPF 0.43.0, shell syntax and whitespace
checks pass.
- Local workflow checks confirm both suites execute and the final status
is correct when neither, either, or both suites fail. The documented
restart configuration matches the production YAML; its 28 s split
retains the next output at 36 s.
- [GitHub workflow
39970138677](https://github.com/NCAR/fire_behavior/actions/runs/39970138677)
passes on implementation revision `5de15ed`: formatting, all four
builds, all 238 PR CTests and all 67 restart CTests. Downloaded reports
confirm all 15 restart cases, 320 zero-error file comparisons and 30
loading probes pass in CI. Both suite summaries and eight JUnit reports
are retained in the uploaded test-evidence artifact; the 36
approved-reference comparisons also have zero error.

The PBS run used clean candidate `9a64864`, whose complete tree equals
merge `ce551ee`. Later `c8aeeaa` changes documentation, and `5de15ed`
changes CI selection and documentation; production code, model cases,
harness and approved baseline remain identical to the PBS-validated
revision. PBS evidence is preserved under
`/glade/derecho/scratch/frediani/tmp/cfbm-P00-develop-20261022T230324-0600/`;
CI-update evidence is under
`/glade/derecho/scratch/frediani/tmp/cfbm-P00-restart-ci-20261023T082618-0600/`.
Repository validation history retains the original failures and records
the successful PBS rerun.

Documentation-only follow-up `2c6b5b9` adds the [test-count
breakdown](https://github.com/NCAR/fire_behavior/blob/test-restart-continuity/tests/regression/README.md#how-test-counts-are-reported):
checkpoint validity, file comparisons, the per-build PR CTest table, and
restart invocation counts. Counts and links were checked against the
downloaded reports from workflow 39970138677. Its [CI run
39974535865](https://github.com/NCAR/fire_behavior/actions/runs/39974535865)
uses the unchanged workflow; no executable, harness, case, or baseline
changed.

Documentation follow-up `b2da936` replaces development labels in the
user-facing regression guides with descriptions of moisture
initialization, legacy restart metadata, and coverage limits.
`tests/AGENTS.md` requires this distinction and keeps development
provenance in validation records. Follow-up `56b8320` consolidates all
test guidance there, retains a pointer at `tests/regression/AGENTS.md`,
and requires informative unit-test comments describing the behavior,
setup and expected result. The historical validation file is unchanged.
Guide-label, relative-link, whitespace, and pinned YAPF checks pass.
Workflows 39974535865 and 39977209659 on the preceding documentation
revisions passed both formatting and regression checks.

## Coverage limits

Restart comparisons use the standalone driver and the same
revision/execution layout. NUOPC/ESMX integrations pass in the existing
PR suite, but coupled restart, native WRF-host restart, cross-layout
checkpoint transfer, and combined P01/P04 behavior remain separate
validation work. Synthetic legacy checkpoints test schema compatibility,
not numerical equivalence across scientific revisions. No production
correction for #42 is included in this test PR.

---------

Co-authored-by: masih <masih@ucar.edu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants