Skip to content

Reduce the memory usage that is important for ne1024 simulation #665

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
sjsprecious wants to merge 4 commits into
base: main
Choose a base branch
from
Open

Reduce the memory usage that is important for ne1024 simulation #665

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
636e600
update map file option
sjsprecious Jun 12, 2026
3d6dde4
add option to provide lnd2rof_map consd file
sjsprecious Jun 13, 2026
3ef0240
reuse ocean field existing mesh handle to avoid duplicate full esmf mesh
sjsprecious Jun 19, 2026
9437041
address bill's comments
sjsprecious Jun 26, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion cime_config/config_component.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1412,7 +1412,16 @@
<default_value>idmap</default_value>
<group>run_domain</group>
<file>env_run.xml</file>
<desc>lnd2rof flux mapping file</desc>
<desc>lnd2rof flux mapping file (conservative, conserve/fraction 'consf' mapping)</desc>
</entry>

<entry id="LND2ROF_FRAC_FMAPNAME">
<type>char</type>
<default_value>unset</default_value>
<group>run_domain</group>
<file>env_run.xml</file>
<desc>lnd2rof conservative (destarea, 'consd') fraction mapping file used by the mediator
fraction init; 'unset' computes the weights online (memory-heavy at high resolution)</desc>
</entry>

<entry id="ROF2LND_FMAPNAME">
Expand Down
13 changes: 12 additions & 1 deletion cime_config/namelist_definition_drv.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2269,12 +2269,23 @@
<category>mapping</category>
<input_pathname>abs</input_pathname>
<group>MED_attributes</group>
<desc>lnd to rof mapping, 'unset' or 'idmap' are normal possible values (mapping file given for mizuRoute grids)</desc>
<desc>lnd to rof flux mapping (conservative, conserve/fraction 'consf' mapping); 'unset' or 'idmap'
are normal possible values (mapping file given for mizuRoute grids)</desc>
<values>
<value>$LND2ROF_FMAPNAME</value>
<value samegrid_lnd_rof='true'>idmap</value>
</values>
</entry>
<entry id="lnd2rof_fmap" modify_via_xml="LND2ROF_FRAC_FMAPNAME">
<type>char</type>
<category>mapping</category>
<group>MED_attributes</group>
<desc>lnd to rof conservative destarea ('consd') fraction mapping file read by the mediator

@billsacks billsacks Jun 26, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand correctly, it looks like the difference between this and LND2ROF_FMAPNAME is that this one uses consd mapping and the other uses consf mapping. Is that right? If so, I think it would be useful to extend the comment in LND2ROF_FMAPNAME pointing out that that one uses consf mapping.

@sjsprecious sjsprecious Jun 26, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct. I updated it in the latest commit.

fraction init (med_fraction_mod); 'unset' computes the weights online</desc>
<values>
<value>$LND2ROF_FRAC_FMAPNAME</value>
</values>
</entry>
<entry id="rof2lnd_map" modify_via_xml="ROF2LND_FMAPNAME">
<type>char</type>
<category>mapping</category>
Expand Down
10 changes: 5 additions & 5 deletions mediator/esmFldsExchange_cesm_mod.F90
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3320,7 +3320,7 @@ subroutine esmFldsExchange_cesm(gcomp, phase, rc)
else
if ( fldchk(is_local%wrap%FBImp(complnd, complnd), 'Flrl_rofsur'//trim(suffix), rc=rc) .and. &
fldchk(is_local%wrap%FBExp(comprof) , 'Flrl_rofsur'//trim(suffix), rc=rc)) then
call addmap_from(complnd, 'Flrl_rofsur'//trim(suffix), comprof, mapconsf, map_fracname_lnd2rof, 'unset')
call addmap_from(complnd, 'Flrl_rofsur'//trim(suffix), comprof, mapconsf, map_fracname_lnd2rof, lnd2rof_map)
call addmrg_to(comprof, 'Flrl_rofsur'//trim(suffix), &
mrg_from=complnd, mrg_fld='Flrl_rofsur'//trim(suffix), &
mrg_type='copy_with_weights', mrg_fracname=mrg_fracname_lnd2rof)
Expand All @@ -3341,7 +3341,7 @@ subroutine esmFldsExchange_cesm(gcomp, phase, rc)
else
if ( fldchk(is_local%wrap%FBImp(complnd, complnd), 'Flrl_rofi'//trim(suffix), rc=rc) .and. &
fldchk(is_local%wrap%FBExp(comprof) , 'Flrl_rofi'//trim(suffix), rc=rc)) then
call addmap_from(complnd, 'Flrl_rofi'//trim(suffix), comprof, mapconsf, map_fracname_lnd2rof, 'unset')
call addmap_from(complnd, 'Flrl_rofi'//trim(suffix), comprof, mapconsf, map_fracname_lnd2rof, lnd2rof_map)
call addmrg_to(comprof, 'Flrl_rofi'//trim(suffix), &
mrg_from=complnd, mrg_fld='Flrl_rofi'//trim(suffix), &
mrg_type='copy_with_weights', mrg_fracname=mrg_fracname_lnd2rof)
Expand All @@ -3362,7 +3362,7 @@ subroutine esmFldsExchange_cesm(gcomp, phase, rc)
else
if ( fldchk(is_local%wrap%FBImp(complnd, complnd), 'Flrl_rofgwl'//trim(suffix), rc=rc) .and. &
fldchk(is_local%wrap%FBExp(comprof) , 'Flrl_rofgwl'//trim(suffix), rc=rc)) then
call addmap_from(complnd, 'Flrl_rofgwl'//trim(suffix), comprof, mapconsf, map_fracname_lnd2rof, 'unset')
call addmap_from(complnd, 'Flrl_rofgwl'//trim(suffix), comprof, mapconsf, map_fracname_lnd2rof, lnd2rof_map)
call addmrg_to(comprof, 'Flrl_rofgwl'//trim(suffix), &
mrg_from=complnd, mrg_fld='Flrl_rofgwl'//trim(suffix), &
mrg_type='copy_with_weights', mrg_fracname=mrg_fracname_lnd2rof)
Expand All @@ -3383,7 +3383,7 @@ subroutine esmFldsExchange_cesm(gcomp, phase, rc)
else
if ( fldchk(is_local%wrap%FBImp(complnd, complnd), 'Flrl_rofsub'//trim(suffix), rc=rc) .and. &
fldchk(is_local%wrap%FBExp(comprof) , 'Flrl_rofsub'//trim(suffix), rc=rc)) then
call addmap_from(complnd, 'Flrl_rofsub'//trim(suffix), comprof, mapconsf, map_fracname_lnd2rof, 'unset')
call addmap_from(complnd, 'Flrl_rofsub'//trim(suffix), comprof, mapconsf, map_fracname_lnd2rof, lnd2rof_map)
call addmrg_to(comprof, 'Flrl_rofsub'//trim(suffix), &
mrg_from=complnd, mrg_fld='Flrl_rofsub'//trim(suffix), &
mrg_type='copy_with_weights', mrg_fracname=mrg_fracname_lnd2rof)
Expand All @@ -3404,7 +3404,7 @@ subroutine esmFldsExchange_cesm(gcomp, phase, rc)
else
if ( fldchk(is_local%wrap%FBImp(complnd, complnd), 'Flrl_irrig'//trim(suffix), rc=rc) .and. &
fldchk(is_local%wrap%FBExp(comprof) , 'Flrl_irrig'//trim(suffix), rc=rc)) then
call addmap_from(complnd, 'Flrl_irrig'//trim(suffix), comprof, mapconsf, map_fracname_lnd2rof, 'unset')
call addmap_from(complnd, 'Flrl_irrig'//trim(suffix), comprof, mapconsf, map_fracname_lnd2rof, lnd2rof_map)
call addmrg_to(comprof, 'Flrl_irrig'//trim(suffix), &
mrg_from=complnd, mrg_fld='Flrl_irrig'//trim(suffix), &
mrg_type='copy_with_weights', mrg_fracname=mrg_fracname_lnd2rof)
Expand Down
37 changes: 32 additions & 5 deletions mediator/med_fraction_mod.F90
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -169,7 +169,7 @@ subroutine med_fraction_init(gcomp, rc)
! Initialize FBFrac(:) field bundles

use ESMF , only : ESMF_LogWrite, ESMF_LOGMSG_INFO
use ESMF , only : ESMF_SUCCESS
use ESMF , only : ESMF_SUCCESS, ESMF_FAILURE, ESMF_LogSetError
use ESMF , only : ESMF_GridComp, ESMF_GridCompGet, ESMF_StateIsCreated
use ESMF , only : ESMF_FieldBundle, ESMF_FieldBundleIsCreated, ESMF_FieldBundleDestroy
use ESMF , only : ESMF_FieldBundleGet
Expand All @@ -181,6 +181,7 @@ subroutine med_fraction_init(gcomp, rc)
use med_internalstate_mod , only : InternalState
use med_map_mod , only : med_map_routehandles_init, med_map_rh_is_created
use med_methods_mod , only : State_getNumFields => med_methods_State_getNumFields
use NUOPC , only : NUOPC_CompAttributeGet
use perf_mod , only : t_startf, t_stopf

! input/output variables
Expand Down Expand Up @@ -208,6 +209,8 @@ subroutine med_fraction_init(gcomp, rc)
integer :: n,n1,ns
integer :: maptype
integer :: fieldCount
logical :: isPresent, isSet, lexist
character(len=CX) :: lnd2rof_fmap ! consd (destarea) lnd->rof fraction-map file
logical, save :: first_call = .true.
character(len=*),parameter :: subname=' (med_fraction_init)'
!---------------------------------------
Expand Down Expand Up @@ -584,10 +587,34 @@ subroutine med_fraction_init(gcomp, rc)
if (is_local%wrap%comp_present(complnd)) then
maptype = mapconsd
if (.not. med_map_RH_is_created(is_local%wrap%RH(complnd,comprof,:),maptype, rc=rc)) then
call med_map_routehandles_init( complnd, comprof, &
FBSrc=is_local%wrap%FBImp(complnd,complnd), &
FBDst=is_local%wrap%FBImp(complnd,comprof), &
mapindex=maptype, RouteHandle=is_local%wrap%RH, rc=rc)
! The lnd->rof fraction map may be the only grid-crossing conservative coupling
! map in a configuration, and computing its weights online (ESMF_FieldRegridStore)
! can be prohibitively memory-heavy at high resolution. If the 'lnd2rof_fmap'
! attribute names a file of offline consd (conserve/destarea) fraction weights,
! read the weights from that file; otherwise compute them online.
call NUOPC_CompAttributeGet(gcomp, name='lnd2rof_fmap', value=lnd2rof_fmap, &
isPresent=isPresent, isSet=isSet, rc=rc)
if (ChkErr(rc,__LINE__,u_FILE_u)) return
if (.not. (isPresent .and. isSet)) lnd2rof_fmap = 'unset'

@billsacks billsacks Jun 26, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like this line is redundant with the initialization to 'unset' a few lines above, and that just one or the other could be kept, with the other removed.

@sjsprecious sjsprecious Jun 26, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. I have removed lnd2rof_fmap = 'unset' a few lines above.

if (lnd2rof_fmap /= 'unset') then
inquire(file=lnd2rof_fmap, exist=lexist)
if (.not. lexist) then
call ESMF_LogSetError(ESMF_FAILURE, &
msg=trim(subname)//': lnd2rof_fmap weight file not found: '//trim(lnd2rof_fmap), &
line=__LINE__, file=u_FILE_u, rcToReturn=rc)
return
end if
call med_map_routehandles_init( complnd, comprof, &
FBSrc=is_local%wrap%FBImp(complnd,complnd), &
FBDst=is_local%wrap%FBImp(complnd,comprof), &
mapindex=maptype, RouteHandle=is_local%wrap%RH, &
mapfile=trim(lnd2rof_fmap), rc=rc)
else
call med_map_routehandles_init( complnd, comprof, &
FBSrc=is_local%wrap%FBImp(complnd,complnd), &
FBDst=is_local%wrap%FBImp(complnd,comprof), &
mapindex=maptype, RouteHandle=is_local%wrap%RH, rc=rc)
end if
if (ChkErr(rc,__LINE__,u_FILE_u)) return
end if

Expand Down
10 changes: 8 additions & 2 deletions mediator/med_map_mod.F90
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -296,7 +296,7 @@ subroutine med_map_RouteHandles_initfrom_esmflds(gcomp, flds_scalar_name, llogun
end subroutine med_map_RouteHandles_initfrom_esmflds

!================================================================================
subroutine med_map_routehandles_initfrom_fieldbundle(n1, n2, FBsrc, FBdst, mapindex, RouteHandle, rc)
subroutine med_map_routehandles_initfrom_fieldbundle(n1, n2, FBsrc, FBdst, mapindex, RouteHandle, mapfile, rc)

use ESMF , only : ESMF_LogWrite, ESMF_LOGMSG_INFO, ESMF_SUCCESS, ESMF_LogFlush
use ESMF , only : ESMF_Field, ESMF_FieldBundle, ESMF_RouteHandle
Expand All @@ -313,6 +313,7 @@ subroutine med_map_routehandles_initfrom_fieldbundle(n1, n2, FBsrc, FBdst, mapin
type(ESMF_FieldBundle) , intent(in) :: fBdst
integer , intent(in) :: mapindex
type(ESMF_RouteHandle) , intent(inout) :: RouteHandle(:,:,:)
character(len=*), optional, intent(in) :: mapfile ! read offline weights (ESMF_FieldSMMStore) instead of online regrid
integer , intent(out) :: rc

! local variables
Expand All @@ -333,7 +334,12 @@ subroutine med_map_routehandles_initfrom_fieldbundle(n1, n2, FBsrc, FBdst, mapin
call med_methods_FB_getFieldN(FBDst, 1, flddst, rc)
if (chkerr(rc,__LINE__,u_FILE_u)) return

call med_map_routehandles_initfrom_field(n1, n2, fldsrc, flddst, mapindex, routehandle(n1,n2,:), rc=rc)
if (present(mapfile)) then
call med_map_routehandles_initfrom_field(n1, n2, fldsrc, flddst, mapindex, routehandle(n1,n2,:), &
mapfile=mapfile, rc=rc)
else
call med_map_routehandles_initfrom_field(n1, n2, fldsrc, flddst, mapindex, routehandle(n1,n2,:), rc=rc)
end if
if (chkerr(rc,__LINE__,u_FILE_u)) return

if (dbug_flag > 1) then
Expand Down
6 changes: 5 additions & 1 deletion mediator/med_phases_aofluxes_mod.F90
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -582,7 +582,11 @@ subroutine med_aofluxes_init_ogrid(gcomp, aoflux_in, aoflux_out, rc)
if (chkerr(rc,__LINE__,u_FILE_u)) return
call ESMF_FieldGet(lfield, mesh=lmesh, rc=rc)
if (chkerr(rc,__LINE__,u_FILE_u)) return
is_local%wrap%aoflux_mesh = ESMF_MeshCreate(lmesh, rc=rc)
! Reuse the ocean field's mesh directly rather than building a duplicate full
! ESMF mesh per rank (a meaningful memory savings at high resolution). lmesh
! persists in FBArea(compocn) and aoflux_mesh is never destroyed, so sharing
! the handle is safe (ogrid path).
is_local%wrap%aoflux_mesh = lmesh
call ESMF_MeshGet(lmesh, coordSys=coordSys, rc=rc)
if (chkerr(rc,__LINE__,u_FILE_u)) return
if (coordSys /= ESMF_COORDSYS_CART) then
Expand Down
Loading