Improve treatment of solar spectrum by mo-jmanners · Pull Request #39 · MetOffice/socrates

James Manners (mo-jmanners) · 2026-09-23T16:30:05Z

PR Summary

Improve treatment of solar spectrum

Sci/Tech Reviewer: Dr Stephan Havemann (@Petzi1)
Code Reviewer: Pierre Siddall (@Pierre-siddall)

New scripts are introduced to create a merged high-resolution solar spectrum, in particular: examples/solar/mk_solar_cmip7_cyc23_whi_lya
- This calls a number of new routines to create:
  - a very high-resolution spectrum for the Lyman-alpha region
  - a high-resolution (0.1nm) spectrum for the UV
  - a low-resolution (1nm) CMIP7 mean spectrum for solar cycle 23
- Then merges these three spectra and fits an effective temperature to use for the longwave tail
- The resulting solar spectrum (data/solar/cmip7_cyc23_whi_lya_mean_tfit) is planned to be the standard for creating upcoming SW spectral files.
Fixes are made in src/general/solar_intensity_nu.f90 (previously solar_intensity_90.f90) to properly interpolate a binned solar spectrum with constant solar flux over the defined bins. The longwave tail is now also fitted with a Planck function rather than a Rayleigh-Jeans approximation.
- This slightly changes KGOs for the offline Socrates tests by changing the fraction of the solar spectrum calculated for block 2 of the spectral files (does not affect LFRic/UM runs with existing spectral files).
Spectral variability files made with src/general/make_block_17.f90 will now give variations on a smaller number of "var" bands that will then be mapped to the higher-resolution spectral file sub-bands.
The data type for holding the spectral file information once it is read in has been optimised to reduce memory usage. The most significant changes involve removing high-dimensional arrays that are sparsely filled and replacing them with nested defined types that can be allocated to hold only the required data. These optimisations affect the routines: def_spectrum, read_spectrum, map_sub_bands_mod, radiance_calc, inter_k, out_spectrum and make_block_5.

linked MetOffice/um#143

closes Improve treatment of solar spectrum for correlated-k #38

Code Quality Checklist

(Some checks are automatically carried out via the CI pipeline)

I have performed a self-review of my own code
My code follows the project's style guidelines
Comments have been included that aid understanding and enhance the readability of the code
My changes generate no new warnings

Testing

If shared files have been modified, I have run the UM and LFRic Apps rose stem suites
If any tests fail (rose-stem or CI) the reason is understood and acceptable (eg. kgo changes)
I have added tests to cover new functionality as appropriate (eg. system tests, unit tests, etc.)

Socrates stand-alone quick_tests and slow_tests all pass with changes to KGOs when creating block 2 of spectral files (solar spectrum fraction):

[~/git/socrates_admin/examples](38_solar %=):$ ./quick_tests 
Testing code compiled in /home/users/james.manners/git/socrates_admin/bin
---
Testing simple calls to the runes interface:
Matched gfortran output
OK
---
Testing the runes_nc driver using LFRic diagnostic input:
Matched SW output
Matched LW output
OK
---
Testing raw_input to convert column data into input files:
OK
---
Testing Cl_run_cdl for ICRCCM case 27 (LW) on AER profiles:
OK
---
Testing Cl_run_cdf on multiple profiles:
OK
---
Test Cl_run_cdf for CIRC case 6
(HadGEM, GA7, SES and 300/260 band spectral files):
OK
---
Testing pseudo-spherical geometry code:
Calculating zenith angles:  75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98
OK
---
Testing photolysis and non-LTE code:
Mapping sub-bands to channels.
Matched gfortran output
OK
---
Test McICA code (create mcica_data file and run on CRM profile):
OK
---
Testing creation of aerosol properties for a spectral file:
Running scatter_90...
Running scatter_average_90...
Running prep_spec...
Matched gfortran12 output
OK
---
Testing generating and running with prescribed optical properties:
Running raw_input...
Running Cscatter... please wait...
Running Cscatter... please wait...
Running Cscatter_average...
Running Cscatter_average...
Running prep_opt_profile...
Running Cl_run_cdl...
All done.
OK
---
Testing Crun_mono to calculate radiances:
Matched ref output
OK
---
Testing Ccorr_k to generate correlated-k coefficients:
Matched gfortran12 output
OK
---
Testing flexchem chemistry:
Matched KGO output
OK
---
Testing CDL scripts:
OK
---
All passed.

LFRic Apps trac.log

Developer LFRic Apps tests all pass (no lfric_apps changes required):

Test Suite Results - lfric_apps - lfric_apps_socrates38_solar/run3

Suite Information

Item	Value
Suite Name	lfric_apps_socrates38_solar/run3
Suite User	james.manners
Workflow Start	2026-09-23T15:34:08
Groups Run	developer

Dependency	Reference	Main Like
casim	MetOffice/casim@2026.07.1	True
jules	MetOffice/jules@c22aeea	True
lfric_apps	mo-jmanners/lfric_apps@08ac5f6	True
lfric_core	MetOffice/lfric_core@220a00c	True
moci	MetOffice/moci@2026.07.1	True
SimSys_Scripts	MetOffice/SimSys_Scripts@77a5166	True
socrates	mo-jmanners/socrates@38_solar	True
socrates-spectral	MetOffice/socrates-spectral@2026.07.1	True
ukca	MetOffice/ukca@c372b36	True

Task Information

✅ succeeded tasks - 1216

UM trac.log

UM tests require a linked UM change for broadcast of the changes to the spectral type: MetOffice/um#143

Test Suite Results - um - um_socrates38_solar/run24

Suite Information

Item	Value
Suite Name	um_socrates38_solar/run24
Suite User	james.manners
Workflow Start	2026-09-23T15:33:00
Groups Run	developer

Dependency	Reference	Main Like
casim	MetOffice/casim@2025.12.1	True
jules	MetOffice/jules@2025.12.1	True
moci	MetOffice/moci@2025.12.1	True
mule	MetOffice/mule@2025.10.1	True
shumlib	MetOffice/shumlib@2025.10.1	True
socrates	mo-jmanners/socrates@38_solar	True
SimSys_Scripts	MetOffice/SimSys_Scripts@2025.12.1	True
ukca	MetOffice/ukca@2025.12.1	True
um	mo-jmanners/um@141_solar	False
um_aux	MetOffice/um_aux@2025.12.1	True
um_meta	MetOffice/um_meta@2025.12.1	True

Approvals

Code Owners

Section	Owner	Deputy	State
radiation_control	jamesmanners	stephanhavemann	Pending

Config Owners

No UM Config Owners Required

Task Information

❌ failed tasks - 1

Task	State
script_azspice_metadata_check_validate	failed

✅ succeeded tasks - 878

⌛ waiting tasks - 1

Task	State
housekeep_azspice	waiting

Security Considerations

I have reviewed my changes for potential security issues
Sensitive data is properly handled (if applicable)
Authentication and authorisation are properly implemented (if applicable)

Performance Impact

Performance of the code has been considered and, if applicable, suitable performance measurements have been conducted

AI Assistance and Attribution

Some of the content of this change has been produced with the assistance of Generative AI tool name (e.g., Met Office Github Copilot Enterprise, Github Copilot Personal, ChatGPT GPT-4, etc) and I have followed the Simulation Systems AI policy (including attribution labels)

Documentation

Where appropriate I have updated documentation related to this change and confirmed that it builds correctly

Sci/Tech Review

I understand this area of code and the changes being added
The proposed changes correspond to the pull request description
Documentation is sufficient (do documentation papers need updating)
Sufficient testing has been completed

Please alert the code reviewer via a tag when you have approved the SR

Code Review

All dependencies have been resolved
Related Issues have been properly linked and addressed
CLA compliance has been confirmed
Code quality standards have been met
Tests are adequate and have passed
Documentation is complete and accurate
Security considerations have been addressed
Performance impact is acceptable

…7, WHI and Lyman-alpha data; Fix issues with the interpretation of the solar spectrum in the correlated-k code so that sub-band irradiance is interpolated correctly and the long wavelength tail uses a full Planck function rather than a Rayleigh-Jeans approximation; Extend functionality for solar spectral variability allowing variations at lower spectral resolution than spectral file sub-bands by merging in the high-res (sub-band) solar spectrum; Significant decrease in memory usage for spectrum type.

…ot have an extended file

Dr Stephan Havemann (Petzi1)

I am happy with the changes and additions.

Pierre Siddall (Pierre-siddall)

Hi James Manners (@mo-jmanners), I'm afraid I have quite a large laundry list of issues on this PR. But this feels to me like a case of a few mistakes which have been propogated throughout the rest of the PR. For future reference it would be useful to revisit the coding standards these will help you to avoid a long list of actions from a PR in future. If my comments have missed any context from a scientific perspective please feel free to let me know and I'd be happy to discuss with you.

Pierre Siddall (Pierre-siddall) · 2026-09-28T09:15:37Z

+linenum = 0
+scale_wv=1.0
+scale_irr=1.0
+with open(filename1, 'r') as file:


These set of operation on a file could do with some comments explaining what the operations are doing.

Pierre Siddall (Pierre-siddall) · 2026-09-28T09:17:48Z

+            dataend = linenum
+
+datalen = dataend-datastart-1
+if (l_binned):


Similarly here comments on why the wavelength and irradiance are multiplied by 2 when the line data is binned would be helpful to future developers.

Pierre Siddall (Pierre-siddall) · 2026-09-28T09:19:48Z

+    irradiance1 = np.zeros(datalen)
+
+total_irradiance=0.0
+with open(filename1) as file:


Comments on this set of file operations would be helpful.

Pierre Siddall (Pierre-siddall) · 2026-09-28T09:21:08Z

+
+l_sub_bands = False
+if (len(sys.argv) > 2):
+    with open(filename2, 'r') as file:


Similary, this also needs some documentation or comments added.

Pierre Siddall (Pierre-siddall) · 2026-09-28T09:31:19Z

+    if (l_sub_bands):
+        band = np.zeros(datalen, dtype=int)
+        kterm = np.zeros(datalen, dtype=int)
+        with open(filename2) as file:


Comments would also be helpful here.

Pierre Siddall (Pierre-siddall) · 2026-10-12T13:08:30Z

+! Scale to get the flux at 1 AU.
+  planck_tail = planck_tail * (Sol%radius/astronomical_unit)**2
+
+END FUNCTION planck_tail


Suggested change

END FUNCTION planck_tail

END SUBROUTINE planck_tail

Pierre Siddall (Pierre-siddall) · 2026-10-12T13:42:12Z

+
+
+! Dummy arguments.
+  INTEGER, Intent(IN) :: iu_solar


The casing needs to be kept the same here.

Suggested change

INTEGER, Intent(IN) :: iu_solar

INTEGER, INTENT(IN) :: iu_solar

Pierre Siddall (Pierre-siddall) · 2026-10-12T13:42:28Z

+! Dummy arguments.
+  INTEGER, Intent(IN) :: iu_solar
+!   Unit number for the solar spectrum
+  INTEGER, Intent(INOUT) :: ierr


Suggested change

INTEGER, Intent(INOUT) :: ierr

INTEGER, INTENT(INOUT) :: ierr

Pierre Siddall (Pierre-siddall) · 2026-10-12T13:42:39Z

+!   Unit number for the solar spectrum
+  INTEGER, Intent(INOUT) :: ierr
+!   Error flag
+  TYPE (StrSolarSpec), Intent(OUT) :: SolarSpec


Suggested change

TYPE (StrSolarSpec), Intent(OUT) :: SolarSpec

TYPE (StrSolarSpec), INTENT(OUT) :: SolarSpec

James Manners (mo-jmanners) · 2026-10-14T15:46:20Z

Thanks Pierre,

I think I've dealt with all your comments apart from the ones I've replied to above. I also haven't removed any of the stop statements. The exit statement is not equivalent and not generally appropriate. These routines are stand-alone Socrates programs that are not run in a parallel environment and are not part of LFRic / UM builds.

James

Pierre Siddall (Pierre-siddall) · 2026-10-14T16:19:34Z

Thanks Pierre,

I think I've dealt with all your comments apart from the ones I've replied to above. I also haven't removed any of the stop statements. The exit statement is not equivalent and not generally appropriate. These routines are stand-alone Socrates programs that are not run in a parallel environment and are not part of LFRic / UM builds.

James

Hi James, thanks for flagging this, I'll have a further chat about the stop statements with the SSD team tommorow as your right in saying that stop will have no effect on the code if not part of the UM/LFRIC builds. However,my concern is more to do with maintaining the same standards across all the code respositories and maintainability in the future should this become part of the UM/LFRIC builds for whatever reason and my gut feeling is this might still be a no I'm afraid but I'll see what I can do.

Pierre Siddall (Pierre-siddall) · 2026-10-19T13:58:09Z

Hi James Manners (@mo-jmanners), regarding the use of stop staments I have consulted with the working practices again and the rest of the SSD team and I think the verdict is if you could report the error via the Ereport function in your code we'd be grateful. This will ensure maintainability if the solar spectrum is intgerrated into the builds further down the line.

…MetOffice#44 and replace stop statements with ereport calls.

James Manners (mo-jmanners) · 2026-10-20T10:05:00Z

Thanks Pierre Siddall (@Pierre-siddall), I've introduced a socrates_ereport module as part of a longer term plan to disentangle the modules_core routines from their UM versions. This work will continue in #44 to properly replace ereport with socrates_ereport throughout the socrates code.

For now, this lets me provide a cleaner stop message for 'usage' errors for these stand-alone programs.

Pierre Siddall (Pierre-siddall) · 2026-10-21T07:54:10Z

Thanks Pierre Siddall (Pierre Siddall (@Pierre-siddall)), I've introduced a socrates_ereport module as part of a longer term plan to disentangle the modules_core routines from their UM versions. This work will continue in #44 to properly replace ereport with socrates_ereport throughout the socrates code.

For now, this lets me provide a cleaner stop message for 'usage' errors for these stand-alone programs.

Hi James Manners (@mo-jmanners) , another SSSE has explained the usage of stop statements to me futher and how to handle them in socrates in the future. Apologies again for my confusion regarding the strong wording on our documentation. I'm happy that I understand this is the way to procced now.

Pierre Siddall (Pierre-siddall)

I'm happy that the changes now make sense within the context of socrates and hence this PR is good to be tested, apologies again James Manners (@mo-jmanners) for my confusion on this, this has definetly been a learning experience for me on future socrates tickets.

James Manners (mo-jmanners) added 7 commits July 15, 2026 14:40

Merge remote-tracking branch 'upstream/main'

094f070

Merge remote-tracking branch 'refs/remotes/origin/main'

2a18bf7

MetOffice#38: Correct number of dimensions in def_spectrum

d82f56b

MetOffice#38: Satisfy UM copyright checker

c93a63a

MetOffice#38: Fix for compatibility with old spectral files that do n…

16549cb

…ot have an extended file

MetOffice#38: Correct number of unallocated integers in spectrum

7c6f49d

James Manners (mo-jmanners) added this to the Autumn 2026 milestone Sep 23, 2026

James Manners (mo-jmanners) self-assigned this Sep 23, 2026

James Manners (mo-jmanners) added enhancement New feature or request Linked UM This PR is linked to a MetOffice/um PR KGO This PR contains changes to KGO optimisation This PR adds optimisations or edits OpenMP code labels Sep 23, 2026

github-actions Bot requested a review from Dr Stephan Havemann (Petzi1) September 23, 2026 16:30

Dr Stephan Havemann (Petzi1) approved these changes Sep 24, 2026

View reviewed changes

github-actions Bot requested a review from Pierre Siddall (Pierre-siddall) September 24, 2026 16:10

James Manners (mo-jmanners) added 2 commits September 30, 2026 16:28

Merge branch 'main' into 38_solar

47f6fc2

MetOffice#38: Remove obsolete KGO file.

1aee20f

Harrison Nicholls (nichollsh) mentioned this pull request Oct 13, 2026

Update the FWL fork of SOCRATES to 2607 FormingWorlds/SOCRATES#32

Closed

Pierre Siddall (Pierre-siddall) requested changes Oct 14, 2026

View reviewed changes

This comment was marked as duplicate.

Sign in to view

MetOffice#38: Add comments and minor fixes from code review.

f389ffe

MetOffice#38: Bring forward use of socrates_ereport module planned for …

55577b5

…MetOffice#44 and replace stop statements with ereport calls.

James Manners (mo-jmanners) requested a review from Pierre Siddall (Pierre-siddall) October 20, 2026 10:05

Pierre Siddall (Pierre-siddall) approved these changes Oct 21, 2026

View reviewed changes

	INTEGER, Intent(IN) :: iu_solar
	INTEGER, INTENT(IN) :: iu_solar

	INTEGER, Intent(INOUT) :: ierr
	INTEGER, INTENT(INOUT) :: ierr

	TYPE (StrSolarSpec), Intent(OUT) :: SolarSpec
	TYPE (StrSolarSpec), INTENT(OUT) :: SolarSpec

Conversation

James Manners (mo-jmanners) commented Sep 23, 2026 • edited by james-bruten-mo Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Summary

Code Quality Checklist

Testing

LFRic Apps trac.log

Test Suite Results - lfric_apps - lfric_apps_socrates38_solar/run3

Suite Information

Task Information

UM trac.log

Test Suite Results - um - um_socrates38_solar/run24

Suite Information

Approvals

Code Owners

Config Owners

Task Information

Security Considerations

Performance Impact

AI Assistance and Attribution

Documentation

Sci/Tech Review

Code Review

Uh oh!

Dr Stephan Havemann (Petzi1) left a comment

Choose a reason for hiding this comment

Uh oh!

Pierre Siddall (Pierre-siddall) left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

This comment was marked as duplicate.

Uh oh!

James Manners (mo-jmanners) commented Oct 14, 2026

Uh oh!

Pierre Siddall (Pierre-siddall) commented Oct 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Pierre Siddall (Pierre-siddall) commented Oct 19, 2026

Uh oh!

James Manners (mo-jmanners) commented Oct 20, 2026

Uh oh!

Pierre Siddall (Pierre-siddall) commented Oct 21, 2026

Uh oh!

Pierre Siddall (Pierre-siddall) left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

James Manners (mo-jmanners) commented Sep 23, 2026 •

edited by james-bruten-mo

Loading

Pierre Siddall (Pierre-siddall) commented Oct 14, 2026 •

edited

Loading