[svahl991]

Description

Removes the parts of testinput_tier_1/instruments/radiance/cpr_cloudsat_obs_2009113003.nc4 that don't follow the OSDF (IFrame / FrameRows / FrameCols) dimension rules. These rules allow 1D or 2D variables only, and every 2D variable must use Location as its first dimension.

Removed:

• GsiHofX/ReflectivityAttenuated(Location_Extended, Channel), along with the Location_Extended dimension and the now-empty GsiHofX group. The OSDF reader skipped this variable; it was the only skipped variable in the ufo tier-1 tests.
• The nrecs dimension and the nrecs(nrecs) variable, which nothing uses.

Everything else is unchanged. All other variables are bit-identical, and the global and string attributes are kept.

Why GsiHofX is removed rather than reshaped

GsiHofX/ReflectivityAttenuated is not a flattened (Location, X) array. It holds GSI H(x) for the 147 companion locations (7 records × 21) that the ObsSpace extension creates at runtime. Those locations don't exist when the file is read, so the variable can't be expressed on Location.

The variable was also not actually checking anything. Since ufo #3588 / ufo-data #477, the CRTM active-sensor path only simulates companion profiles that the Average Observations To Model Levels filter has filled in, and test_ObsOperator doesn't run filters. H(x) is therefore all missing, and the old vector ref: GsiHofX comparison passed with "No observations".

The original values can still be recovered from git history if they are needed.

Companion PR

• ufo PR 4418: feature/cpr-cloudsat-for-osdf changes cpr_cloudsat_gfs_HofX.yaml to rms ref: 0.0, with a comment explaining why. That reference fails if H(x) ever has real values, which is the signal to add a proper filter-pipeline test with matching data.
  https://github.com/JCSDA-internal/ufo/pull/4418
  run-ci-on-draft=true

Testing

• ufo_instrument_cpr_cloudsat_gfs_HofX passes with the ObsGroup and OSDF (FrameCols) containers.
• ufo_data_validate_cpr_cloudsat_obs_2009113003.nc4 passes.

[svahl991]

This PR works, but has been moved to draft status until we decide if we want to merge this and https://github.com/JCSDA-internal/ufo/pull/4418 as a "resolution" for https://github.com/JCSDA-internal/ufo/issues/4419 and https://github.com/JCSDA-internal/ioda/issues/1844

[svahl991]

Note that this PR cannot be merged without also merging https://github.com/JCSDA-internal/ufo/pull/4418, however the UFO PR can merge without this one.

[lewisn-met]

Thanks for debugging this issue! I agree that this and the companion UFO PR aren't suitable as long-term solutions, but given that they aren't a regression, I'm happy to merge until a fix is found. Probably best not to close the original issue though.

[srherbener]

Thanks!