Identity-only SSSOM refresh mismatches tool provenance and corrupts YAML descriptions
Related: #1236 (bounded Potato material-scope hold). This is a preexisting
metadata defect exposed when preparing its separate identity-policy candidate;
it is not evidence that the underlying mapping rows were generated incorrectly.
Reproduction 1: inconsistent tool/version pair
scripts/consolidate_chemical_mappings.py:3063 replaces
mapping_tool_version with its own script SHA-256, but copies the predecessor's
mapping_tool unchanged. A valid one-row water SSSOM with the actual historical
reconstruction metadata produces:
| Field |
Before |
After identity-only refresh |
| mapping_tool |
kg-microbe/scripts/mim_conservative_refresh.py |
kg-microbe/scripts/mim_conservative_refresh.py |
| mapping_tool_version |
sha256:58ec62b9b01a28f4f3b6b47ff319a11396617d4a07ada409acc2742bcb77ae6a |
sha256:0d651d97ed0881c335f361f49486be2d9fc8902878487e3588569b4e4db98912 |
The latter hash identifies consolidate_chemical_mappings.py, not the script
named by mapping_tool. This reproducibly publishes an internally inconsistent
provenance pair, even though the scientific row is unchanged. The actual local
SSSOM validator passes both outputs; a second cycle is byte-identical and
therefore does not detect or repair the mismatch.
Reproduction 2: a literal quote is added to the description
The same actual output independently exposes another metadata corruption at
scripts/consolidate_chemical_mappings.py:3067: the code assumes the description
is a double-quoted, one-line YAML scalar, removes an optional closing quote and
unconditionally adds a quote after its fingerprint. The valid plain YAML scalar
in the tiny fixture changes semantically from:
Tiny independent provenance reproduction.
to this parsed string (the final double quote is part of the value):
Tiny independent provenance reproduction. Ingredient identity policy sha256:61786a46effa48de9901f77713b172db16a7d6797f35711d5c15c01b0e0ea926."
The second cycle preserves the corrupted value; deterministic output is not
proof that the source description was preserved. The current immutable unified
baseline also uses a plain, not quoted/folded scalar:
mapping_set_description: Conservative MIM candidate; reviewed manifest sha256:9bb29d5605d93dea351be9624d99c5d8ada57d4b9831b22764957b784bd685af.
Therefore the real identity-only candidate route has the same affected input
shape. Its baseline bytes remain unchanged. Single-quoted and multiline YAML
descriptions are adjacent unsupported forms that need regression coverage;
this report does not claim they were present in the current baseline.
Saved reproduction (tiny owned files only, no production reads/writes):
reproduce_identity_refresh_provenance.pyprovenance-repro-01/result.jsonprovenance-repro-01/tiny-baseline.tsvprovenance-repro-01/first.tsv.gz and second.tsv.gz
Both cycles report one row read, zero removals, and zero relabelings. Their
identical compressed SHA-256 is
1f9b9efe4de192d70cf1200a5defb6bd069b4e9114c5c24c5e5c8729c08f49c2.
The baseline SHA-256 is
51d29c1258fe37dfc5a6ebf2f5e6f404c28a0e8ebd240c3e03150f28bcfa59b1.
Minimal correction and acceptance
Parse and edit the semantics of only the three affected top-level YAML
fields (mapping_tool, mapping_tool_version, and
mapping_set_description), not line suffixes that assume a particular scalar
style. Preserve unrelated metadata, all assertion dates, the literal TSV
header and unrelated serialized data rows. Do not mutate the whole metadata
mapping as a side effect of reformatting it. Missing tool/version fields must
gain the truthful pair; malformed or ambiguous metadata must fail safely.
When this entry point performs identity-only refresh, set mapping_tool to
kg-microbe/scripts/consolidate_chemical_mappings.py alongside the actual script
fingerprint in mapping_tool_version. Do not misidentify the current writer in
order to retain predecessor provenance; bind the immutable baseline identity
and original metadata in the separate candidate/delta report instead.
Add hermetic tests using a different predecessor tool/version that require:
- The refreshed tool names the actual identity-refresh script and its version
is that script's content hash.
- Plain, single-quoted, double-quoted, folded (
>), and literal (|)
descriptions retain their original parsed meaning before the one appended
current policy fingerprint, including intended quotes and newlines.
- A missing tool/version pair is inserted accurately; unrelated metadata,
assertion dates, and complete serialized rows remain unchanged except for
existing documented policy effects.
- Repeating the refresh is byte-identical and there is exactly one current
policy fingerprint, without a stray quote or lost description content.
Regenerate only a separate reviewed candidate after that fix. The current
candidate is diagnostic, not promotable. Keep supported MIM rows, release pin,
and historical fixtures unchanged. Update the native artifact-pair expectations
only after independently verifying the replacement candidate and exact delta.
Identity-only SSSOM refresh mismatches tool provenance and corrupts YAML descriptions
Related: #1236 (bounded Potato material-scope hold). This is a preexisting
metadata defect exposed when preparing its separate identity-policy candidate;
it is not evidence that the underlying mapping rows were generated incorrectly.
Reproduction 1: inconsistent tool/version pair
scripts/consolidate_chemical_mappings.py:3063replacesmapping_tool_versionwith its own script SHA-256, but copies the predecessor'smapping_toolunchanged. A valid one-row water SSSOM with the actual historicalreconstruction metadata produces:
The latter hash identifies consolidate_chemical_mappings.py, not the script
named by
mapping_tool. This reproducibly publishes an internally inconsistentprovenance pair, even though the scientific row is unchanged. The actual local
SSSOM validator passes both outputs; a second cycle is byte-identical and
therefore does not detect or repair the mismatch.
Reproduction 2: a literal quote is added to the description
The same actual output independently exposes another metadata corruption at
scripts/consolidate_chemical_mappings.py:3067: the code assumes the descriptionis a double-quoted, one-line YAML scalar, removes an optional closing quote and
unconditionally adds a quote after its fingerprint. The valid plain YAML scalar
in the tiny fixture changes semantically from:
to this parsed string (the final double quote is part of the value):
The second cycle preserves the corrupted value; deterministic output is not
proof that the source description was preserved. The current immutable unified
baseline also uses a plain, not quoted/folded scalar:
Therefore the real identity-only candidate route has the same affected input
shape. Its baseline bytes remain unchanged. Single-quoted and multiline YAML
descriptions are adjacent unsupported forms that need regression coverage;
this report does not claim they were present in the current baseline.
Saved reproduction (tiny owned files only, no production reads/writes):
reproduce_identity_refresh_provenance.pyprovenance-repro-01/result.jsonprovenance-repro-01/tiny-baseline.tsvprovenance-repro-01/first.tsv.gzandsecond.tsv.gzBoth cycles report one row read, zero removals, and zero relabelings. Their
identical compressed SHA-256 is
1f9b9efe4de192d70cf1200a5defb6bd069b4e9114c5c24c5e5c8729c08f49c2.The baseline SHA-256 is
51d29c1258fe37dfc5a6ebf2f5e6f404c28a0e8ebd240c3e03150f28bcfa59b1.Minimal correction and acceptance
Parse and edit the semantics of only the three affected top-level YAML
fields (
mapping_tool,mapping_tool_version, andmapping_set_description), not line suffixes that assume a particular scalarstyle. Preserve unrelated metadata, all assertion dates, the literal TSV
header and unrelated serialized data rows. Do not mutate the whole metadata
mapping as a side effect of reformatting it. Missing tool/version fields must
gain the truthful pair; malformed or ambiguous metadata must fail safely.
When this entry point performs identity-only refresh, set
mapping_tooltokg-microbe/scripts/consolidate_chemical_mappings.pyalongside the actual scriptfingerprint in
mapping_tool_version. Do not misidentify the current writer inorder to retain predecessor provenance; bind the immutable baseline identity
and original metadata in the separate candidate/delta report instead.
Add hermetic tests using a different predecessor tool/version that require:
is that script's content hash.
>), and literal (|)descriptions retain their original parsed meaning before the one appended
current policy fingerprint, including intended quotes and newlines.
assertion dates, and complete serialized rows remain unchanged except for
existing documented policy effects.
policy fingerprint, without a stray quote or lost description content.
Regenerate only a separate reviewed candidate after that fix. The current
candidate is diagnostic, not promotable. Keep supported MIM rows, release pin,
and historical fixtures unchanged. Update the native artifact-pair expectations
only after independently verifying the replacement candidate and exact delta.