Skip to content

Make the .mec reader say what is wrong instead of crashing or hanging - #33

Merged
remislp merged 1 commit into
masterfrom
fix/mec-reader
Nov 12, 2026
Merged

remislp merged 1 commit into
masterfrom
fix/mec-reader

Conversation

@remislp

@remislp remislp commented Nov 12, 2026

Copy link
Copy Markdown
Member

A survey of one collection of 29 .mec files found every way mec_load and mec_get_list failed. Each is now handled:

file is not a .mec file qHETGLY15.mec is an HJCFIT text printout
with a .mec name. mec_get_list raises
ValueError quoting how the file begins.
index points at no record Bgtvlt30.mec: the bytes where its only record
should start give version 0 and over a
billion states, and mec_load then read a
"drawing" of over a billion cells, one empty
read past the end of the file at a time -
for ever. The record header is now checked,
and so is the drawing's size against what is
left of the file.
header contradicts itself qmechsDC1.mec @42458: 5 states, but 2+1+3+0
of the four kinds. ValueError, naming both.
rates use undeclared states one record shared by qmechs.mec,
qmechdem102.mec, qmechdemw.mec and
qmechsDC1.mec declares 3 states; its rates
connect 6. ValueError, naming the state.
no state names, or names CKSIM3B.MEC stores none; independent-subunit
for subunits only models store their subunits' state names (8
for 16 states) and their basic rate
constants' names (12 for 48 rates). Stored
names are now used only when there is one per
state or rate; otherwise states are S1, S2,
... and rates q(i,j). Putting subunit names on
the first states would label the mechanism
wrongly. These records now load.
fields read past the record six trailing integers, never used, are not in
every record: in demo.mec each record ends at
its state names, so reading them took 24 bytes
of the next record and ran off the end of the
file on the last. They are no longer read.

Every failure is now a ValueError naming the file, the record and what is wrong - never an IndexError, an EOFError or a hang.

Compared record by record with master across the whole collection: all 352 records that loaded before load identically (states, rate names, cycles, layout, Q to 12 decimals); 8 more now load; 5 are refused as contradictory.

Not done, and documented in mec_load: the constraints an independent-subunit model stores are still read but not applied. Read as "rate = factor x rate" they hold in some records and contradict the stored rates by factors up to a million in others, so they are not reliable enough to fill in rates from. qmechsDC1.mec @68054, which stores only its 12 basic rate constants, loads with its other 36 rates at zero, as ten other records already load with zero rates.

Also removes the hand-over to dcio.formats.mec. The module docstring said .mec reading had moved there; no such module exists in dcio, so the reader here was always the one used. Had one appeared, every .mec file would have been read by it instead, silently, without any of the above.

The rules are tested without files; the tests that load the actual records need SCALCS_MEC_DIR and skip without it.

A survey of one collection of 29 .mec files found every way mec_load and
mec_get_list failed. Each is now handled:

  file is not a .mec file      qHETGLY15.mec is an HJCFIT text printout
                               with a .mec name. mec_get_list raises
                               ValueError quoting how the file begins.
  index points at no record    Bgtvlt30.mec: the bytes where its only record
                               should start give version 0 and over a
                               billion states, and mec_load then read a
                               "drawing" of over a billion cells, one empty
                               read past the end of the file at a time -
                               for ever. The record header is now checked,
                               and so is the drawing's size against what is
                               left of the file.
  header contradicts itself    qmechsDC1.mec @42458: 5 states, but 2+1+3+0
                               of the four kinds. ValueError, naming both.
  rates use undeclared states  one record shared by qmechs.mec,
                               qmechdem102.mec, qmechdemw.mec and
                               qmechsDC1.mec declares 3 states; its rates
                               connect 6. ValueError, naming the state.
  no state names, or names     CKSIM3B.MEC stores none; independent-subunit
  for subunits only            models store their subunits' state names (8
                               for 16 states) and their basic rate
                               constants' names (12 for 48 rates). Stored
                               names are now used only when there is one per
                               state or rate; otherwise states are S1, S2,
                               ... and rates q(i,j). Putting subunit names on
                               the first states would label the mechanism
                               wrongly. These records now load.
  fields read past the record  six trailing integers, never used, are not in
                               every record: in demo.mec each record ends at
                               its state names, so reading them took 24 bytes
                               of the next record and ran off the end of the
                               file on the last. They are no longer read.

Every failure is now a ValueError naming the file, the record and what is
wrong - never an IndexError, an EOFError or a hang.

Compared record by record with master across the whole collection: all 352
records that loaded before load identically (states, rate names, cycles,
layout, Q to 12 decimals); 8 more now load; 5 are refused as contradictory.

Not done, and documented in mec_load: the constraints an independent-subunit
model stores are still read but not applied. Read as "rate = factor x rate"
they hold in some records and contradict the stored rates by factors up to a
million in others, so they are not reliable enough to fill in rates from.
qmechsDC1.mec @68054, which stores only its 12 basic rate constants, loads
with its other 36 rates at zero, as ten other records already load with zero
rates.

Also removes the hand-over to dcio.formats.mec. The module docstring said
.mec reading had moved there; no such module exists in dcio, so the reader
here was always the one used. Had one appeared, every .mec file would have
been read by it instead, silently, without any of the above.

The rules are tested without files; the tests that load the actual records
need SCALCS_MEC_DIR and skip without it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@remislp
remislp merged commit 9edab78 into master Nov 12, 2026
7 of 8 checks passed
@remislp
remislp deleted the fix/mec-reader branch November 12, 2026 20:26
@remislp remislp mentioned this pull request Nov 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant