Parse PSS/E v34 RAW files and tolerate two record shapes that aborted the build - #61
Merged
Merged
Conversation
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
jd-lara
marked this pull request as ready for review
September 22, 2026 18:57
m-bossart
self-requested a review
September 23, 2026 16:56
m-bossart
requested changes
Sep 23, 2026
m-bossart
left a comment
There was a problem hiding this comment.
Proposal: reject duplicate TRANSFORMER records in the pti → PowerModels translation
Move the duplicate-transformer rejection out of the openapi layer and into
_psse2pm_transformer! (src/pm_io/psse.jl).
Why there
ptistays a faithful record reader._psse2pm_transformer!iteratespti_data["TRANSFORMER"], aVectorin file order, so
"keep the first record" is guaranteed by construction rather than by sorting on pm
indexdownstream.- Every consumer of
parse_file/PowerModelsDatagets the PSS/E behavior, not just the
openapi build. - The rule is keyed on the PSS/E identity, not on a display name that
branch_name_formattercan change.
Changes in _psse2pm_transformer!
- Before the loop, create a
Setof seen identities. - For each record, build the key
(I, J, K, uppercase(strip(CKT))).
This one key covers two-winding (K == 0) and three-winding records.
It uses the RAW bus numbers, not the routed node-bus endpoints. - If the key was already seen,
@warnwith the buses and CKT, thencontinue.
Otherwise record it and build the entry as today. - Write the stripped CKT into the two-winding
source_id(currently the raw
pop!(transformer, "CKT")).
Three-winding already strips it for"circuit".
Changes in openapi
- Remove
_is_duplicate_branchand its call inread_branches!
(src/openapi/branch.jl). register!goes back to being the only guard.
A duplicate name reaching it then means a parser bug, and it fails loudly.
Tests
- Add a parse-level test in
test/test_parse_psse.jlusing
synthetic_v35_duplicate_transformer_names.raw.
It checks the warning, a single 201–202 transformer entry holding the first record's
values (x = 0.05, rating 40), and that the 203–204 transformer is untouched. - Add a record whose CKT differs only in whitespace (e.g.
' 1'vs'1 ') to the
fixture, or to a new fixture.
Both current duplicate records use'1 ', so the fixture doesn't cover the real case. - Keep the openapi test in
test/test_openapi_branch.jlas a build test, but drop the
warning expectation.
It then checks that the build succeeds and that one circuit survives.
v34 shares v35's file structure but keeps the v33 field order in seven records and appends its new fields at the end, where v35 inserts them mid-record. The parser treated any file with @! column comments as v35, so a real rawd34 export failed on its first generator record. Stage 1 - Detect the revision from the REV header field for every file; @! comments only mark columns. Accept 29, 30, 32, 33, 34 and 35. - Replace the three-way table selector with one (sections, dtypes, defaults) registry per layout family, so parsing and defaulting can no longer disagree about the version. - Add v34 tables for LOAD, GENERATOR, TWO-TERMINAL DC, VSC sublines, FACTS, SWITCHED SHUNT and the transformer winding lines, where a real v34 export writes NODn last, after CNXAn, unlike the manual's summary. - Accept the three-field v34 substation 'S' terminal, defaulting its id. - Wire the v35 default variants into the v35 registry; they were defined but never used. Stage 2 - Branch on two version-family constants instead of literal revision strings, with dedicated v34 arms for generator and switched shunt. Fixes exposed by the v34 file - Strip double as well as single quotes from string fields; DC-line and FACTS names carried literal quote characters. - Give the v35 VSC subline table its sixteenth field. NREG had been read as RMPCT and RMPCT dropped; the fixture only had fifteen fields.
…g the build Two-winding transformers on the same bus pair with the same circuit id collided in the id registry. Keep the record read first and drop later duplicates with a warning naming the PSS/E record. How PSS/E resolves the repeat is still to be confirmed. The VSC reader errored when neither converter was TYPE 1, the shape PSS/E writes for an out-of-service line. Gate that check on the in-service flag and keep such lines as unavailable with placeholder setpoints and a warning.
Move the duplicate check from read_branches! into _psse2pm_transformer!, where the RAW identity is still available. Records are keyed by their RAW bus numbers and the stripped, case-folded circuit id, covering two- and three-winding transformers alike. The record read first is kept; later duplicates are dropped with a warning naming the buses and circuit id. The two-winding source_id now stores the stripped circuit id, as the three-winding path already did. register! is again the only guard in the OpenAPI layer, so a duplicate name reaching it is a parser bug and fails loudly. The fixture's duplicate now differs only in circuit-id whitespace, '1 ' against ' 1', which is the case that occurs in practice.
m-bossart
force-pushed
the
ac/v34_updates
branch
from
September 25, 2026 00:26
2909a79 to
ddfc2ee
Compare
m-bossart
self-requested a review
September 25, 2026 00:37
m-bossart
approved these changes
Sep 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
v34 parsing
Duplicate two-winding transformers
Out-of-service VSC lines
Verification