Move jd/market_models2 onto the OpenAPI 1.x line - #302
Merged
Merged
Conversation
Adds supports_events(::Type{<:PSY.Component}) plus
get_initial_parameter_value/get_parameter_multiplier methods for the
event parameter types, enabling outage-event discovery and defaults.
Add PowerOperationsProblemTemplate.events, set_event_model!/get_event_models so a template can hold EventModels ahead of build-time distribution to DeviceModels.
Populate EventModel.attribute_device_map from the system's supplemental attributes during template validation, validate each event's time-series mapping, and distribute the event to every DeviceModel whose device type carries the attribute. Preserve event-model identity across the template deep copy performed at DecisionModel construction (mirroring the existing PNM-matrix-sharing trick) so callers can inspect discovery results on the same EventModel object they attached to the template. Also fix a name collision from the earlier event-model work: POM's get_value(::StateVariableValueCondition) was defined without qualifying IOM.get_value, which created a separate local generic function that shadowed IOM's get_value(::InitialCondition) for every other unqualified caller in the package (storage, hybrid, thermal generation, AGC), silently breaking all initial-conditions-consuming builds.
Two distinct event models of the same contingency type discovering the same device type can't both be registered under the device model's single (contingency type, device type) events slot. Replace the silent-skip guard in _build_device_model_events! with a loud error when the existing registration belongs to a different event model than the one being processed; re-discovering the same event model for the same key stays a no-op. Add a covering test case. Also reword the _deepcopy_template override comment to describe the deepcopy-unsafe solver-cache behavior directly instead of citing a PR number.
Wires DeviceModel.events into the ArgumentConstructStage: add_parameters! creates AvailableStatusParameter and AvailableStatusChangeCountdownParameter containers for devices carrying a matching supplemental attribute, and a generic add_to_expression! offsets those parameters into the system balance via _balance_expression_targets, replacing PSI's four per-network methods with one. add_event_arguments! now overrides the no-op stub for PSY.StaticInjection devices.
Loads (StaticPowerLoad/PowerLoadDispatch/PowerLoadInterruption) and FixedOutput devices now inject an ActivePowerOffsetParameter (and ReactivePowerOffsetParameter on reactive-capable networks) directly into the system balance expression when an event is attached, on top of the generic status/countdown parameters from the StaticInjection default.
…vices Implements add_event_constraints! for ThermalGen/RenewableGen/ElectricLoad across active-only and reactive-capable networks, bounding dispatch by available capacity during an outage event and adding a quadratic reactive power bound on reactive-capable networks.
Ports HydroGen/HydroPumpTurbine/EnergyReservoirStorage add_event_constraints! methods and their pump/input-output contingency-constraint helpers, plus fixes a get_variable/add_constraints_container! instance-vs-type bug in the shared reactive-power contingency helper (never previously exercised by any test).
Covers full-template build+solve across CopperPlate/PTDF/DCP/ACP network models with a FixedForcedOutage event, the PTDF two-target balance offset and ACP reactive-offset paths for load events, and a recurrent-solve mock test confirming a forced outage drives thermal output to zero. Also strengthens the initial-conditions exclusion test to confirm event parameters land in the main container while staying absent from the IC container.
…es style Aligns the new E2E thermal-outage testset with the naming convention already established in test_model_decision.jl (both are exported by IOM and used unqualified there).
The prior testset only checked that ReactivePowerOffsetParameter and ReactivePowerBalance both exist, which proves nothing about the offset being wired into the balance (the expression is allocated for every ACP build regardless of events). Add a recurrent-solve mock testset that checks JuMP.coefficient of the offset parameter's variable in the balance expression directly, so the test fails if the wiring is ever removed.
Adds an Outage events subsection covering the availability parameters and per-device-family outage constraints added when an EventModel is attached to a template. public.md needs no changes: it registers symbols via a blanket @autodocs, which already picks up the new exports.
Record the completed event framework port in pom_port_plan.md (Workstream C and execution order), and add the SDD plan and design spec artifacts for the branch.
Add an objective to the forced-outage mock test so it fails when ActivePowerOutageConstraint is removed, assert the constraint's baked RHS equals the device's max active power, cover the FixedOutput offset path, and smoke-test the event condition accessors. Also correct the ReactivePowerOutageConstraint docstring/docs math to match the max((Q^max)^2, (Q^min)^2) implementation, and update the stale "no event framework" claim in .claude/CLAUDE.md.
`IS.get_uuid` is gone in IS4: supplemental attributes are addressed by integer
id, which is what the security-constrained outage pass on main already uses
(`IS.get_id` / `PSY.get_supplemental_attribute(sys, id)`). Retype
`EventModel.attribute_device_map` to `Dict{Int, ...}` and follow that idiom in
build-time event discovery.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NjXEBDreXLvbXw3pLKtAVR
These were ported from PSI/SSS and then commented out because `mock_construct_device!` errored on `add_event_model = true`; the events port made that path work, so uncomment them. Every ported count matched reality unchanged: +24 LessThan (one ActivePowerOutageConstraint per timestep for the one device the mock attributes), +48 for storage (charge and discharge), and +48 variables only under `built_for_recurrent_solves`, where event parameters are JuMP parameters rather than Float64. The two storage `EnergyTargetFeedforward` testsets stay commented: they are blocked on the StorageSystemsSimulations feedforward port, not on events. The comment now says so. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NjXEBDreXLvbXw3pLKtAVR
Counts and set types pass through a units error, a swapped variable, or a missing square. The shared upper-bound path is component-neutral (`IOM.get_max_active_power` has one POM method, which passes `PSY.SU`), so the existing thermal RHS check already covers units for renewable/load/hydro too; what is unchecked is the LHS choice per family and the three builders that compute their own right-hand sides. Added: - each family's outage constraint carries its own power variable, coefficient 1; - storage input/output constraints are not swapped, with bounds recomputed from the fixture in system base; - the reactive constraint is q^2 bounded by the *squared* reactive limit: tight exactly at the limit, symmetric in sign, satisfied inside and violated outside; - outages drive storage charge and discharge, and hydro pump turbine and pump variables, to zero under an objective that maximizes them; - load offsets reach the balance row with the event-parameter multiplier of 1.0; - a thermal outage costs exactly what forcing the same unit off costs, on a copperplate UC problem. The last one needs a commitment template: the outage bounds active power from above only, so a dispatch formulation enforcing `p >= p_min` is infeasible under an outage. That is what PSI's `has_outage` feedforward override relaxes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NjXEBDreXLvbXw3pLKtAVR
Most event testsets opened with the same ten lines of fixture plumbing. Moved that into test_utils/events_test_utils.jl: - `fixed_outage_event` — the FixedForcedOutage EventModel with its status series; - `build_outage_model` — attach the attribute, build a DecisionModel for it; - `mock_event_container` — mock-construct a device model with the outage attached; - `outaged_name` / `outaged_device` — the attributed device, read back off the availability parameter rather than picked from the system a second time; - `maximize_under_outage` — force the outage and maximize what it should suppress; - `outage_zero_output_cases` — the device families that check drives to zero. The three near-identical behavioral testsets (thermal, storage, hydro pump) are now one loop over that last table, and the load-offset coefficient testset that the multiplier check subsumes is gone. Lookups by name use the three-argument `PSY.get_component` instead of scanning `get_components`. 35 testsets, all passing; test_events.jl is 170 lines shorter with more coverage. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NjXEBDreXLvbXw3pLKtAVR
POM builds the event parameters but has nowhere to put values between solves; PSI owns the state arrays, the clock and the RNG. Today PSI also owns the *meaning* of an outage, fused into SimulationState indexing: what a contingency type says about occurrence and duration, how a countdown decays, how availability and balance offsets follow from it. That is domain logic, and it belongs here. `src/event_models/event_runtime.jl` is that half, as pure functions of values with no state type in sight: - `outage_occurred` / `time_to_recover` — the only per-contingency-type behavior, deterministic from a profile for FixedForcedOutage, a Bernoulli draw on the caller's rng for GeometricDistributionForcedOutage; - `advance_countdown`, `countdown_trajectory`, `availability_from_countdown`, `availability_trajectory`, `outage_power_offset`, `countdown_steps` — the arithmetic that carries an outage across steps and projects it into the next decision model's horizon; - `event_step_values` — one call per device per step, returning everything a runtime needs to write; - `event_parameter_keys` — the parameters to update and the order to write them, restricted to what the built model actually has, so a runtime stops hardcoding POM's parameter types; - `required_inputs` / `is_triggered` — a condition declares the values it needs, the runtime resolves them, and the condition is evaluated as a function of its own inputs. Only `DiscreteEventCondition` asks for the runtime's state object, and it asks explicitly. Divergences from PSI are marked `TODO(events)` at their definitions: the offset window (whole outage here, first step only in PSI) and rounding a duration that does not divide the state resolution. `mean_time_to_recovery` units are the caller's to state, since PowerSystems documents minutes and PSI reads hours. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NjXEBDreXLvbXw3pLKtAVR
A runtime needs the write order, not just the keys: `event_parameter_keys` covers what a built model has, but PSI also writes into simulation state datasets the container knows nothing about, and it needs the same ordering there. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NjXEBDreXLvbXw3pLKtAVR
`event_step_values` decayed the countdown and sampled a new outage in one call, so a runtime that wanted the decay had to call it every step — which also re-sampled — while one that only called it when the condition held froze any outage in progress. Wiring PSI to it hit exactly that: with a ContinuousCondition the countdown decayed twice per step and a two-step outage lasted one. `may_start` splits the two. The countdown decays on every call; only starting a new outage is gated. A runtime now makes one call per device per step and passes the condition's verdict, which is what the condition was always supposed to mean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NjXEBDreXLvbXw3pLKtAVR
PNM #356 and #357 landed on psy6 and the pin was already reverted to rev = "psy6"; only the comment describing the temporary pin remained.
Replaces the two runtime isa checks in template_validation. The event time-series validation drops its try/catch for PSY.has_time_series, which also stops a non-missing ArgumentError being reported as missing data. The security-constrained outage claim gates on a dispatched _needs_planned_outage_optin method.
- event_runtime: replace the Union{String,Nothing} sentinel with a
predicate plus a total accessor, drop five ternaries, use iszero.
The predicate tests the mapped value, not just key presence: the
default mappings register every key a contingency type accepts.
- event_constraints: merge the byte-identical ThermalGen and HydroGen
methods onto a two-member Union bound.
- event_model: extract the filter-and-error preamble repeated across
17 construct sites into _for_each_event_devices.
- event_arguments: replace the _EventLoadFormulations Union alias with
a supports_event_offset trait, which records that PowerLoadShift is
excluded rather than leaving it implicit.
- tests: pass moi_tests' 8th argument, which a misplaced paren had been
discarding so the quadratic-constraint assertion never ran; drop three
count-only testsets a coefficient-level testset supersedes, keeping
their size assertions.
- Resolve event devices from the discovery map template validation already filled, instead of probing every device's supplemental attributes twice per build; the mock now fills that map too. - Implement IOM.share_template_references! for the events instead of overriding IOM's private _deepcopy_template through invoke. - Dispatch the active-power outage lhs on the device family and route all six construct sites through one helper. - Share VariableTarget between StateVariableValueCondition and StateValueInput; drop the field-shuttling converter. - Narrow EventKey's component bound to PSY.Component. - Compute each countdown step once for both trajectories.
…etFeedforward Adds the EnergyTargetFeedforward struct, accessors and export; the storage argument method (EnergyTargetParameter plus the StorageEnergyShortageVariable slack) and constraint method, which validates that target_period is the horizon end because the storage slack is a one-step container; and a _add_energy_target_constraints! helper shared with the hydro reservoir target. The attach-time source-conflict guard now covers both target feedforwards. Re-enables the two storage feedforward testsets, adds coefficient-level, mid-horizon-rejection and attribute-collision tests, documents the feedforward in the formulation table and guide, and records pending work in .claude/pending-work.md.
Port outage events, and add the runtime interface PSI calls
IOM #164 (share_template_references!) is merged, so the [sources] pins go back to rev = "main" in the root and test projects. The docs project follows the upstream PowerTimeSeriesOpenAPIModels -> InfrastructureTimeSeriesOpenAPIModels rename, which is what broke the Documentation CI job. feedforward_interface.jl's header still claimed the event infrastructure lived in PowerSimulations and had not been ported; it describes the real fallbacks now. get_empty_timeseries_mapping gains a typed error for a non-event PSY.Contingency instead of a bare MethodError, and outage_power_offset's whole-duration semantics are documented as the live behavior now that PSI #1664 delegates both offset writes here.
IS4 absorbed lk/loss-curve-units-v2, so get_loss, get_loss_function and get_converter_loss_from/_to now return a LossCurve wrapper that declares its own unit system. get_proportional_term and get_constant_term are defined on the inner curve types only, so every reader had to unwrap; twelve sites needed it, including the two HVDCVSCConverterPowerConstraint methods. Curve-shape branching moves from isa checks to dispatch on the unwrapped ValueCurve: _hvdc_linear_loss_terms and _hvdc_pwl_len_segments replace the "only accepts LinearCurve" guards and the "Should not be here" fallback, each with a method that errors and names the offending type. _get_quadratic_term's untyped `= 0.0` fallback was a silent-failure hazard: a LossCurve-wrapped QuadraticCurve hit it and returned zero, dropping the a*I^2 term and disarming the guard that exists to refuse quadratic curves under LinearLossConverter. It now dispatches per curve type and errors on anything else. _loss_curve_value converts a curve to system base from whatever base it declares, replacing the "assume the curve is authored in device base" base_factor. This reproduces the old arithmetic exactly for a DeviceBaseUnit curve and corrects it for the NaturalUnit curves the data actually carries, which the previous code scaled as if they were per-unit. The unit-system dispatch is resolved once per device, above the time loop, so the abstract field type PSY declares keeps its cost at build time and the per-step arithmetic stays plain Float64.
…e-independent EnergyLimitFeedforward is the storage twin of ReservoirLimitFeedforward: same FeedforwardIntegralLimitConstraint, same per-block trench math, differing only in the parameter it reads. EnergyLimitParameter already existed, so the port is the struct plus a shared _add_integral_limit_constraints! that both types dispatch into, mirroring the _add_energy_target_constraints! split. The two testsets fenced in test_storage_device_models are live again; they had been written against BookKeeping and BatteryAncillaryServices, which POM does not have, so they run against StorageDispatchWithReserves over both battery fixtures. The source-conflict guard dispatched on a four-member Union of concrete feedforward types, which EnergyLimitFeedforward would have grown to five. It is a trait now, so each participating type opts in with one method. The property cuts across the hierarchy rather than along it -- WaterLevelBudgetFeedforward does not opt in -- so a shared supertype would have been the wrong tool. The mixed-type fallback is kept: two feedforwards of different types read different containers and cannot collide. meta strings interpolated a DataType directly, which renders module-qualified whenever the name is not visible in the active module. The test runner executes each file in a fresh module, so these keys were latently unstable; nameof makes them independent of the caller's module. The slack lookup in the bound- feedforward constraint had to move in lockstep with its registration, and the test expectations that build the same keys had to follow, or the read side asks for a qualified key the write side no longer stores. Also from the port reviews: the one-step storage shortage slack accepts a Vector of devices rather than only a FlattenIteratorWrapper, so a Vector caller cannot fall through to the unbounded full-horizon generic; and the hydro coefficient testset asserts its constraint set through a dispatched predicate instead of isa.
The formulation library tabulated five of POM's feedforward types and the explanation page gave guidance for four. The reservoir target and limit, water level budget, hydro usage limit and new energy limit feedforwards now have rows carrying their parameter, constraint container and exact constraint expression, and a guidance bullet each. Two behaviours are worth stating because neither is visible from the table: ReservoirLimitFeedforward dispatches on any PSY.Component despite its name, and HydroUsageLimitFeedforward reads ActivePowerVariable directly and nets served regulation reserves when the device model carries a service model.
Two breaking PowerSystems changes, no shims for either. #1791 renamed the per-unit marker `DU`/`DeviceBaseUnit` to `CU`/`ComponentBaseUnit`. In src that is a dispatch signature, not just a name: `_loss_curve_ratio_to_system_base(::PSY.DeviceBaseUnit, ...)` would stop being called and fall through to the error method, silently losing the per-unit branch of the loss-curve rescale. #1790 dropped the `MW`/`kV`/`OHMS`/`SIEMENS` constants and stopped exporting `MVA`/`MVAr`, leaving Unitful's `u"..."` macro as the only spelling. `PSY.MW` no longer resolves; `test/includes.jl` already does `using PowerSystems`, which re-exports `@u_str`, so `u"MW"` needs no new import. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LP6eB1zx4eyE3hd7tpvSue
Follow the ComponentBaseUnit rename and the u"..." natural units
remove dead transformer refactor code
The OpenAPI model packages pinned `jd/openapi_deps_update`, which predates the 0.1.0 release, its precompilation fix and the Julia 1.13 work; point them at main. InfrastructureSystems moves from the jd/openapi-1x-suite branch to IS4, where it merged. Restore the CI Julia matrices to '1' now that the 1.13 fix has landed.
…ilds again PowerSystems #1783 replaced must_run with commitment_mode and the Bool status with OperationalStates, and left no shim. The feedforwards' _is_must_run trait moves to utils/psy_utils.jl and reads commitment_mode == MUST_RUN for ThermalGen and HydroPumpTurbine (false for anything else, so SELF_SCHEDULED and RELIABILITY stay committable); is_online reads status as ONLINE or STARTUP. The 23 must-run sites and the eleven Bool status reads go through them, as do the test utilities' duration checks, and IOM.get_must_run forwards to the trait for IOM's own start-up and ramp code. SwitchedAdmittance lost its Y field: the shunt model spans the blocks and takes solved_admittance, else the engaged blocks, as the fixed susceptance. FixValueParameter metas were string(::Type), which carries the module prefix in the test workers now that the variable types live in IOM; the four sites use nameof like the other feedforward metas. Tests construct with the enums and the current shunt fields, wrap their HVDC and converter losses in LossCurve on the component base the readers documented, and the FACTS shunt test asks for BYP instead of a nothing mode the OpenAPI 0.1.0 schema cannot encode. The test environment pins InfrastructureOptimizationModels to rh/cost_coefficient_ratio (IOM #166) until it merges: IOM main still calls the convert_cost_coefficient signature InfrastructureSystems removed.
Follow PowerSystems #1783 and the OpenAPI 1.x structs so the suite builds again
rodrigomha
force-pushed
the
rh/market_models_1x
branch
from
September 14, 2026 21:29
0a1110f to
2a87dd6
Compare
… the OpenAPI 1.x line Brings main and the #1783 repair (#301) to the market-models branch and moves every environment to the 1.x line: InfrastructureSystems IS4, PowerSystems jd/openapi-1x (with #1795), the PowerOpenAPIModels subpackages on main, PowerSystemCaseBuilder, PowerFlowFileParser and PowerTableDataParser on jd/openapi-1x. Two pins stay temporary: IOM rh/market_models_1x (jd/market_models2 plus main, IOM #167) until that merge lands, and PowerFlows jd/openapi-1x-pins (#449) for the LossCurve reader. Conflicts: the template gains both new fields (market_model, events) and both validation blocks (the market network check, the outage-event discovery); the IOM.get_must_run forwarder in market_bid_plumbing.jl goes, since the repair implements it on the _is_must_run trait in psy_utils.jl; the pin files take the 1.x side.
rodrigomha
force-pushed
the
rh/market_models_1x
branch
from
September 14, 2026 21:34
2a87dd6 to
a9daa6a
Compare
main is merged there now, so the integration branch this pointed at is gone.
jd-lara
self-requested a review
September 14, 2026 21:52
jd-lara
approved these changes
Sep 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Moves
jd/market_models2onto the OpenAPI 1.x line by mergingjd/openapi-1x-pins, which carriesmainand the #1783 repair (#301). One merge commit.Environments
Every project pins the 1.x line: InfrastructureSystems
IS4, PowerSystemsjd/openapi-1x(with #1795), the PowerOpenAPIModels subpackages onmain, PowerSystemCaseBuilder, PowerFlowFileParser and PowerTableDataParser onjd/openapi-1x. The interim pins to the #1784 merge commit, the pre-1.x InfrastructureSystems commit andrh/update_enumsare gone. InfrastructureOptimizationModels stays onjd/market_models2, which now carriesmain(merged there directly, so it has #164, #165 and #166). One pin is temporary and says so in a comment: PowerFlowsjd/openapi-1x-pins(#449), for theLossCurvereader; back topsy6once it merges.Conflicts and how they were resolved
PowerOperationsProblemTemplategains both new fields,market_modelandevents, in that order, and the constructor fills both.template_validation.jlkeeps both appended blocks: the market network-formulation check and the outage-event discovery.market_bid_plumbing.jldrops theIOM.get_must_runforwarder; the repair implements it on the_is_must_runtrait inutils/psy_utils.jl, and thePointToPointBidoperation-cost seam stays.Results
Full suite on the pinned stack, Julia 1.12: 130,648 tests pass across all 56 files, the market-model tests included.
🤖 Generated with Claude Code
https://claude.ai/code/session_01XgUkJVy9F23R3G5F8XFaLj