Skip to content

ShortfallEvents and LOLEv Implementation - #115

Open
akrivi wants to merge 13 commits into
mainfrom
al/shortfallEvents
Open

akrivi wants to merge 13 commits into
mainfrom
al/shortfallEvents

Conversation

@akrivi

@akrivi akrivi commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator

This PR adds a new ShortfallEvents result specification that records shortfall events directly during simulation.

We define an "adequacy event" as a set of event-periods that are contiguous at the highest available temporal resolution [1].
Events are recorded separately for each Monte Carlo sample at both system and regional levels.

ShortfallResult does not retain sufficient information to reconstruct individual events.
 At the same time, obtaining this information from ShortfallSamplesResult requires storing and postprocessing the full regions × timestamps × samples shortfall tensor.


ShortfallEvents instead stores event boundaries and accumulated unserved energy, avoiding that dense tensor, which provides a more compact representation for event analysis, particularly when shortfalls are sparse.

Screenshot 2026-09-06 at 14 29 20

The implementation adds the following metrics at both system and regional levels:

  1. LOLEv: the expected number of shortfall events per simulation horizon, estimated by averaging event counts across Monte Carlo samples [2].
  2. MeanEventDuration: mean duration across all recorded events.
  3. MaxEventDuration: maximum duration among all recorded events.
  4. MeanEventEnergy: mean unserved energy across all recorded events.
  5. MaxEventEnergy: maximum unserved energy among all recorded events.

With the implementation of LOLEv, all metrics defined in [2] are now included in PRAS.

Additionally, this PR extends PRASFiles to support JSON export of ShortfallEventsResult. This includes both aggregate event-level metrics and optionally raw event records at the system and regional levels.

Benchmarks

1) RTS-GMLC with 10× regional load: 3 regions, 8,784 hourly timesteps (2,000 MC samples, 6 threads)

The run recorded 2,000 system events and 6,000 regional events across all samples.

Result specification assess time (s) Max RSS (GB) Result size (MB)
ShortfallSamples() 3.513 1.514 421.843
ShortfallEvents() 3.455 0.669 2.131

2) 13-region system: 8,761 hourly timesteps (2,000 MC samples, 6 threads)

The run recorded 362 system events and 1,839 regional events across all samples.

Result specification assess time (s) Max RSS (GB) Result size (MB)
ShortfallSamples() 12.929 4.084 1,823.200
ShortfallEvents() 10.090 0.685 2.122

Note on Result size: ShortfallSamples stores every region-timestep-sample value, whereas ShortfallEvents stores event records plus per-sample bookkeeping. Both results also reference regional metadata.

Event storage is particularly compact when shortfalls are infrequent or persist across consecutive timesteps, while many separate short events can reduce or eliminate the size advantage.

[1] Stephen et al. (2022), "Clarifying the Interpretation and Use of the LOLE Resource Adequacy Metric"
[2] NERC (2018), "Probabilistic Adequacy and Measures Technical Reference Report"

@codecov-commenter

codecov-commenter commented Sep 9, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 89.41606% with 29 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.01%. Comparing base (0deb435) to head (ce21b06).

Files with missing lines Patch % Lines
PRASCore.jl/src/Results/ShortfallEvents.jl 87.96% 13 Missing ⚠️
PRASCore.jl/src/Results/metrics.jl 67.56% 12 Missing ⚠️
PRASCore.jl/src/Simulations/recording.jl 96.22% 2 Missing ⚠️
PRASFiles.jl/src/Results/write.jl 90.47% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #115      +/-   ##
==========================================
+ Coverage   84.57%   85.01%   +0.43%     
==========================================
  Files          45       46       +1     
  Lines        2658     2929     +271     
==========================================
+ Hits         2248     2490     +242     
- Misses        410      439      +29     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Threaded assembly and non-hourly event-energy conversion lack direct coverage, and the event docstring misstates the raw energy field.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds event-level shortfall recording, associated adequacy metrics, and JSON export support.

Changes:

  • Introduces ShortfallEventsResult and event recording.
  • Adds LOLEv and duration/energy metrics.
  • Adds JSON exports, tests, examples, and documentation.
File summaries
File Description
PRASFiles.jl/test/runtests.jl Tests shortfall and event exports.
PRASFiles.jl/src/Results/write.jl Implements event-result JSON export.
PRASFiles.jl/src/Results/utils.jl Defines exported event structures and conversions.
PRASFiles.jl/src/PRASFiles.jl Imports event APIs and exports saveevents.
PRASCore.jl/test/Simulations/runtests.jl Tests event recording and metrics.
PRASCore.jl/test/Results/shortfall.jl Tests event interval selection.
PRASCore.jl/src/Simulations/utils.jl Enables regional event shortfall lookup.
PRASCore.jl/src/Simulations/recording.jl Records system and regional events.
PRASCore.jl/src/Results/ShortfallEvents.jl Implements event results, accessors, and metrics.
PRASCore.jl/src/Results/Results.jl Registers and exports event functionality.
PRASCore.jl/src/Results/metrics.jl Defines event metric types.
PRAS.jl/examples/pras_adequacy_metrics.jl Demonstrates LOLEv and event metrics.
docs/src/resourceadequacy.md Documents LOLEv.
docs/src/PRASFiles/index.md Documents event JSON exports.
docs/src/PRASCore/api.md Adds event APIs to the reference.
docs/src/PRAS/results.md Documents event semantics and usage.
docs/src/images/shortfall-events.svg Illustrates event aggregation.
docs/make.jl Adds PRASFiles documentation navigation.
Review details
  • Files reviewed: 17/18 changed files
  • Comments generated: 3
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

"""
struct ShortfallEvents <: ResultSpec end

usesamplepartitions(::ShortfallEvents) = true
Comment on lines +277 to +280
function MeanEventEnergy(x::ShortfallEventsResult{N,L,T,P,E}) where {N,L,T,P,E}
p2e = conversionfactor(L, T, P, E)
energies = [
p2e * event_energy(ev)
with positive shortfall. A `ShortfallEventsResult` can be indexed by sample
number to retrieve system-wide events or by region name and sample number to
retrieve regional events. Each event records its starting timestep, ending
timestep and unserved energy.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants