Skip to content

Place GOPACS redispatch orders through Distro Energy - #135

Merged
Miggets7 merged 51 commits into
mainfrom
feature/distro-gopacs-orders
Nov 13, 2026
Merged

Miggets7 merged 51 commits into
mainfrom
feature/distro-gopacs-orders

Conversation

@Miggets7

@Miggets7 Miggets7 commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Description

Closes #133. Part of openremote/denhaag#209, which stays open until the orders are tested on production.

Adds DistroGopacsHandler, which places GOPACS redispatch orders through Distro Energy's GOPACS Connector API. It is switched on per EmsDistroEnergyAsset with gopacsEnabled and places orders under the asset's portfolio, the same one the day-ahead submissions use. EmsOptimisationService copies the redispatch announcement from the EmsGOPACSAsset to every EmsDistroEnergyAsset under the same parent, so a deployment needs no attribute links and the GOPACS and Distro handlers stay independent.

The handler pre-fills order type and quantity from the announcement and the batteries. The operator sets a price and confirms, and nothing is ordered automatically. After that it polls the order status, replaces open orders when the price changes, and cancels them when the announcement closes, on a manual cancel, or when gopacsEnabled is turned off or the asset is deleted. Setup is described in ems/docs/DistroEnergyIntegration.md.

JAX-RS proxy and wire records for creating, reading, replacing and
cancelling GOPACS orders and listing their settlements. Status and
order type stay strings so new values in the draft API do not break
deserialisation.
Announcement inputs are read-only and meant to be filled through
attribute links from an EmsGOPACSAsset, so the Distro integration does
not depend on the GOPACS handlers. Order type and quantity are
pre-filled by the handler and can be overridden; price, confirm and
cancel are set by the operator. The existing day-ahead panels are
renamed so both groups read clearly in the asset view.
Pre-fills order type and quantity from the linked announcement: the
EAN effectivity category picks the side, and the batteries under the
parent optimisation asset size a constant power that their state of
charge can hold for the whole window. On operator confirmation the
order is posted to Distro; a price change replaces open orders, a
closed announcement or the cancel flag cancels them, and a poll tracks
status until a final one, recording the trades of settled orders.

Tokens come from OpenRemote's OAuth filter, which caches and refreshes
the client-credentials token.
One handler per EmsDistroEnergyAsset while gopacsEnabled is true,
reconciled from both the persistence route and attribute events under
a lock, independent of the day-ahead handler. Other gopacs* attribute
events are forwarded to the running handler.
Unit spec for the effectivity-to-side mapping, battery sizing,
expiration and the attribute-driven create/replace/cancel/poll flows;
WireMock spec for the OAuth token call, bearer reuse and wire format;
service spec for the gopacsEnabled reconcile.
Configuration, the attribute links from EmsGOPACSAsset, order sizing,
the order lifecycle and the new tests. Also replace the made-up
effectivity example in the GOPACS docs with a real category.
Track an order as PENDING before posting it, so a create that times
out after Distro accepted it is still polled, cancelled and settled.
Only a 4xx untracks it again.

Store a polled order before fetching its settlements, so a failed fetch
is retried next poll without repeating the STATUS entry. Read every
settlements page. Skip the suggestion when the batteries cannot hold
the 100 kW minimum. Drop queued work once the handler is undeployed and
close the client only after a running order action has finished.
Day-ahead stays on the left, so the two Distro features no longer
interleave on the asset page.
Operators work in these two panels, so they sit next to the day-ahead
panels while configuration, status and history stay on the right.
Turning gopacsEnabled off or deleting the asset stopped polling but left
PENDING and SUBMITTED orders live at Distro, where they could still be
matched with nothing tracking them. Both now cancel the open orders
before closing the client. A manager stop still leaves them in place,
since the next start restores and keeps tracking them.
The updated Distro spec takes at most one decimal for the quantity in MW
and two for the price, requires a price of 0 or more, needs the order
to expire at least 15 minutes before start, and only lets SUBMITTED
orders be replaced.

Round an operator-entered quantity down to 100 kW steps and the price
to cents. Refuse a negative price, on confirm and on a price change,
and refuse a window that starts within 15 minutes. Replace only the
SUBMITTED orders of the announcement.
GOPACS no longer takes bids for an announcement once its bid validity
end has passed. Such an order went out with Distro's default expiry and
stayed live without any chance to help. Refuse it when the validity end
is known.
The updated Distro spec says the settlements endpoint is not implemented
yet. A settled order whose trade fetch failed stayed tracked for a
retry, so it would have been polled every five minutes forever. Record
the failure as SETTLEMENTS_FAILED and drop the order instead.
The handler built a pending GopacsOrder from a request and a replace
request from an order by hand. GopacsOrder.pending(request) and
toRequest(price) now own those copies.
Routing every gopacs* attribute also sent the handler's own writes
(orders, history, last sync, the pre-filled type and quantity) back to
it, and each queued a task that did nothing. Route on the explicit
DistroGopacsHandler.INPUTS set instead.
Build the three updateSuggestion log messages with String.format
instead of long string concatenations.
An undeploy less than a second after a confirm or cancel skipped the
delayed reset, leaving the flag true. Setting it to true again after
re-enabling was then an unchanged value and was ignored. Deploy now
resets either flag that is still true.
Log the handler config at deploy (without the secret), every changed
input at FINE, and confirms, cancels, placed and replaced orders and
status changes at INFO. Warn when a flag is ignored because it is
already true or because no handler is running for the asset.
A bare HTTP 403 does not say whether the tenant, the IP whitelist or the
URL is wrong. Log Distro's error body, cut to 500 characters, and record
it in the order history. HTTP errors are logged without the stack trace,
which adds nothing for them.
gopacsOrderType was free text, so an operator could type anything and
only learn at confirm time that it was refused. An enum value type
renders as a dropdown in the manager UI. The handler now works with the
enum and only turns it into a string for the Distro request.
placeOrder built the refusal reason through a long else-if chain. The
checks now live in a static validateInput that returns early, so
placeOrder only reads the attributes, refuses or sends.

The quarter-hour alignment check is gone: the window is linked from the
GOPACS announcement and is read-only, so it is always aligned. The
separate rounding log is folded into the Placing log, which now also
shows the quantity the operator entered.
gopacsOrderPrice is now a POSITIVE_NUMBER (min 0). OpenRemote validates
every attribute event against the value constraints, from the UI as
well as from rules and links, so a negative price never reaches the
handler. The handler's own negative-price checks on create and replace
are removed.
CI runs spotlessCheck, which failed on both test files. Formatting only.
Distro confirmed the settlement endpoints go live shortly and that the
trades are available as soon as an order is SETTLED. The Javadoc and
docs no longer say the endpoints are missing. Behaviour is unchanged:
one fetch per settled order, no retry.
The Distro GOPACS order handler reads the announcement from six
read-only gopacs* inputs, but nothing in the extension filled them:
each deployment had to add attribute links from the EmsGOPACSAsset by
hand and copy the current values once.

EmsOptimisationService now copies contractedEAN and the redispatch
announcement attributes to every EmsDistroEnergyAsset that shares the
GOPACS asset's parent. It does so on a change of one of those GOPACS
attributes, on a save of either asset, and at start. Only inputs that
differ are written, so the redispatch poller rewriting the same
announcement does not rewrite the Distro asset on every poll. Nothing
is copied while a parent has more than one GOPACS asset.
The Distro asset no longer needs attribute links from the GOPACS asset:
EmsOptimisationService copies the announcement to the Distro asset under
the same parent. The setup steps, attribute table and workflow diagram
now describe the copy.
@Miggets7 Miggets7 added the Feature Development of new feature label Oct 28, 2026
@Miggets7
Miggets7 requested review from igorrutka and wborn October 28, 2026 15:59
@Miggets7 Miggets7 self-assigned this Oct 28, 2026
Price-change reactions run on the shared executor, which does not keep
their order. An older reaction that ran last re-submitted its obsolete
price. replaceOrders now reads the price from the asset, like the
announcement path, and skips orders already at that price.
The announcement inputs are copied to the Distro asset whether the
order handler runs or not, and later copies skip unchanged values.
Enabling the handler after a copy left order type and quantity empty,
or holding a suggestion for an earlier announcement. deploy() now
redoes the suggestion when an announcement input was written after
type and quantity, which keeps values the operator entered since.
A manual cancel that failed was dropped, and polls only retried
cancels of orders for another announcement. The operator cancel now
remembers the orders it was asked to cancel, in memory, and the poll
retries a failed one through the same path as the announcement case.
A restart forgets the request; the CANCEL_FAILED history entry
remains.
Copying to the Distro asset checked for exactly one GOPACS sibling,
but copying from a GOPACS asset did not. With two GOPACS assets under
one parent, their updates could overwrite each other's announcement on
the Distro asset. Copying from the GOPACS side now goes through
syncDistroGopacsInputsTo, so both directions share the check.
Describes the per-battery capacity limit, the stripped ean. prefix,
the pre-fill on start, replaces at the asset's current price, the
cancel and settlement retries, trades of partly matched orders and
one settlement data point per order. Drops the MATCHED status, which
Distro no longer reports, and asks to remove old attribute links, as
their rewrites would make the handler redo the pre-filled order.
@Miggets7
Miggets7 requested a review from wborn November 11, 2026 13:01

@wborn wborn left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The previous findings are addressed, and no further blocking issues were identified. A few non-blocking edge cases were found that may be worth investigating: price changes while orders are pending, and stale announcement inputs after deleting or moving an asset.

All 14 changed files were reviewed, and all 120 existing targeted tests passed locally. Additional tests confirmed these edge cases using a mocked Distro API. Live Distro testing was not performed.

This review was AI-assisted.

@wborn
wborn dismissed stale reviews from themself November 11, 2026 13:53

Previously requested changes have been addressed; the remaining feedback is non-blocking.

A price change only replaced SUBMITTED orders, so an order still PENDING at
that moment, or one whose replace failed, kept the old price. Each poll now
also replaces the orders it refreshed that are not at the asset's price.
Orders the operator asked to cancel are skipped, which keeps the replace and
the cancel retry on disjoint orders.

@wborn wborn left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The order lifecycle handles the previously reported scenarios, but an uncaught storage exception can permanently stop order polling. That should be addressed before merge. Persisting order references before submission is a separate, non-blocking resilience improvement.

All 14 changed files were reviewed. The 122 existing targeted tests passed locally. An additional test using OpenRemote’s actual ContainerScheduledExecutor confirmed that a storage exception terminates recurring polling. A separate test verified that no order-reference save is sent before the create request. Live Distro testing and crash/restart testing were not performed.

This review was AI-assisted.

A periodic task that throws is never run again by the scheduled executor,
so a failed asset read during a database outage stopped every order
refresh, cancel and replace retry and settlement fetch until a redeploy.
The poll is now wrapped and logs the failure, as the redispatch poll does.
The order is tracked before it is posted, but saving it on the asset is
asynchronous, so a crash just after Distro accepted it left an order that
could still be matched with nothing tracking it. Orders now carry the
asset id in their metadata, and the first poll after deploy lists the
portfolio's PENDING, SUBMITTED and LOCKED orders and tracks those of this
asset it does not track yet. A failed listing is retried on the next poll.
@Miggets7
Miggets7 requested a review from wborn November 11, 2026 14:58

@wborn wborn left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The polling exception guard addresses the previous blocking finding, and the new reconciliation path recovers active orders missing from local storage. However, the upgrade path for existing Distro assets needs to be provided and documented before merge.

All 15 changed files were reviewed. The 126 existing targeted tests passed. Two additional upgrade tests confirmed that loading a pre-upgrade asset does not add the new GOPACS attributes and that announcement synchronization skips those missing attributes; newly created assets receive the announcement correctly.

A full database upgrade and live Distro testing were not performed. The password-protected developer portal prevented independently rechecking the external API specification.

This review was AI-assisted.

Comment thread ems/docs/DistroEnergyIntegration.md Resolved
@Miggets7
Miggets7 requested a review from wborn November 11, 2026 15:35
Distro assets created before GOPACS orders have none of the gopacs*
attributes, and loading them does not add the new descriptors. A Flyway
migration now adds every missing one with its type and meta, leaves
attributes an asset already has alone, and adds gopacsEnabled empty so
ordering stays off until an operator enables it. The guide explains the
upgrade, including how to refresh the asset viewer config.

@wborn wborn left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How is Distro portfolio access isolated between OpenRemote realms?

The OAuth credentials are configured globally, while the portfolio is configurable per EmsDistroEnergyAsset and passed directly to the Distro API.

If the same OAuth client has access to multiple portfolios, could a user in one realm configure an asset to submit orders under another realm's portfolio?

Does Distro enforce portfolio-level authorization, or should OpenRemote validate the realm-to-portfolio association or use realm-specific credentials?

The existing day-ahead integration appears to have a similar assumption, but it would be good to clarify whether this setup is safe for multi-realm deployments.

The Distro credentials are environment variables shared by every
realm. Distro scopes each OAuth client to its portfolio, so realms
cannot reach another customer's portfolio, but all of them act under
the shared one. Per-realm credentials are tracked in #142.
@Miggets7

Copy link
Copy Markdown
Contributor Author

Distro assigns a portfolio to each OAuth client and enforces that on its side. Setting another customer's portfolio on an EmsDistroEnergyAsset fails at Distro, because the configured client has no access to it. The limitation is that the manager has one Distro client, set through DISTRO_GOPACS_CLIENT_ID/DISTRO_GOPACS_CLIENT_SECRET. Day-ahead works the same way through DISTRO_ENERGY_CLIENT_KEY. So every realm acts under the same portfolio.

03cdd3b documents that limit in the Distro guide. A manager serves a single Distro customer, and every realm that enables the integration acts on that customer's behalf. Multi-realm deployments that serve separate customers should wait for #142.

#142 moves the credentials onto the asset. That follows the pattern OpenRemote already uses for Agent.OAUTH_GRANT and GatewayAsset.CLIENT_SECRET. Each realm then brings its own Distro client, and Distro's per-client portfolio check keeps the realms apart. The trade-off is that MetaItemType.SECRET only hides the value in the UI, so any user in the same realm with read access can still fetch it through the API. The issue covers GOPACS and day-ahead and keeps the environment variables as a fallback for single-customer managers.

@Miggets7
Miggets7 requested a review from wborn November 13, 2026 13:56

@wborn wborn left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM now!

@Miggets7
Miggets7 added this pull request to the merge queue Nov 13, 2026
Merged via the queue into main with commit d997b51 Nov 13, 2026
2 checks passed
@wborn
wborn deleted the feature/distro-gopacs-orders branch November 13, 2026 14:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Feature Development of new feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

EMS: place GOPACS redispatch orders through Distro Energy

2 participants