Repository navigation
Place GOPACS redispatch orders through Distro Energy - #135
Conversation
JAX-RS proxy and wire records for creating, reading, replacing and cancelling GOPACS orders and listing their settlements. Status and order type stay strings so new values in the draft API do not break deserialisation.
Announcement inputs are read-only and meant to be filled through attribute links from an EmsGOPACSAsset, so the Distro integration does not depend on the GOPACS handlers. Order type and quantity are pre-filled by the handler and can be overridden; price, confirm and cancel are set by the operator. The existing day-ahead panels are renamed so both groups read clearly in the asset view.
Pre-fills order type and quantity from the linked announcement: the EAN effectivity category picks the side, and the batteries under the parent optimisation asset size a constant power that their state of charge can hold for the whole window. On operator confirmation the order is posted to Distro; a price change replaces open orders, a closed announcement or the cancel flag cancels them, and a poll tracks status until a final one, recording the trades of settled orders. Tokens come from OpenRemote's OAuth filter, which caches and refreshes the client-credentials token.
One handler per EmsDistroEnergyAsset while gopacsEnabled is true, reconciled from both the persistence route and attribute events under a lock, independent of the day-ahead handler. Other gopacs* attribute events are forwarded to the running handler.
Unit spec for the effectivity-to-side mapping, battery sizing, expiration and the attribute-driven create/replace/cancel/poll flows; WireMock spec for the OAuth token call, bearer reuse and wire format; service spec for the gopacsEnabled reconcile.
Configuration, the attribute links from EmsGOPACSAsset, order sizing, the order lifecycle and the new tests. Also replace the made-up effectivity example in the GOPACS docs with a real category.
Track an order as PENDING before posting it, so a create that times out after Distro accepted it is still polled, cancelled and settled. Only a 4xx untracks it again. Store a polled order before fetching its settlements, so a failed fetch is retried next poll without repeating the STATUS entry. Read every settlements page. Skip the suggestion when the batteries cannot hold the 100 kW minimum. Drop queued work once the handler is undeployed and close the client only after a running order action has finished.
Day-ahead stays on the left, so the two Distro features no longer interleave on the asset page.
Operators work in these two panels, so they sit next to the day-ahead panels while configuration, status and history stay on the right.
Turning gopacsEnabled off or deleting the asset stopped polling but left PENDING and SUBMITTED orders live at Distro, where they could still be matched with nothing tracking them. Both now cancel the open orders before closing the client. A manager stop still leaves them in place, since the next start restores and keeps tracking them.
The updated Distro spec takes at most one decimal for the quantity in MW and two for the price, requires a price of 0 or more, needs the order to expire at least 15 minutes before start, and only lets SUBMITTED orders be replaced. Round an operator-entered quantity down to 100 kW steps and the price to cents. Refuse a negative price, on confirm and on a price change, and refuse a window that starts within 15 minutes. Replace only the SUBMITTED orders of the announcement.
GOPACS no longer takes bids for an announcement once its bid validity end has passed. Such an order went out with Distro's default expiry and stayed live without any chance to help. Refuse it when the validity end is known.
The updated Distro spec says the settlements endpoint is not implemented yet. A settled order whose trade fetch failed stayed tracked for a retry, so it would have been polled every five minutes forever. Record the failure as SETTLEMENTS_FAILED and drop the order instead.
The handler built a pending GopacsOrder from a request and a replace request from an order by hand. GopacsOrder.pending(request) and toRequest(price) now own those copies.
Routing every gopacs* attribute also sent the handler's own writes (orders, history, last sync, the pre-filled type and quantity) back to it, and each queued a task that did nothing. Route on the explicit DistroGopacsHandler.INPUTS set instead.
Build the three updateSuggestion log messages with String.format instead of long string concatenations.
An undeploy less than a second after a confirm or cancel skipped the delayed reset, leaving the flag true. Setting it to true again after re-enabling was then an unchanged value and was ignored. Deploy now resets either flag that is still true.
Log the handler config at deploy (without the secret), every changed input at FINE, and confirms, cancels, placed and replaced orders and status changes at INFO. Warn when a flag is ignored because it is already true or because no handler is running for the asset.
A bare HTTP 403 does not say whether the tenant, the IP whitelist or the URL is wrong. Log Distro's error body, cut to 500 characters, and record it in the order history. HTTP errors are logged without the stack trace, which adds nothing for them.
gopacsOrderType was free text, so an operator could type anything and only learn at confirm time that it was refused. An enum value type renders as a dropdown in the manager UI. The handler now works with the enum and only turns it into a string for the Distro request.
placeOrder built the refusal reason through a long else-if chain. The checks now live in a static validateInput that returns early, so placeOrder only reads the attributes, refuses or sends. The quarter-hour alignment check is gone: the window is linked from the GOPACS announcement and is read-only, so it is always aligned. The separate rounding log is folded into the Placing log, which now also shows the quantity the operator entered.
gopacsOrderPrice is now a POSITIVE_NUMBER (min 0). OpenRemote validates every attribute event against the value constraints, from the UI as well as from rules and links, so a negative price never reaches the handler. The handler's own negative-price checks on create and replace are removed.
CI runs spotlessCheck, which failed on both test files. Formatting only.
Distro confirmed the settlement endpoints go live shortly and that the trades are available as soon as an order is SETTLED. The Javadoc and docs no longer say the endpoints are missing. Behaviour is unchanged: one fetch per settled order, no retry.
The Distro GOPACS order handler reads the announcement from six read-only gopacs* inputs, but nothing in the extension filled them: each deployment had to add attribute links from the EmsGOPACSAsset by hand and copy the current values once. EmsOptimisationService now copies contractedEAN and the redispatch announcement attributes to every EmsDistroEnergyAsset that shares the GOPACS asset's parent. It does so on a change of one of those GOPACS attributes, on a save of either asset, and at start. Only inputs that differ are written, so the redispatch poller rewriting the same announcement does not rewrite the Distro asset on every poll. Nothing is copied while a parent has more than one GOPACS asset.
The Distro asset no longer needs attribute links from the GOPACS asset: EmsOptimisationService copies the announcement to the Distro asset under the same parent. The setup steps, attribute table and workflow diagram now describe the copy.
Price-change reactions run on the shared executor, which does not keep their order. An older reaction that ran last re-submitted its obsolete price. replaceOrders now reads the price from the asset, like the announcement path, and skips orders already at that price.
The announcement inputs are copied to the Distro asset whether the order handler runs or not, and later copies skip unchanged values. Enabling the handler after a copy left order type and quantity empty, or holding a suggestion for an earlier announcement. deploy() now redoes the suggestion when an announcement input was written after type and quantity, which keeps values the operator entered since.
A manual cancel that failed was dropped, and polls only retried cancels of orders for another announcement. The operator cancel now remembers the orders it was asked to cancel, in memory, and the poll retries a failed one through the same path as the announcement case. A restart forgets the request; the CANCEL_FAILED history entry remains.
Copying to the Distro asset checked for exactly one GOPACS sibling, but copying from a GOPACS asset did not. With two GOPACS assets under one parent, their updates could overwrite each other's announcement on the Distro asset. Copying from the GOPACS side now goes through syncDistroGopacsInputsTo, so both directions share the check.
Describes the per-battery capacity limit, the stripped ean. prefix, the pre-fill on start, replaces at the asset's current price, the cancel and settlement retries, trades of partly matched orders and one settlement data point per order. Drops the MATCHED status, which Distro no longer reports, and asks to remove old attribute links, as their rewrites would make the handler redo the pre-filled order.
wborn
left a comment
There was a problem hiding this comment.
The previous findings are addressed, and no further blocking issues were identified. A few non-blocking edge cases were found that may be worth investigating: price changes while orders are pending, and stale announcement inputs after deleting or moving an asset.
All 14 changed files were reviewed, and all 120 existing targeted tests passed locally. Additional tests confirmed these edge cases using a mocked Distro API. Live Distro testing was not performed.
This review was AI-assisted.
Previously requested changes have been addressed; the remaining feedback is non-blocking.
A price change only replaced SUBMITTED orders, so an order still PENDING at that moment, or one whose replace failed, kept the old price. Each poll now also replaces the orders it refreshed that are not at the asset's price. Orders the operator asked to cancel are skipped, which keeps the replace and the cancel retry on disjoint orders.
wborn
left a comment
There was a problem hiding this comment.
The order lifecycle handles the previously reported scenarios, but an uncaught storage exception can permanently stop order polling. That should be addressed before merge. Persisting order references before submission is a separate, non-blocking resilience improvement.
All 14 changed files were reviewed. The 122 existing targeted tests passed locally. An additional test using OpenRemote’s actual ContainerScheduledExecutor confirmed that a storage exception terminates recurring polling. A separate test verified that no order-reference save is sent before the create request. Live Distro testing and crash/restart testing were not performed.
This review was AI-assisted.
A periodic task that throws is never run again by the scheduled executor, so a failed asset read during a database outage stopped every order refresh, cancel and replace retry and settlement fetch until a redeploy. The poll is now wrapped and logs the failure, as the redispatch poll does.
The order is tracked before it is posted, but saving it on the asset is asynchronous, so a crash just after Distro accepted it left an order that could still be matched with nothing tracking it. Orders now carry the asset id in their metadata, and the first poll after deploy lists the portfolio's PENDING, SUBMITTED and LOCKED orders and tracks those of this asset it does not track yet. A failed listing is retried on the next poll.
wborn
left a comment
There was a problem hiding this comment.
The polling exception guard addresses the previous blocking finding, and the new reconciliation path recovers active orders missing from local storage. However, the upgrade path for existing Distro assets needs to be provided and documented before merge.
All 15 changed files were reviewed. The 126 existing targeted tests passed. Two additional upgrade tests confirmed that loading a pre-upgrade asset does not add the new GOPACS attributes and that announcement synchronization skips those missing attributes; newly created assets receive the announcement correctly.
A full database upgrade and live Distro testing were not performed. The password-protected developer portal prevented independently rechecking the external API specification.
This review was AI-assisted.
Distro assets created before GOPACS orders have none of the gopacs* attributes, and loading them does not add the new descriptors. A Flyway migration now adds every missing one with its type and meta, leaves attributes an asset already has alone, and adds gopacsEnabled empty so ordering stays off until an operator enables it. The guide explains the upgrade, including how to refresh the asset viewer config.
wborn
left a comment
There was a problem hiding this comment.
How is Distro portfolio access isolated between OpenRemote realms?
The OAuth credentials are configured globally, while the portfolio is configurable per EmsDistroEnergyAsset and passed directly to the Distro API.
If the same OAuth client has access to multiple portfolios, could a user in one realm configure an asset to submit orders under another realm's portfolio?
Does Distro enforce portfolio-level authorization, or should OpenRemote validate the realm-to-portfolio association or use realm-specific credentials?
The existing day-ahead integration appears to have a similar assumption, but it would be good to clarify whether this setup is safe for multi-realm deployments.
The Distro credentials are environment variables shared by every realm. Distro scopes each OAuth client to its portfolio, so realms cannot reach another customer's portfolio, but all of them act under the shared one. Per-realm credentials are tracked in #142.
|
Distro assigns a portfolio to each OAuth client and enforces that on its side. Setting another customer's portfolio on an 03cdd3b documents that limit in the Distro guide. A manager serves a single Distro customer, and every realm that enables the integration acts on that customer's behalf. Multi-realm deployments that serve separate customers should wait for #142. #142 moves the credentials onto the asset. That follows the pattern OpenRemote already uses for |
Description
Closes #133. Part of openremote/denhaag#209, which stays open until the orders are tested on production.
Adds
DistroGopacsHandler, which places GOPACS redispatch orders through Distro Energy's GOPACS Connector API. It is switched on perEmsDistroEnergyAssetwithgopacsEnabledand places orders under the asset'sportfolio, the same one the day-ahead submissions use.EmsOptimisationServicecopies the redispatch announcement from theEmsGOPACSAssetto everyEmsDistroEnergyAssetunder the same parent, so a deployment needs no attribute links and the GOPACS and Distro handlers stay independent.The handler pre-fills order type and quantity from the announcement and the batteries. The operator sets a price and confirms, and nothing is ordered automatically. After that it polls the order status, replaces open orders when the price changes, and cancels them when the announcement closes, on a manual cancel, or when
gopacsEnabledis turned off or the asset is deleted. Setup is described inems/docs/DistroEnergyIntegration.md.