Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 48 additions & 0 deletions contracts/risk-core/scripts/ops/README-mark-keeper.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,3 +54,51 @@ cp numo-mark-callback.service numo-mark-signer.service numo-mark-keeper.service
systemctl daemon-reload
systemctl enable --now numo-mark-callback numo-mark-signer numo-mark-keeper
```

## RETIRED 2026-09-22 — the cNGN feed signer is not being refunded

`0xc9f1ffDE…20fdc`, the EOA that pushed prices into the cNGN spot feed
`0x41512C6a2af5AcD219EbCcfaF34f7088A2999ABC`, ran out of gas on **2026-09-11** (balance
0.0000017 ETH, nonce 65,466) and is deliberately **not** being refunded. The mark keeper is a
downstream casualty: it reads that feed, the read reverts `BLF_DataTooOld()`, and it fails closed
rather than marking to a frozen price — which is correct behaviour.

**Nothing that trades depends on either feed this signer wrote to.** Verified 2026-09-22:

| feed | why it is dead |
| --- | --- |
| `0x41512C6a` cNGN spot | spot moved to the static feeds at the SRM cutover, 2026-09-10 |
| `0xDAe566ad` market 1 USDC | deliberately moved off in the market-1-inert batch, 2026-09-09, `marginFactor 0` |

Those were the signer's **only** two destinations (95 and 5 of its last 100 transactions).

Supporting evidence, all read off chain rather than assumed:

- `totalPosition`, `totalLongPosition`, `totalShortPosition` on the SEP16 future are **0**, under
both the DFXM and the SRM.
- The SEP16 manager `0xcE01f3D7…4d49` is `allowedModules = false` on Matching, so its accounts
cannot settle a trade whatever any feed says.
- Spot's static feeds answer (`0xec4ad7B2` -> 743376685636834), while the live feed reverts. A real
spot settlement landed on 2026-09-20 (tx `0xb3df1d1d…`) with this feed already 9 days stale.

### The alert retires itself, rather than being switched off

`check_mark_staleness.py` now returns early when open interest is zero, in the same shape as its
existing settled-series exit. The premise is re-checked on **every run**, so if anyone ever opens a
position on this future the alert resumes on its own. A retirement that depends on a human
remembering to undo it is how a venue ends up with an unmonitored market.

If the open-interest read itself fails, it alerts and exits 1 — it will not infer "safe to skip"
from a failure to check. Both paths were exercised before this was written.

### To un-retire

Fund `0xc9f1ffDE…20fdc` (it burns ~0.01 ETH/month at ~1 update/min; 0.05 ETH is ~5 months), and
restart `numo-mark-keeper.service`. The alert needs no change. Do this **before** re-pointing any
market at a live feed, not after.

### Still to do on the ops box (not done from here)

`numo-mark-keeper.service` and `numo-mark-alert-ssm.service` are still enabled. The keeper is
harmless — it fails closed every cycle — but it is noise in the journal, and the alert timer is now
a no-op that still costs an RPC round trip a minute. Stopping and disabling both is the tidy-up.
32 changes: 31 additions & 1 deletion contracts/risk-core/scripts/ops/check_mark_staleness.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,12 +27,18 @@
from pathlib import Path

sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
from mark_keeper import artifact, get_series, get_spot, load_env_file # noqa: E402
from mark_keeper import artifact, get_series, get_spot, load_env_file, run # noqa: E402

MARK_AGE_WARN_SEC = 45 * 60
MARK_DRIFT_WARN_BPS = 150


def total_position(rpc: str, future: str, manager: str) -> int:
"""Open interest on the future, as the manager accounts for it."""
out = run(["cast", "call", future, "totalPosition(address)(uint256)", manager, "--rpc-url", rpc])
return int(out.split()[0].replace(",", ""))


def alert(webhook: str | None, msg: str) -> None:
print(msg, file=sys.stderr)
if not webhook:
Expand All @@ -54,6 +60,30 @@ def main() -> int:
fut = artifact("CNGN_SEP16_2026_FUTURE.json")
future, feed, sub_id = fut["future"], fut["spotFeed"], str(fut["subId"])

# Retired 2026-09-22 while open interest is zero, in the same shape as the settled-series exit
# below: this alert exists because "if marks stop, the losing side of any open position stops
# being margined". With no position there is no losing side and nothing to margin, so the mark
# being stale harms no one and paging about it is noise.
#
# Deliberately a CONDITION, not a deletion or a disabled timer. The premise is checked on every
# run, so the day anyone opens a position here the alert resumes by itself. A retirement that
# needs a human to remember to undo it is how a venue ends up with an unmonitored market.
#
# Context: the cNGN spot feed 0x41512C6a has been stale since 2026-09-11 (its updater ran out of
# gas) and is not being refunded. Spot trading is unaffected -- it reads the static feeds it was
# moved to at the SRM cutover on 2026-09-10 -- and market 1 was likewise moved off its live feed
# on 2026-09-09. Both feeds this signer wrote to are abandoned by design.
try:
manager = fut["manager"]
oi = total_position(rpc, future, manager)
if oi == 0:
print(f"no open interest on {future} (manager {manager}); nothing to margin, mark alert n/a")
return 0
except Exception as exc:
# Fail LOUD: if we cannot establish that open interest is zero, we must not assume it.
alert(webhook, f"NUMO MARK ALERT\nOPEN INTEREST CHECK FAILED (cannot confirm the mark alert is safe to skip): {exc}")
return 1

problems = []
try:
series = get_series(rpc, future, sub_id)
Expand Down
Loading