Part of #431 (memory-exhaustion thrash). Prerequisite for lxc-oomd.
Problem
Node-level events (a container frozen/killed due to memory thrash by lxc-oomd) need to reach a human, but we don't want to hard-couple detection to email. We need a decoupled notification mechanism: node-side tools POST an event; the web app surfaces it.
Deliverable
A notification queue in the create-a-container manager app:
- Inbound webhook endpoint (e.g.
POST /api/v1/notifications, admin-API-key authenticated) that accepts a structured event payload.
- Persistence: a
Notifications table (Sequelize model + migration).
- UI surface: a notifications list (on
/agents or a new page) showing recent events, severity, related container/owner, and read/ack state.
- Optional email sink: reuse
create-a-container/utils/email.js as one configurable output, not the primary path.
Payload contract (initial)
{
"source": "lxc-oomd",
"severity": "warning|critical",
"node": "opensource-phxdc-pve1",
"ctid": 392,
"owner": "mbachelder",
"action": "freeze|kill|bump|quarantine|detect",
"message": "CT 392 frozen: memory PSI full avg10=83 for 45s",
"evidence": { "psiFullAvg10": 83.6, "refaultRate": "...", "topProcs": ["..."] },
"ts": 1771234560
}
Acceptance criteria
- Webhook accepts and persists the payload; rejects unauthenticated calls.
- Events appear in the UI with owner/container/severity and can be acked.
- Email sink is optional and configurable.
- Contract documented so the lxc-oomd hook script can target it.
Notes
This is the fully buildable piece today (pure app code, no node access). It is the integration target for the lxc-oomd packaged hook script.
Part of #431 (memory-exhaustion thrash). Prerequisite for lxc-oomd.
Problem
Node-level events (a container frozen/killed due to memory thrash by lxc-oomd) need to reach a human, but we don't want to hard-couple detection to email. We need a decoupled notification mechanism: node-side tools POST an event; the web app surfaces it.
Deliverable
A notification queue in the
create-a-containermanager app:POST /api/v1/notifications, admin-API-key authenticated) that accepts a structured event payload.Notificationstable (Sequelize model + migration)./agentsor a new page) showing recent events, severity, related container/owner, and read/ack state.create-a-container/utils/email.jsas one configurable output, not the primary path.Payload contract (initial)
{ "source": "lxc-oomd", "severity": "warning|critical", "node": "opensource-phxdc-pve1", "ctid": 392, "owner": "mbachelder", "action": "freeze|kill|bump|quarantine|detect", "message": "CT 392 frozen: memory PSI full avg10=83 for 45s", "evidence": { "psiFullAvg10": 83.6, "refaultRate": "...", "topProcs": ["..."] }, "ts": 1771234560 }Acceptance criteria
Notes
This is the fully buildable piece today (pure app code, no node access). It is the integration target for the lxc-oomd packaged hook script.