Skip to content

Notification queue + inbound webhook in create-a-container #434

Description

@runleveldev

Part of #431 (memory-exhaustion thrash). Prerequisite for lxc-oomd.

Problem

Node-level events (a container frozen/killed due to memory thrash by lxc-oomd) need to reach a human, but we don't want to hard-couple detection to email. We need a decoupled notification mechanism: node-side tools POST an event; the web app surfaces it.

Deliverable

A notification queue in the create-a-container manager app:

  • Inbound webhook endpoint (e.g. POST /api/v1/notifications, admin-API-key authenticated) that accepts a structured event payload.
  • Persistence: a Notifications table (Sequelize model + migration).
  • UI surface: a notifications list (on /agents or a new page) showing recent events, severity, related container/owner, and read/ack state.
  • Optional email sink: reuse create-a-container/utils/email.js as one configurable output, not the primary path.

Payload contract (initial)

{
  "source": "lxc-oomd",
  "severity": "warning|critical",
  "node": "opensource-phxdc-pve1",
  "ctid": 392,
  "owner": "mbachelder",
  "action": "freeze|kill|bump|quarantine|detect",
  "message": "CT 392 frozen: memory PSI full avg10=83 for 45s",
  "evidence": { "psiFullAvg10": 83.6, "refaultRate": "...", "topProcs": ["..."] },
  "ts": 1771234560
}

Acceptance criteria

  • Webhook accepts and persists the payload; rejects unauthenticated calls.
  • Events appear in the UI with owner/container/severity and can be acked.
  • Email sink is optional and configurable.
  • Contract documented so the lxc-oomd hook script can target it.

Notes

This is the fully buildable piece today (pure app code, no node access). It is the integration target for the lxc-oomd packaged hook script.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions