Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion apps/api/src/cora/agent/prompts/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,8 @@
KNOWN_PROMPT_TEMPLATES: dict[UUID, str] = {
RUN_DEBRIEF_PROMPT_TEMPLATE_ID: "RunDebrief v1: terminal-Run AAR narrative + advisory choice",
CAUTION_DRAFTER_PROMPT_TEMPLATE_ID: (
"CautionDrafter v1: terminal-Run Caution proposal with 5-choice verdict"
"CautionDrafter v1: terminal-Run Caution proposal with 5-choice verdict; "
"payload now also carries the terminal snapshot's frame-tally capture_progress"
),
}

Expand Down
68 changes: 67 additions & 1 deletion apps/api/src/cora/agent/prompts/caution_drafter.py
Original file line number Diff line number Diff line change
Expand Up @@ -55,11 +55,26 @@

## Read scope (v1)

v1 reads: terminal Run event + Run aggregate state + existing
v1 reads: terminal Run event (including its frame-tally snapshot, when
the terminal event carries one) + Run aggregate state + existing
Active Cautions for the target (via `CautionLookup` port). Deferred
to v2 per design memo: RunDebriefer's prior Decision for the same Run
(needs `DecisionLookup` port; deferred until pilot UX surfaces need).

Named boundary, not a TODO: this agent has no recurrence signal across
Runs on the same target. It reads one terminal event at a time, so it
cannot tell "this Run was short" from "this Asset is short on a third
of its Runs, by a consistent margin." Originating a Caution from a
repeated shortfall therefore cannot happen in v1; a shortfall alone
stays `NoAction` (see the system prompt's "Frame-count context"
section). A pilot observation grounds this, measured 2026-08-21: every
recorded completion declares the same expectation of 1541 saved
frames; 616 met it exactly and 364 fell short by about 11 frames on
average. A shortfall there is common and systematic, not exceptional,
which is exactly why a single occurrence is weak evidence and why a
cross-Run recurrence signal, once it can be sourced without a new
lookup/port, is the next slice.

## Structured output schema

JSON Schema with five fields, four always-required + one
Expand Down Expand Up @@ -404,6 +419,41 @@

If no Active Caution matches, propose new (pick the severity tier).

## Frame-count context (capture_progress)

The input payload may carry `capture_progress`: the terminal snapshot's
frame tallies for a witnessed capture. Not every Run is a witnessed
capture; when the field is absent, draw no conclusion from its absence.

A `frames_saved` shortfall against `frames_saved_expected` is a fact about
ONE RUN. RunDebriefer already records exactly that fact on its own
Decision, as `DataSuspect`; do not duplicate it here. A Caution is a claim
about an ASSET, and an asset-level claim needs a pattern, which a single
Run cannot establish. This payload carries one Run, so it does not carry
sufficient evidence to originate a Caution from a shortfall.

A shortfall alone must never produce a proposal. `NoAction` is the
correct verdict for an isolated shortfall.

Refusing here discards nothing. The tallies stay on the terminal event
permanently, and RunDebriefer's verdict is a durable Decision; the
occurrence is not lost by declining to propose from it.

Exception: when an `existing_cautions` entry for the same target already
describes frame loss or detector trouble, a fresh shortfall is
corroboration that the condition persists, and `ProposeSupersede` is
available under the "Lookback then propose" rule above. This is a
secondary path, not the main mechanism: reach for it only when the
existing Caution already names the pattern, never as a way to originate
one from a shortfall alone.

Never compare a saved count with a collected count. They come from
different instruments and their totals are not the same quantity, so a
difference between them means nothing. Each count is only ever comparable
with its own `_expected` pair (`frames_saved` against
`frames_saved_expected`; `frames_collected` against
`frames_collected_expected`).

## Categories (closed 6-value set)

Pick the ONE that most narrowly fits:
Expand Down Expand Up @@ -493,6 +543,20 @@ class CautionDrafterPayload:

`informed_by_decision_id` is reserved for v2 when DecisionLookup
ports ship; v1 always None.

`capture_progress` mirrors `RunDebriefPayload`'s field of the same
name exactly: the terminal snapshot's frame tallies
(`frames_saved` / `frames_saved_expected`, `frames_collected` /
`frames_collected_expected`), plus `reading_age_seconds_before_terminal`
when the tallies carry a parseable timestamp pair, or `None` for a
Run with no witnessed-capture snapshot. Absence is ordinary, not a
fault: not every Run is a witnessed capture. See
`extract_capture_progress` in `_terminal_run_helpers` for the
extraction and the full provenance discussion. Unlike RunDebriefer,
which reports a shortfall as a fact about the one Run it just
watched, CautionDrafter's job is to decide whether the ASSET
warrants a standing advisory; see the system prompt's "Frame-count
context" section for the current framing of that distinction.
"""

terminal_event_type: str
Expand All @@ -510,6 +574,7 @@ class CautionDrafterPayload:
interrupted_at: str | None
candidate_targets: tuple[CandidateTarget, ...] = field(default_factory=tuple)
existing_cautions: tuple[ExistingCaution, ...] = field(default_factory=tuple)
capture_progress: dict[str, int] | None = None


def build_caution_drafter_chat_request(
Expand Down Expand Up @@ -580,6 +645,7 @@ def _payload_to_json_safe(payload: CautionDrafterPayload) -> dict[str, Any]:
}
for ec in payload.existing_cautions
],
"capture_progress": payload.capture_progress,
}


Expand Down
2 changes: 2 additions & 0 deletions apps/api/src/cora/agent/subscribers/caution_drafter.py
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,7 @@
CAUTION_DRAFTER_AGENT_NAME,
)
from cora.agent.subscribers._terminal_run_helpers import (
extract_capture_progress,
extract_interrupted_at,
extract_reason,
)
Expand Down Expand Up @@ -492,6 +493,7 @@ async def apply(self, event: StoredEvent, conn: ConnectionLike) -> None:
interrupted_at=interrupted_at,
candidate_targets=candidate_targets,
existing_cautions=existing_cautions,
capture_progress=extract_capture_progress(event),
)
# The Agent's declared model, not the module default: that
# declaration is what `define_agent` gated against the approved
Expand Down
53 changes: 53 additions & 0 deletions apps/api/tests/unit/agent/test_caution_drafter_prompt.py
Original file line number Diff line number Diff line change
Expand Up @@ -168,6 +168,59 @@ def test_payload_with_no_candidate_targets_serialises_empty_list() -> None:
assert parsed["existing_cautions"] == []


@pytest.mark.unit
def test_build_request_carries_capture_progress_when_snapshot_present() -> None:
"""The frame tallies must reach the user message JSON verbatim so the
LLM can read them under the system prompt's "Frame-count context"
section."""
payload = _payload(
capture_progress={
"frames_saved": 1528,
"frames_saved_expected": 1541,
"frames_collected": 1541,
"frames_collected_expected": 1541,
"reading_age_seconds_before_terminal": 13,
}
)
request = build_caution_drafter_chat_request(payload)
json_blob = request.user_message.text[request.user_message.text.index("{") :]
parsed = json.loads(json_blob)
assert parsed["capture_progress"] == {
"frames_saved": 1528,
"frames_saved_expected": 1541,
"frames_collected": 1541,
"frames_collected_expected": 1541,
"reading_age_seconds_before_terminal": 13,
}


@pytest.mark.unit
def test_build_request_capture_progress_defaults_to_none() -> None:
"""A Run with no witnessed-capture snapshot must still build a valid
request, with `capture_progress` travelling as JSON `null` rather
than being omitted."""
request = build_caution_drafter_chat_request(_payload())
json_blob = request.user_message.text[request.user_message.text.index("{") :]
parsed = json.loads(json_blob)
assert parsed["capture_progress"] is None


@pytest.mark.unit
def test_system_prompt_forbids_originating_from_a_lone_shortfall() -> None:
"""Pin the load-bearing invariant, not a section title: a single Run's
shortfall must never by itself produce a Caution proposal. A future
reword of the section is fine; this rule collapsing back into a
per-run alert is not."""
assert "A shortfall alone must never produce a proposal" in CAUTION_DRAFTER_SYSTEM_PROMPT


@pytest.mark.unit
def test_system_prompt_forbids_comparing_saved_and_collected_counts() -> None:
"""Pin the other load-bearing invariant: saved and collected tallies
come from different instruments and are never mutually comparable."""
assert "Never compare a saved count with a collected count" in CAUTION_DRAFTER_SYSTEM_PROMPT


@pytest.mark.unit
def test_decision_context_constant_matches_design_lock() -> None:
"""The context value used by the subscriber is `CautionProposal`."""
Expand Down
Loading
Loading