Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
83 changes: 78 additions & 5 deletions docs/features/replay.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,13 +32,83 @@ aiui run "$(python -c 'import praisonaiui.replay_app as m; print(m.__file__)')"
`/#/replay?run=<id>` works too. In a full dashboard, Replay is in the sidebar under
**Control**.

## Watch a run live: the Run button

For learning, demos and debugging, open a page with a **Run** button. Each click records
a new run, and the page draws it as it happens:

```python
from praisonai.replay import watch

watch(flow, input="solar sails") # an Agent, AgentTeam, AgentFlow or agents.yaml
```

```bash
praisonai replay watch agents.yaml
```

On the **Map**, each step and agent shows where the run is: dashed means not reached,
blue (pulsing) means running now, green means done and red means failed. After a run,
**▶ Play** moves through it again one step at a time, and the map rewinds with it. One
run goes at a time. Each run of a team starts its tasks afresh; an Agent keeps its chat
history between runs.

## Show progress inside your own app

`create_replay_app()` is the same view as a small ASGI app, without the dashboard. Mount
it in any FastAPI or Starlette app:

```python
from praisonai.replay import record, replay_app

app.mount("/replay", replay_app()) # read-only: shows recorded runs

# wherever your app starts work:
with record(run_id, live=True):
team.start()
```

Then show one run, live, in your page, for example in an `<iframe>`:

```html
<iframe src="/replay/?run=RUN_ID" style="width:100%;height:640px"></iframe>
```

| URL | Shows |
|---|---|
| `/replay/` | The run list and the selected run |
| `/replay/?run=<id>` | Only that run, no list. A run that has not started yet is waited for. |

`replay_app(team)` adds the Run box to the mounted page. Without the PraisonAI wrapper,
pass any `run(input, run_id)` callable to `praisonaiui.features.replay.create_replay_app`;
it must record the run under `run_id`. For a page you build yourself, the view is one
module:

```html
<div id="progress"></div>
<script type="module">
import { render } from "/replay/replay.js";
render(document.getElementById("progress"), { base: "/replay", run: "RUN_ID", approve: false });
</script>
```

`render()` options: `base` (where the API is mounted), `run` (show only this run),
`runner` (`{input, placeholder, ask}` to show the Run box) and `approve` (`false` hides
the approve and deny buttons; the embedded app has no approvals API, so answer approvals
where your app asks for them).

Anyone who can reach the mounted app can read the runs, and with a runner can start one.
Keep it on localhost or behind your app's login. `POST /api/replay/run` only accepts
`Content-Type: application/json`, so another website cannot start a run from a visitor's
browser.

## What you see

| Part | What it shows |
|---|---|
| **Runs** | Recorded runs, newest first. A run that is still being written has a **Live** badge. |
| **Timeline** (default) | One lane per agent, plus **Manager** and **You**. Time runs left to right. LLM calls and tool calls are blocks in their agent's lane. Arrows show hand-offs, delegation, routing and approval requests. Parallel branches get their own sub-lanes. A call or step repeated back to back is one block marked ×N. Tasks and steps run across the top. |
| **Map** | The team or flow as a graph, drawn from the graph recorded at the start of the run and fitted to the page (−, Fit and + zoom; drag to pan). Agents that took part but are not in the graph, for example after a hand-off to an agent outside the team, appear in an **Outside the team** group with their hand-offs. Older recordings without a graph get a "who talked to whom" map built from the events. Nodes you have not reached yet have a dashed outline; the node the cursor is on has a bold one. |
| **Map** | The team or flow as a graph, drawn from the graph recorded at the start of the run and fitted to the page (−, Fit and + zoom; drag to pan). Agents that took part but are not in the graph, for example after a hand-off to an agent outside the team, appear in an **Outside the team** group with their hand-offs. Older recordings without a graph get a "who talked to whom" map built from the events. Each node shows its state at the cursor: dashed means not reached, blue means running, green means done (×N when it ran more than once) and red means failed; the node the cursor is on has a bold outline. |
| **What happened** | One plain sentence per step, for example *"Researcher asked web_search for 'solar sail facts' — 1.2 s, $0.002."* Hovering or focusing a sentence highlights its block, and the other way round. |
| **Details** | Three tabs for the selected step. **What happened**: when, who, how long, cost, tokens, output. **Why**: only what the recording holds, such as the input given to a tool, the context passed in a hand-off and its reason, the manager's task snapshot, the route a step chose, and an approval's decision and where an automatic approval came from. **Raw JSON**: the events themselves. |
| **Cost & latency** | Adds a bar under each block (taller and darker means more cost, or more time) and per-agent totals in the lane labels. |
Expand All @@ -65,7 +135,8 @@ anything after the cursor is drawn as an outline.
| Enter or Space | Open the focused block or node |
| ← → in the tabs | Switch between What happened, Why and Raw JSON |

The scrubber under the timeline does the same with a mouse or touch. On a live run the
The scrubber under the timeline does the same with a mouse or touch, and **▶ Play** steps
through on its own. On a live run the
cursor follows new events; if you step back, **Jump to live** takes you to the latest.

## Approving from Replay
Expand All @@ -92,11 +163,13 @@ an exported file.
| Endpoint | Returns |
|---|---|
| `GET /api/replay/runs` | `[{run_id, session_id, path, started_at, ended_at?, status, kind, name, agents, events, cost_usd, live}]`, newest first |
| `GET /api/replay/runs/{id}/events?after_seq=N` | `{events, graph, run}`; `after_seq` keeps only events with a larger `sequence_num` |
| `GET /api/replay/runs/{id}/events?after_seq=N` | `{events, graph, run}`; `after_seq` keeps only events with a larger `sequence_num`; `wait=1` answers `{events: [], graph: null, run: null}` instead of 404 for a run that has not started yet |
| `GET /api/replay/runs/{id}/stream` | Server-sent events: `event: ctx` per trace event (with `id:` = `sequence_num`), `ping` every 15 s, `end` once the run has finished and gone quiet for 2 s. Honours `after_seq` and `Last-Event-ID`. |

Run ids must match `^[A-Za-z0-9._-]+$`. Replay is read-only: it never writes trace
files. Half-written last lines, unknown fields and unknown event types are skipped
With a runner (`create_replay_app(run=...)`), `POST /api/replay/run` with
`{"input": "..."}` starts a run and returns `{run_id}` (409 while one is going).

Run ids must match `^[A-Za-z0-9._-]+$`. Replay never writes trace files itself. Half-written last lines, unknown fields and unknown event types are skipped
safely, and schema 1.0 recordings still open.

The **Traces** page reads the same files: each run is a trace, and its spans come from
Expand Down
148 changes: 145 additions & 3 deletions src/praisonaiui/features/replay.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,9 @@
Routes (contract v1, section 5):

* ``GET /api/replay/runs`` run picker rows, newest first
* ``GET /api/replay/runs/{run_id}/events`` ``{events, graph, run}``; ``?after_seq=N``
* ``GET /api/replay/runs/{run_id}/events`` ``{events, graph, run}``; ``?after_seq=N``;
``?wait=1`` answers ``{events: [], graph: null, run: null}`` (not 404) for a run
that has not written its first event yet
* ``GET /api/replay/runs/{run_id}/stream`` SSE tail of the JSONL file

The trace directory is ``$PRAISONAI_TRACES_DIR``, else ``$PRAISON_HOME/traces``,
Expand All @@ -12,20 +14,30 @@
The browser side lives in ``templates/frontend/plugins/views/replay-core.js``
(pure functions) and ``replay.js`` (DOM/SVG). :func:`standalone_html` inlines
both into one self-contained page for ``praisonai replay export --format html``.

:func:`create_replay_app` is the same view as a small ASGI app (no dashboard) to
mount inside your own app, optionally with a Run box that starts runs::

app.mount("/replay", create_replay_app(run=my_runner))
"""

from __future__ import annotations

import asyncio
import html
import inspect
import json
import logging
import threading
import time
import uuid
from pathlib import Path
from typing import Any, Dict, List, Optional
from typing import Any, Callable, Dict, List, Optional
from urllib.parse import quote

from starlette.concurrency import run_in_threadpool
from starlette.requests import Request
from starlette.responses import JSONResponse, StreamingResponse
from starlette.responses import FileResponse, HTMLResponse, JSONResponse, StreamingResponse
from starlette.routing import Route

from . import _trace_jsonl as tj
Expand Down Expand Up @@ -233,6 +245,8 @@ async def _runs(self, request: Request) -> JSONResponse:
async def _events(self, request: Request) -> JSONResponse:
path, err = _resolve(request)
if err is not None:
if err.status_code == 404 and request.query_params.get("wait") == "1":
return JSONResponse({"events": [], "graph": None, "run": None})
return err
after_seq, err = _parse_after_seq(request)
if err is not None:
Expand Down Expand Up @@ -297,3 +311,131 @@ async def gen():
media_type="text/event-stream",
headers={"Cache-Control": "no-cache", "Connection": "keep-alive", "X-Accel-Buffering": "no"},
)


# ── embeddable app ─────────────────────────────────────────────────────

#: Longest input the Run box accepts (characters).
MAX_RUN_INPUT = 20_000


def _embed_page(base: str, title: str, cfg: Dict[str, Any]) -> str:
"""The page at the app's root: the view, mounted with *cfg*, and nothing else."""
return (
"<!doctype html>\n<html lang=\"en\">\n<head>\n<meta charset=\"utf-8\">\n"
"<meta name=\"viewport\" content=\"width=device-width, initial-scale=1\">\n"
f"<title>{html.escape(title)}</title>\n"
f"<link rel=\"icon\" href=\"data:image/svg+xml,{quote(REPLAY_MARK_SVG.replace('currentColor', '#1A56DB'))}\">\n"
"<style>body{margin:0;font-family:system-ui,-apple-system,'Segoe UI',sans-serif}"
"#replay-root{padding:12px;box-sizing:border-box;max-width:100%}</style>\n"
"</head>\n<body>\n<main id=\"replay-root\"></main>\n"
"<script type=\"module\">\n"
f"import {{ render }} from {_json_for_script(base + '/replay.js')};\n"
f"render(document.getElementById('replay-root'), {_json_for_script(cfg)});\n"
"</script>\n</body>\n</html>\n"
)


def create_replay_app(
run: Optional[Callable[[str, str], Any]] = None,
*,
title: str = "PraisonAI Replay",
input: Optional[str] = "",
placeholder: str = "",
):
"""PraisonAI Replay as a small ASGI app to mount in your own app.

Serves the view at ``/`` (no dashboard around it), its two scripts, and the
``/api/replay/...`` routes, all relative to where it is mounted::

from fastapi import FastAPI
from praisonaiui.features.replay import create_replay_app

app = FastAPI()
app.mount("/replay", create_replay_app())
# <iframe src="/replay/?run=RUN_ID"> shows one run live; /replay/ lists runs

Args:
run: Optional ``run(input, run_id)``. When given, the page shows an
input box and a **Run** button; each click calls it in a worker
thread with a new ``run_id``, and the page follows that run live.
It must record the run under ``run_id`` (``praisonai.replay.app()``
does this for an agent, team or flow). One run at a time.
title: Page title.
input: Text the input box starts with; ``None`` for no input box
(just a Run button, e.g. for a YAML file that has its own topic).
placeholder: Placeholder text for the input box.

Traces are read from the same folder as the rest of Replay
(``$PRAISONAI_TRACES_DIR`` > ``$PRAISON_HOME/traces`` > ``~/.praison/traces``).
Anyone who can reach the app can read the runs (and, with ``run``, start
them): keep it on localhost or behind your app's auth.
"""
from starlette.applications import Starlette

feature = ReplayFeature()
busy = threading.Lock()

async def page(request: Request) -> HTMLResponse:
base = str(request.scope.get("root_path", "")).rstrip("/")
cfg: Dict[str, Any] = {"base": base, "approve": False}
wanted = request.query_params.get("run")
if wanted and tj.is_valid_run_id(wanted):
cfg["run"] = wanted
if run is not None:
cfg["runner"] = {"input": input or "", "placeholder": placeholder, "ask": input is not None}
return HTMLResponse(_embed_page(base, title, cfg), headers={"Cache-Control": "no-store"})

def script(path: Path):
async def serve(request: Request) -> FileResponse:
return FileResponse(path, media_type="text/javascript", headers={"Cache-Control": "no-cache"})
return serve

def _call(text: str, run_id: str) -> None:
try:
result = run(text, run_id) # type: ignore[misc]
if inspect.isawaitable(result):
asyncio.run(result)
except Exception:
logger.exception("replay run %s failed", run_id)
finally:
busy.release()

async def start(request: Request) -> JSONResponse:
# JSON only: a cross-site form post cannot set this content type without
# a CORS preflight, which this app never answers.
if not request.headers.get("content-type", "").lower().startswith("application/json"):
return JSONResponse({"error": "send JSON: {\"input\": \"...\"}"}, status_code=415)
try:
body = await request.json()
except ValueError:
return _bad_request("body must be JSON")
text = body.get("input", "") if isinstance(body, dict) else None
if not isinstance(text, str) or len(text) > MAX_RUN_INPUT:
return _bad_request(f"input must be a string of at most {MAX_RUN_INPUT} characters")
if not busy.acquire(blocking=False):
return JSONResponse({"error": "a run is already going; wait for it to finish"}, status_code=409)
run_id = f"run-{uuid.uuid4().hex[:12]}"
try:
threading.Thread(target=_call, args=(text, run_id), name=f"replay-{run_id}", daemon=True).start()
except Exception:
busy.release()
raise
# Answer once the trace file exists (usually milliseconds), so the page
# can open the run straight away; it waits on its own if this times out.
path = tj.run_path(run_id)
for _ in range(40):
if path is not None and path.is_file():
break
await asyncio.sleep(0.05)
return JSONResponse({"run_id": run_id})

routes = [
Route("/", page, methods=["GET"]),
Route("/replay.js", script(REPLAY_VIEW_JS), methods=["GET"]),
Route("/replay-core.js", script(REPLAY_CORE_JS), methods=["GET"]),
*feature.routes(),
]
if run is not None:
routes.append(Route("/api/replay/run", start, methods=["POST"]))
return Starlette(routes=routes)
52 changes: 51 additions & 1 deletion src/praisonaiui/templates/frontend/plugins/views/replay-core.js
Original file line number Diff line number Diff line change
Expand Up @@ -1024,6 +1024,56 @@ export function layoutRunMap(graph, events, opts = {}) {
};
}

/** End events that mean "this node finished one run", and which node that is. */
const RUN_ENDS = {
step_end: (nid, ev) => nid === ev.node_id,
task_end: (nid, ev) => nid === ev.node_id,
agent_end: (nid) => nid.startsWith('agent:') || nid === MANAGER_LANE,
tool_call_end: (nid) => nid.startsWith('tool:'),
};

/**
* Each map node's state after the first ``upTo + 1`` events (sorted, as in
* buildModel().events): ``{events, runs, state}`` where state is "running"
* (started, not ended yet), "done", "failed" or "skipped" (from the last end's
* ``data.status``). Nodes without an entry were not reached by then. The flow's
* start/end nodes count as reached once the run started/ended.
*/
export function nodeStates(events, upTo = Infinity, nodes = []) {
const out = new Map();
const last = Math.min(events.length - 1, upTo);
let ended = false;
for (let i = 0; i <= last; i++) {
const ev = events[i];
const type = ev && ev.event_type;
if (type === 'workflow_end' || type === 'session_end') ended = true;
for (const nid of nodeIdsForEvent(ev)) {
const st = out.get(nid) || { events: 0, runs: 0, open: 0, last: null };
st.events += 1;
if (PAIRS[type]) st.open += 1;
else if (END_OF[type]) {
st.open = Math.max(0, st.open - 1);
if (RUN_ENDS[type] && RUN_ENDS[type](nid, ev)) {
st.runs += 1;
const status = str(dataOf(ev).status);
st.last = status === 'failed' || dataOf(ev).error ? 'failed' : (status === 'skipped' ? 'skipped' : 'done');
}
}
out.set(nid, st);
}
}
for (const n of nodes || []) {
if (out.has(n.id)) continue;
if ((n.kind === 'start' && last >= 0) || (n.kind === 'end' && ended)) out.set(n.id, { events: 1, runs: 1, open: 0, last: 'done' });
}
for (const st of out.values()) {
st.state = st.open > 0 ? 'running' : (st.last || 'done');
delete st.open;
delete st.last;
}
return out;
}

/** Words for edge kinds on the map (labels never rely on colour). */
export const EDGE_WORDS = Object.freeze({
handoff: 'hands off',
Expand All @@ -1045,7 +1095,7 @@ export function edgeText(edge) {

const api = {
PALETTE, SHAPES, HUMAN_LANE, MANAGER_LANE, EDGE_WORDS,
buildModel, narrate, layoutMap, layoutRunMap, observedGraph, deriveGraph, nodeIdsForEvent, pairEvents, sortEvents,
buildModel, narrate, layoutMap, layoutRunMap, observedGraph, deriveGraph, nodeIdsForEvent, nodeStates, pairEvents, sortEvents,
eventId, formatDuration, formatCost, argPreview, truncate, stripPrefix, edgeText,
};
if (typeof globalThis !== 'undefined') globalThis.PraisonReplayCore = api;
Expand Down
Loading
Loading