Feed recording, sessions and cost through one input that does not depend on the Go listeners #1497

Part of #1437 (assessment: laptop data plane on Praxis) · prep work, useful to Cortex on its own

Status: there is no shipped Cortex–Praxis integration today (the paused cmd/cortex-praxis converter is not one), and the interim path is not decided (LLM traffic to praxis-ai as an inference server, or all traffic through Praxis via #1491). Nothing here depends on that choice: each item is useful to Cortex now, and would make a possible move to Praxis easier whichever path is taken.

Problem

If Cortex's data plane ever moves to Praxis, the session store, archive, cost ledger, usage aggregator and agentop must keep working unchanged. Today they are fed only from inside Cortex's Go listeners, so nothing outside those listeners, Praxis included, has anything to plug into. Three things tie recording to the Go proxy:

1. Events are built in 11 places. Each listener builds pipeline.SessionEvent literals by hand, and they record different things for the same kind of exchange.
2. Part of the cost is computed inside a plugin, from data the event does not carry.
3. There is no way to feed events from outside the process.

What Cortex does today

Event construction — 11 literals plus the Deny helpers:

• forward proxy: server.go:1234, :1523; transparent.go:246, :296
• reverse proxy: server.go:457, :634, :827
• ext_proc: server.go:355, :442, :478, :532
• shared: sessionevent.Deny, DenyResponse

Where the listeners differ:

• When they record. ext_proc appends a row only when a parser, an invocation or a plugin entry is present (extproc/server.go:351). Both proxies always record ("Always record", forwardproxy/server.go:1545, reverseproxy/server.go:815). So the same failing request leaves a row on the proxies and none on ext_proc.
• Response-phase rejections. Only the forward proxy records them (DenyResponse). The reverse proxy leaves them unrecorded on purpose; ext_proc returns without recording.

Some fields differ by design and are not drift: only the forward proxy honours redirects (MarkRedirectable), so only it sets requestedHost; only it strips Upgrade, so only it sets upgradeStripped; and tls is reverse-proxy mTLS by definition (EventTLS).

The back end is already clean. Every event goes through one choke point, appendLocked (Append reaches it through append; AppendPair calls it directly). It sets SessionID and Seq, interns strings, then calls the Recorders (usage, ledger, archive).

• The Recorders read the event plus the shared rate table: usage prices an event that carries no cost record from its own rates.
• A replacement sink must also keep the store's optional hooks: Rekeyer, Deleter, Clearer, SeqSeeder, EntryStarter, PriorKeeper. The archive's SeqSeeder sets the store's numbering.

Writes outside Append. Listeners also write to the store directly:

• Rekey: reverseproxy:617, :822, extproc:520
• Claim / ClaimProcess (forwardproxy/server.go:1395, :1398), TouchProcess (:493, :986, :1276)
• Tunnel pairing: AppendPair takes a session id and returns a *session.Bucket (server.go:2633); AppendBucket (transparent.go:240) and AppendTrailing (transparent.go:296) take one. The bucket is what lets a tunnel's close row follow a Rekey and be refused after eviction.

Listeners also read the store, for attribution (#1498) and to build the plugin session view (#1499): ActiveSession, View, SessionForProcess, SessionForClient, ChainSession, ProcessHints, IsAgentProcess. Those are out of scope here.

Cost.

• Computed by settle.Settle(pctx, rates), called from inference-parser.
• It reads pctx.ResponseHeaders (LiteLLM cost headers and Content-Type), pctx.Host and the parsed inference; pctx.Path only in log lines.
• Publish puts the result in Extensions.Custom under two keys (cost and the legacy litellm-budget-track), which reach the event as plugins[...].
• A modelled cost can already be computed from an event alone: usage does it (usage.go:764-860). What can't be recomputed from an event is the gateway's own cost figure, which comes from response headers the event doesn't carry, and the avoided-cost estimate, which needs the request body (avoided.go:53-88).

No ingestion. The session API registers GET routes and two DELETEs only. (GET /v1/events is the SSE stream, so an ingestion route needs a different path.) (sessionapi/server.go:199-209).

Not a consequence of this: cortex-envoy builds a store but no ledger, usage aggregator or archive (warnCostLedgerInert), so /v1/usage answers 404 there. That's deliberate: the ledger is off in Kubernetes by design. Wiring the aggregator there is a separate, two-line choice.

Proposal

1. One set of event builders in core/listener/internal/sessionevent: Request, Response, TunnelOpen, TunnelClose, each taking (pctx, ListenerFacts{status, failure, bytes, reason, duration}). Replace all 11 literals. Decide in one place when a listener records, which removes the ext_proc and response-rejection differences above.
2. One sink interface the listeners write through — Append, the tunnel-pairing calls, Rekey, Claim* — so "feed the control plane" is one interface, not a struct and its methods.
   • Pairing tunnel rows by requestId instead of *Bucket needs a request-id index that survives rekey, adoption and eviction, and a batch form, because AppendPair guarantees nothing lands between the open and close rows (agentop's fold depends on it).
3. Record the cost-relevant response headers on the response event — X-Litellm-Response-Cost, -Original, -Discount-Amount, -Margin-Amount, Content-Type — so the gateway figure can be recomputed from an event. The settle input change itself is Price a request from plain values instead of the plugin request context #1501. Persisting these headers widens the trust surface discussed in security: a gateway's cost header is trusted after a plausibility check, not authenticated #1027 (gateway cost headers are unauthenticated).
4. A loopback-only event-ingestion endpoint (NDJSON, on a path other than /v1/events) into the same sink. Use the same guards as DELETE /v1/sessions: loopback Host, no Origin, gated like WithClearAllowed.
   • The store assigns SessionID and Seq, so ingested events are renumbered, and their numbers must not collide with what the archive already holds.
   • Any local process could inject priced cost records into the ledger; decide whether ingested cost is trusted (see security: a gateway's cost header is trusted after a plausibility check, not authenticated #1027).
5. Version the event schema. sessionEventWire (session.go:493) has no version.
   • The archive has its own formatVersion and stores events with interned fields blanked, and wire-coverage tests already exist (session_test.go:267-370). What's missing is a version on the API's event JSON and golden fixtures.
   • State in docs/session-api.md which fields only the store sets: seq is documented, sessionId isn't.

Items 1–3 and 5 benefit Cortex today. Item 4 can wait until a second producer exists. One already partly does: startup replay feeds the usage aggregator straight from the archive, bypassing the store (cmd/cortex/archive.go:122-150).

Design questions

1. Does ingestion live on the session API port, or on a separate loopback-only listener?
2. Should the ingesting side recompute cost from the event, or trust a producer-supplied cost record?
3. Should ext_proc record every exchange, as the proxies do, or keep recording only parsed ones?

Done when

• No pipeline.SessionEvent{ literal remains in non-test code under core/listener outside sessionevent.
• core/listener/parity asserts identical values, per direction pair (inbound: ext_proc vs reverse proxy; outbound: ext_proc vs forward proxy), for the fields every listener can know.
• A unit test recomputes the gateway cost figure from a recorded response event.
• (Item 4) A replayed NDJSON capture through the ingestion endpoint produces the same /v1/sessions and /v1/usage as the live run.

Evidence read at main @ e455a04 (2026-10-25); not built or run.