Skip to content

feat: the Decisions service (typed System One decisions) - full spec implementation - #1115

Open
rennf93 wants to merge 22 commits into
slavefrom
feat/decisions-service
Open

rennf93 wants to merge 22 commits into
slavefrom
feat/decisions-service

Conversation

@rennf93

@rennf93 rennf93 commented Sep 23, 2026

Copy link
Copy Markdown
Owner

Summary

Implements the full Decisions service spec (docs/internal/jev-decisions-spec.md): typed System One decisions (choice / score / noul with calibrated confidence) at decision points that are today a full LLM call or a hardcoded heuristic. Three consumer lanes, one wire format, everything default-off and fail-open: with the master flag off, every call site does exactly what it did before.

The spec ships in five rollout stages; this PR builds everything buildable now (Stages 0, 0.5, and the code surfaces of 2.5 and 4). Shadow-data collection, threshold calibration, and arming remain operational follow-ups per the spec's rollout plan.

Stage 0: the service (roboco/services/decisions/)

  • schemas.py: pydantic models mirroring the OpenRouter Decisions wire format; lenient parsing (unknown answer keys ignored, missing confidence = fail-open).
  • client.py: one httpx client for both tiers; single attempt, no retries; per-tier circuit breaker (3 consecutive failures, 5 min open); client-side token caps on every state (diffs 8k, excerpts 2k, descriptions 4k).
  • resolver.py: per-call tier chain - master flag off -> floor; healthy Laya sidecar -> tier 1; sidecar unhealthy/disabled -> OpenRouter if opted in AND the AI Provider screen has a key; missing key fires ONE ack-required CEO notification per boot naming the fix screen. Active tier stamped into every session_id and log line.
  • pilots.py: one function per integration point with the fallback baked in; callers never see raw Jev types. off | shadow | on semantics per pilot (shadow logs verdicts, acts as off).
  • context.py: recipient-context composer for the cognition lane (current task, siblings, parked work, spawn age, journal), best-effort and fail-open.
  • Config: ROBOCO_DECISIONS_ENABLED + tier flags/timeouts/checkpoint id; settings store gains the flag rows and the dotted decisions.pilot.{slug} tri-state rows through a validating chokepoint.

Stage 0.5: the roboco-jev container (the Laya tier)

  • docker/jev.Dockerfile + docker/jev/server.py: CPU-only FastAPI adapter mirroring the OpenRouter Decisions wire shape over Laya's official ONNXAgent (no torch at runtime), fitted-temperature verification at startup (never raw logits), optional timing-safe LAYA_API_KEY.
  • Service entries in both compose twins (byte-synced) and the registry compose: roboco_default by construction, mem-capped 2g, health-probed, no host port. release.yml IMAGES wiring; pull-smoke covers it automatically. Compose posture tests added.

Ops lane (spec 6.1-6.3)

  • self_heal (6.1): transient-vs-regression noul gates self-heal origination; NO_VERDICT/shadow originates exactly as today, only an explicit SKIP suppresses the sweep.
  • parking (6.2): lane choice at _park_provider_unavailable over park_standard / retry_soon / escalate - all already-legal behaviors; below threshold is park_standard.
  • complexity (6.3): spawn-time 0-2 score feeds the ROLE:complexity routing lookup only when the description is meaty and confidence clears the floor; otherwise the static estimated_complexity column stands.

Agent lane (spec 6.4-6.5)

  • preflight_diff (dev surface): composes its whole state server-side from the task record + working diff; one noul per acceptance criterion (cap 6, longest first) plus a hygiene screen; advisory envelope, never blocks or waives i_am_done.
  • triage_failure (dev + QA surfaces): my_regression / flaky / environment; unknown below the confidence floor; a flaky verdict is a citation hint, never a waiver. Role grants in role_config.py, routes, MCP wrappers, and the manifest coverage test.

Cognition lane (spec 6.6 + B28)

  • Steering channel: nullable steering column on a2a_messages (migration 104, purely additive). The steer gate runs in send_chat_message after purpose-dedup: confident steering modes mark the message; the note renders at the recipient's NEXT context boundary (next-spawn briefing block or live turn-queue delivery). Switch-consideration notes carry the recipient's own task list; the switch decision stays with the agent and its PM. No spawns burned; nothing lost; read_a2a unchanged.
  • B28 transcript auto-notes: at spawn finalize, assistant transcript segments are classified for durable decisions/constraints; ON mode persists auto-note journal entries; off by default, never blocking finalization.
  • Dead inbox deleted: the never-wired priority-inbox scaffold in agent_sdk/server.py (queues + /inbox/* routes + InboxResponse).

Panel

  • Feature Flags card descriptions for the three decisions flags (rows appear automatically from the backend registry).
  • New Decisions Pilots card: 7 tri-state (off | shadow | on) rows persisted through the settings store, effective on restart.
  • AI Providers warning banner when the OpenRouter fallback is opted in without a key, backed by GET /api/providers/decisions-status (booleans only, never key material).

Doctrine compliance

  • Fail-open everywhere (the spec's only fail-CLOSED exception, injection screens, is a Tier B candidate not scheduled here).
  • Nothing materializes on a verdict alone; no Jev verdict satisfies, skips, or auto-passes any lifecycle gate; self-review exclusion holds for both agent-lane verbs.
  • One attempt, no retries; every verdict logged with tier, confidence, cost, action, session_id.

Tests

  • ~150 new backend tests: schemas, client transport suite (success, error envelope, timeout, malformed, circuit open, token caps), resolver tier chain (including missing-key notification and Laya fallback), pilot gates + shadow semantics, sidecar wire contract with calibration fixture, call-site wiring, verb envelopes, steer gate + briefing + B28.
  • Full unit suite matches the clean-slave baseline exactly (the remaining failures/errors are pre-existing environment-dependent cases needing live Postgres/git); ruff check + format clean repo-wide; make compose-sync green; panel vitest/eslint/tsc green for the new components.
  • Not runnable here: the sidecar's real HTTP surface against a live container (contract covered offline via the wire fixture) and a live API round-trip for the panel status route (mirrors the sibling provider-key endpoints).

New roboco/services/decisions package: typed-question schemas mirroring
the OpenRouter Decisions wire format, one httpx client for both tiers
(client-side token caps, 3-failure/5-min circuit breaker, single attempt,
fail-open), per-call tier resolver (Laya sidecar default, OpenRouter
opt-in fallback, once-per-boot missing-key CEO notification, 30s cached
health probe), pilot functions with the fallback baked in, and the
recipient-context composer for the cognition lane.

Config: ROBOCO_DECISIONS_ENABLED master flag plus tier flags, timeouts,
and the checkpoint id; settings store gains the three feature-flag rows
and the dotted decisions.pilot.{slug} off|shadow|on tri-state rows with a
validating chokepoint.

Ops-lane wiring, all default-off and fail-open: self-heal origination
gated by the transient-vs-regression noul (6.1), provider-park lane
choice with retry_soon/escalate lanes (6.2), and the spawn-time
complexity score feeding the ROLE:complexity routing lookup (6.3).

Tests: schemas lenient parsing, client transport suite (success, error
envelope, timeout, malformed, circuit open, caps), resolver tier chain
including the missing-key notification, pilot gates and shadow
semantics, sidecar contract fixture, and call-site wiring.
…iner + panel surfaces (spec 6.4-6.5, 8, 10)

Agent lane: two role-scoped do-tools. preflight_diff (dev surface)
composes its whole state server-side from the task record plus the
working diff, answers one noul per criterion (cap 6, longest first) plus
a hygiene screen, and renders an advisory envelope that never blocks or
waives i_am_done. triage_failure (dev + QA surfaces) classifies a red
test into my_regression / flaky / environment, unknown below the
confidence floor, and a flaky verdict is a citation hint, never a
waiver. Both degrade to a graceful no-verdict envelope when Decisions
is off; the tool-manifest build and MCP server registry carry the new
entries, and the coverage test pins them.

roboco-jev container: docker/jev.Dockerfile + docker/jev/server.py
(FastAPI adapter mirroring the OpenRouter Decisions wire shape over
Laya's ONNXAgent, fitted-temperature verification at startup, optional
timing-safe LAYA_API_KEY), service entries in both compose twins and
the registry compose (roboco_default by construction, mem-capped,
health-probed, no host port), release.yml IMAGES wiring, and compose
posture tests.

Panel: Feature Flags card descriptions for the three decisions flags, a
Decisions Pilots card (7 tri-state off|shadow|on rows persisted through
the settings store), the AI Providers warning banner when the
OpenRouter fallback is opted in without a key (backed by a new
GET /api/providers/decisions-status that never returns key material).
…es (spec 6.6, 7.1-B28)

The steering channel: a nullable steering mode column on a2a_messages
(migration 104 off 103_agentrole_devops, purely additive). The Decisions
steer gate runs in A2AService.send_chat_message after purpose-dedup and
before the row insert, classifying delivery against the recipient's
composed work context: steer_switch_consideration / steer_now mark the
message (confidence floor 0.8, CEO conversations untouched); anything
else is today's pull-only behavior. Marked messages render at the
recipient's NEXT context boundary: the next-spawn briefing block in
_write_agent_briefing, or a live turn-queue note via PrompterLiveRegistry
when the recipient has a session. Rendering injects context, never
commands: switch-consideration notes carry the recipient's own task list,
and the switch decision stays with the agent and its PM. Every message
stays readable via read_a2a regardless of verdict.

The dead priority-inbox scaffold in agent_sdk/server.py (in-memory
queues, /inbox/poll|ack|count, InboxResponse) is deleted: nothing ever
queued or polled it in production.

B28 transcript auto-notes: at spawn finalize the assistant segments of
the newest transcript (already opened for token sums) are classified by
a batched noul pass and, in ON mode, persisted as auto-note journal
entries; off by default, best-effort, never blocking finalization.

Also: the new do-tool bodies' free-text fields (error_excerpt,
test_name) join the WAF freetext shield so the gateway false-positive
corpus stays clean; ruff clean repo-wide with justified per-file
exemptions following the established pattern.
@rennf93 rennf93 self-assigned this Sep 23, 2026
@rennf93 rennf93 moved this from Backlog to In progress in RoboCo Kanban Sep 23, 2026
@github-actions github-actions Bot added dependencies pyproject.toml / uv.lock or panel package manifests ci GitHub Actions and CI configuration area: panel Touches panel/ (Next.js control panel) area: api Touches roboco/api/ (FastAPI routes, schemas, app) area: services Touches roboco/services/ (business logic, side effects) area: db Touches roboco/db/ or roboco/models/ area: alembic Touches alembic/ (database migrations) tests Test suite changes area: gateway Touches roboco/services/gateway/ (Choreographer, verb surface) area: orchestrator Touches roboco/runtime/ (agent spawner, dispatch loops) area: mcp Touches roboco/mcp/ (MCP server entry points) build Makefile / Docker / Docker Compose / packaging labels Sep 23, 2026
@github-actions

Copy link
Copy Markdown

Thanks for opening your first pull request on RoboCo!

Quick checklist before review (most of these are enforced by CI, but worth a glance):

  • make quality — ruff format check, ruff check, mypy, pytest (≥80% coverage), and the rest of the gate
  • Panel changes pass pnpm lint and pnpm exec tsc --noEmit (run from panel/)
  • No # noqa / # type: ignore shortcuts; pre-existing violations in touched files are fixed
  • Added an entry under ## [Unreleased] in CHANGELOG.md
  • Signed the CLA (the bot will prompt you on this PR)
  • Signed your commits — master requires verified signatures (SSH signing setup)
  • Updated any affected docs under docs/

See CONTRIBUTING.md for the full workflow and the Code of Conduct for the community standards we follow.

Welcome aboard — a maintainer will review shortly.

…end guard, NAS arming, decision_log persistence

Tier B second wave (spec 7.1, built default-off by CEO pick; arming
awaits shadow data): all 42 remaining rows as pilots with call-site
wiring in four lane modules (pilots_gateway/dispatch/content/infra).
Doctrine constraints enforced per row: B2/B14 injection screens are
fail-CLOSED (below-confidence counts as flagged, may only add
suspicion); B5/B9/B39/B34/B16 may only add edges/reviews/notifications/
postmortems; B41/B30 may only prune injected memory/context; B19/B31/
B42/B43/B35 advise, reorder, and scaffold but never decide gates;
B36/B37 skip/hold only above tight floors; B12 dedups only at 0.9
confidence. Every unset settings row is byte-identical to the prior
behavior.

decisions-audit board program (#15, Auditor, daily): decision_log
persistence (migration 105) fed from the log_action chokepoint - every
pilot verdict lands in one history with tier, mode, answers, confidence,
action, and spend; buffered fire-and-forget writes over their own
session, drop-on-failure. The daily cycle aggregates yesterday against
the trailing-week baseline, sends the CEO an ack-required brief, opens
one held analysis task for the Auditor, and prunes past
decisions_log_retention_days (default 90). Recommendations are
requirements for the CEO; the engine never adjusts a knob itself.

Spend guard (spec 3.1/9): two 402s within a rolling hour or daily
OpenRouter-tier spend over decisions_cost_alert_usd fires one
ack-required CEO alert; startup key check wired into the app lifespan.

NAS arming (CEO 2026-09-23): the build compose ships the master flag ON
with a pilot mix via ROBOCO_DECISIONS_PILOTS_ON/SHADOW env lists -
cheap-failure/advisory/add-only pilots ON, expensive-wrong-answer pilots
SHADOW - while the registry compose ships everything OFF (panel rows
override everywhere). panel row > env list > off is enforced in the
pilot_mode chokepoint.

Container renamed roboco-jev -> roboco-decisions (vendor-neutral per
spec 2: the container runs Laya, not Jev; the model slug stays in
config). Dockerfile, server, both compose twins, registry compose,
release.yml, panel copy, config default, and all fixtures.

Panel: Decisions Pilots card restructured to a data-driven 50-row
render over three sections (Tier A / Cognition lane / Tier B).

Tests: ~400 new (gateway/runtime/services batches, board registry
fifteen-program parity, engine units, compose posture, calibration and
persistence suites). Full unit suite at clean-slave baseline parity;
ruff clean and formatted; compose-sync green.
@rennf93

rennf93 commented Sep 23, 2026

Copy link
Copy Markdown
Owner Author

Update: the full spec is now implemented - Tier B second wave included.

Added since the PR opened (all default-off; NAS arming posture per CEO direction):

  • Tier B second wave, all 42 rows (spec 7.1) across four lane modules: gateway (B1 plan quality, B20 commit intent, B31 findings mapping, B34 idle legitimacy, B39 branch staleness, B41 lesson prune, B43 assembled coherence), dispatch/runtime (B14 external-PR triage fail-closed, B25 idle reaping, B30 context pruning, B32 budget wrap-up, B33 delta brief, B35 review-queue priority, B36 board evidence skip, B37 respawn verdict, B38 submit-now confidence, B40 park cause, B42 PM closure confidence, B44 silent exit), content (B3 intake preroute, B4/B23 X+changelog, B10 segment classify, B11 vault prefilter, B18 secretary NL, B21 telegram gate, B22 memory distill, B24 proactive domain, B26 note sufficiency), infra (B2 injection screen fail-closed, B5 collision edge, B6 CI-watch route, B7 heal severity, B8 release worthy, B9 second review, B12 notify dedup, B13 board due-early, B15 stranded response, B16 coroner, B17 dep-update risk, B19 release readiness). Doctrine constraints per row (add-only / prune-only / advisory / fail-closed) are enforced in the wiring.
  • decisions-audit board program ([9d2b0f97] Create feature branch and initial commit #15, Auditor, daily): decision_log table (migration 105) fed from the log_action chokepoint; daily aggregates (volume, confidence, actions, spend, yesterday vs. trailing-week baseline) as an ack-required CEO brief + one held Auditor analysis task; retention pruning. Auditor recommendations are requirements - the CEO decides; nothing auto-applies.
  • Spend guard: two 402s in a rolling hour or daily fallback spend over decisions_cost_alert_usd -> one CEO alert; startup key check in the app lifespan.
  • NAS arming (mix): build compose ships the master flag ON with ROBOCO_DECISIONS_PILOTS_ON (25 cheap-failure/advisory pilots ON) and ROBOCO_DECISIONS_PILOTS_SHADOW (25 expensive-wrong-answer pilots SHADOW); registry compose ships everything OFF. Panel rows override env lists override off.
  • Container renamed roboco-jev -> roboco-decisions (vendor-neutral per spec 2).
  • Panel: Decisions Pilots card now renders all 50 pilots in three sections.

Still excluded by the spec's own lockouts (section 6 "Explicitly NOT pilots", section 7.2 Tier C): review-gate verdicts (QA severity, PR findings, conventions waivers), KB/RAG reranking, claim guards, deliberative-core replacement. Building those needs an explicit spec change.

Verification: full unit suite at clean-slave baseline parity (the remaining 36 failures/39 errors are pre-existing environment-dependent cases), ~345 new tests, ruff clean + formatted, compose-sync green, panel vitest/eslint/tsc green.

… bug fixes

- mypy: 601 errors fixed across decisions package, tests, engines
  (session annotations, Optional narrowing, Mapping[str, DecisionQuestion]
  for decide_for_pilot/client.decide, _types.py Protocol entry for
  _build_decisions_audit_prompt)
- real bugs: a2a steering imported get_live_registry from the wrong
  module and never awaited deliver(); spend_guard/resolver constructed
  NotificationService(session) though it takes no args (now
  _create_notification(params, db_session=session))
- panel: settings page test mocks DecisionsPilotsCard (no
  QueryClientProvider in that tree)
- xenon: 44 C/D blocks split into private helpers, behavior preserved
- regenerate verb tables (preflight_diff, triage_failure)
@github-actions github-actions Bot added the area: agents Touches agents/ (prompts, role config) label Sep 24, 2026
…seline agent lane, laya state budget, steer ctx threading, lib pin, key cache
… hermetic intake scoped-uuid mock, decide() helper extraction (xenon B), heartbeat renew flake window, isinstance modernization, uv lock refresh
@github-actions

Copy link
Copy Markdown

DevOps image toolchain smoke results

DevOps toolchain smoke check (agent-devops)

docker-cli 28.4.0

@github-actions

Copy link
Copy Markdown

Agent Image Smoke (Playwright) results

Playwright image size delta (real docker inspect sizes)

image before after delta
agent-qa-fe 2114MB 2114MB +0MB
agent-ux 2025MB 2025MB +0MB

Headless chromium launch smoke check

PLAYWRIGHT_SMOKE_OK
PLAYWRIGHT_SMOKE_OK

Playwright MCP server smoke check

Version 0.0.78
Version 0.0.78

Headless browser verification screenshot (ux-qa image)

Screenshot of a live panel page (/login), taken from inside agent-ux:after. See the ux-qa-panel-screenshot build artifact.

@github-actions

Copy link
Copy Markdown

DevOps image toolchain smoke results

DevOps toolchain smoke check (agent-devops)

docker-cli 28.4.0

@github-actions

Copy link
Copy Markdown

Agent Image Smoke (Playwright) results

Playwright image size delta (real docker inspect sizes)

image before after delta
agent-qa-fe 2117MB 2117MB +0MB
agent-ux 2028MB 2028MB +0MB

Headless chromium launch smoke check

PLAYWRIGHT_SMOKE_OK
PLAYWRIGHT_SMOKE_OK

Playwright MCP server smoke check

Version 0.0.78
Version 0.0.78

Headless browser verification screenshot (ux-qa image)

Screenshot of a live panel page (/login), taken from inside agent-ux:after. See the ux-qa-panel-screenshot build artifact.

1 similar comment
@github-actions

Copy link
Copy Markdown

Agent Image Smoke (Playwright) results

Playwright image size delta (real docker inspect sizes)

image before after delta
agent-qa-fe 2117MB 2117MB +0MB
agent-ux 2028MB 2028MB +0MB

Headless chromium launch smoke check

PLAYWRIGHT_SMOKE_OK
PLAYWRIGHT_SMOKE_OK

Playwright MCP server smoke check

Version 0.0.78
Version 0.0.78

Headless browser verification screenshot (ux-qa image)

Screenshot of a live panel page (/login), taken from inside agent-ux:after. See the ux-qa-panel-screenshot build artifact.

@github-actions

Copy link
Copy Markdown

DevOps image toolchain smoke results

DevOps toolchain smoke check (agent-devops)

docker-cli 28.4.0

@github-actions

Copy link
Copy Markdown

Agent Image Smoke (Playwright) results

Playwright image size delta (real docker inspect sizes)

image before after delta
agent-qa-fe 2117MB 2117MB +0MB
agent-ux 2028MB 2028MB +0MB

Headless chromium launch smoke check

PLAYWRIGHT_SMOKE_OK
PLAYWRIGHT_SMOKE_OK

Playwright MCP server smoke check

Version 0.0.78
Version 0.0.78

Headless browser verification screenshot (ux-qa image)

Screenshot of a live panel page (/login), taken from inside agent-ux:after. See the ux-qa-panel-screenshot build artifact.

@github-actions

Copy link
Copy Markdown

DevOps image toolchain smoke results

DevOps toolchain smoke check (agent-devops)

docker-cli 28.4.0

@github-actions

Copy link
Copy Markdown

Agent Image Smoke (Playwright) results

Playwright image size delta (real docker inspect sizes)

image before after delta
agent-qa-fe 2117MB 2117MB +0MB
agent-ux 2028MB 2028MB +0MB

Headless chromium launch smoke check

PLAYWRIGHT_SMOKE_OK
PLAYWRIGHT_SMOKE_OK

Playwright MCP server smoke check

Version 0.0.78
Version 0.0.78

Headless browser verification screenshot (ux-qa image)

Screenshot of a live panel page (/login), taken from inside agent-ux:after. See the ux-qa-panel-screenshot build artifact.

The base typed-decisions checkpoint cannot gate (probe: 5/14 direction
agreement, positives under every floor), so the path to the Laya tier
actually deciding is fine-tuning on our own decisions. This wires the
whole collection-to-training pipeline:

- decision_log rows now carry state+questions exactly as sent on the
  wire (the client stamps its post-cap payloads onto the parsed result)
  plus outcome/outcome_at for ground truth (migration 106, additive).
- record_outcome() labels every unlabeled row of one (pilot, session_id)
  subject inside a savepoint; never raises, never overwrites.
- outcomes.py is the single label registry: (pilot, slug) -> per-question
  gold probabilities. Unknown or unusable slugs are skipped by the
  exporter, never guessed; superseded_by_fix_task is the first unusable.
- the self-heal sweep labels past transient gates from telemetry:
  cleared_after_gate / still_failing_after_window / superseded_by_fix_task,
  on every cycle (all-green ones included), best-effort, sweep never
  breaks; window via self_heal_outcome_window_hours (24h default).
- python -m roboco.services.decisions.export emits the upstream laya
  fine-tune notebook's JSONL shape (id/workflow/state/questions/gold),
  oldest-first, atomic write.
@github-actions

Copy link
Copy Markdown

DevOps image toolchain smoke results

DevOps toolchain smoke check (agent-devops)

docker-cli 28.4.0

@github-actions

Copy link
Copy Markdown

Agent Image Smoke (Playwright) results

Playwright image size delta (real docker inspect sizes)

image before after delta
agent-qa-fe 2117MB 2117MB +0MB
agent-ux 2028MB 2028MB +0MB

Headless chromium launch smoke check

PLAYWRIGHT_SMOKE_OK
PLAYWRIGHT_SMOKE_OK

Playwright MCP server smoke check

Version 0.0.78
Version 0.0.78

Headless browser verification screenshot (ux-qa image)

Screenshot of a live panel page (/login), taken from inside agent-ux:after. See the ux-qa-panel-screenshot build artifact.

CEO direction 2026-09-26: the self-heal engine is already fully armed in
the NAS/dev compose pair (detect + notify + originate, both toggles on);
its laya transient gate moves from the ON list to the SHADOW list so the
sweep logs laya's verdict against real CI outcomes instead of gating on
it - the labeled training feed for the fine-tuning corpus (spec section
12). ON pilots 24, SHADOW pilots 26; twin stays byte-identical; compose
contract tests green.
@github-actions

Copy link
Copy Markdown

DevOps image toolchain smoke results

DevOps toolchain smoke check (agent-devops)

docker-cli 28.4.0

@github-actions

Copy link
Copy Markdown

Agent Image Smoke (Playwright) results

Playwright image size delta (real docker inspect sizes)

image before after delta
agent-qa-fe 2117MB 2117MB +0MB
agent-ux 2028MB 2028MB +0MB

Headless chromium launch smoke check

PLAYWRIGHT_SMOKE_OK
PLAYWRIGHT_SMOKE_OK

Playwright MCP server smoke check

Version 0.0.78
Version 0.0.78

Headless browser verification screenshot (ux-qa image)

Screenshot of a live panel page (/login), taken from inside agent-ux:after. See the ux-qa-panel-screenshot build artifact.

Per the outcome surface map (spec 12.1), grades the unlabeled rows of
the task-trajectory pilots against what the tasks table says happened
after each decision, so the high-frequency surfaces - not just self-heal
- produce labeled training rows:

- idle_legitimacy: the owned tasks' fate (submitted-after, rotted,
  wait-resolved) proves which idle option was true.
- respawn_verdict: delivered/cancelled/stalled-past-rot on the wedged
  task proves spawn / kill-task / hold-task-for-human.
- submit_now_confidence + pm_closure_confidence: QA bounce vs clean
  delivery, as soft low / pole-high golds.
- plan_quality: bounced = inadequate (pole), shipped unreplanned = soft
  adequate-strong, stalled leans inadequate.
- preflight_diff: pass-only grading (criteria held, hygiene clean);
  bounce rows stay for Wave 2 finding-matching.

One periodic labeler loop on the orchestrator (rides the master
decisions flag; writes only outcome columns), rules as pure functions,
conflicting fates stay unlabeled, rot/grade windows in settings.
@github-actions

Copy link
Copy Markdown

DevOps image toolchain smoke results

DevOps toolchain smoke check (agent-devops)

docker-cli 28.4.0

@github-actions

Copy link
Copy Markdown

Agent Image Smoke (Playwright) results

Playwright image size delta (real docker inspect sizes)

image before after delta
agent-qa-fe 2117MB 2117MB +0MB
agent-ux 2028MB 2028MB +0MB

Headless chromium launch smoke check

PLAYWRIGHT_SMOKE_OK
PLAYWRIGHT_SMOKE_OK

Playwright MCP server smoke check

Version 0.0.78
Version 0.0.78

Headless browser verification screenshot (ux-qa image)

Screenshot of a live panel page (/login), taken from inside agent-ux:after. See the ux-qa-panel-screenshot build artifact.

…stamps, triage history, instance session keys

Completes the gradeable surface per the spec 12.1 map: labeled pilots go
from 7 to 12 (plus 2 partial), with per-question ground truth for the
batched pilot.

- migration 107 + decision_log.question_outcomes: per-question fate maps
  for batched pilots whose questions each carry their own truth;
  persist.record_question_outcomes writes them savepoint-wrapped,
  never overwriting.
- findings_mapping graded from the task_review_findings ledger:
  verified/addressed = resolved (gold true), still-open-past-rot =
  re_raised (gold false), waived = recorded but unusable.
- parking graded at three evidence moments: repark within the window
  (spawn_exit, escalate gold), provider lift on resume (probe path,
  latency splits retry_soon vs park_standard), re-limit bail
  (escalate); stale rows past the rot horizon are retired as unusable,
  never guessed.
- triage_failure graded from the test's own later history in
  decision_log: the same test failing later on file-disjoint diffs
  proves flake; no recurrence plus task delivery proves the regression.
- second_review_eligibility + assembled_coherence join the trajectory
  rules (post-gate bounce vs clean delivery).
- the eight singleton pilots (release:worthy, secondreview:gate,
  strategy:stranded, readiness:risk, notify:dedup, secretary:kind/
  :assignee, distill, coroner:{kind}) get content-addressed session
  keys via the state_key helper: per-instance outcome labeling is now
  possible for all of them.
- exporter consumes question_outcomes through the registry's fate map;
  ci_watch_route + dep_update_risk explicitly deferred in the map (need
  persisted CI run history - faking it from one latest-conclusion read
  would poison the corpus).

Gates: 8967 unit tests, ruff, xenon B/A/A.
@github-actions

Copy link
Copy Markdown

DevOps image toolchain smoke results

DevOps toolchain smoke check (agent-devops)

docker-cli 28.4.0

@github-actions

Copy link
Copy Markdown

Agent Image Smoke (Playwright) results

Playwright image size delta (real docker inspect sizes)

image before after delta
agent-qa-fe 2117MB 2117MB +0MB
agent-ux 2028MB 2028MB +0MB

Headless chromium launch smoke check

PLAYWRIGHT_SMOKE_OK
PLAYWRIGHT_SMOKE_OK

Playwright MCP server smoke check

Version 0.0.78
Version 0.0.78

Headless browser verification screenshot (ux-qa image)

Screenshot of a live panel page (/login), taken from inside agent-ux:after. See the ux-qa-panel-screenshot build artifact.

…rrection capture

Completes the derivable surface from the spec 12.1 map. Labeled pilots
go from 12 to 16 (complexity, heal_severity, ci_watch_route urgency,
dep_update_risk), with the last two unlocked by finally persisting what
the pull-based telemetry layer used to throw away:

- migration 108: ci_runs (per-reading run history, deduplicated on
  project+workflow+observed stamp, pruned by the labeler pass) and
  memory_retrieval_log (what institutional memory was injected into
  which spawn - the foundation the knowledge-family pilots grade
  against once a stable note->lesson identity exists).
- the ci-watch and self-heal engines record their already-fetched
  samples as run history; no extra GitHub calls.
- ci_watch_route urgency graded from the post-decision run sequence
  (green-first = flaky, repeated reds = hard); the route choice stays
  counterfactual with no gold. dep_update_risk graded from post-bump
  failures vs a clean sequence.
- complexity and heal_severity graded from the delivered work's
  realized size (commit count + elapsed time); heal_severity's session
  is now keyed per breach fingerprint and the caller passes it.
- secretary confirm capture: the CEO running a pilot-filled directive
  unchanged proves the kind it picked (fates confirmed:<kind> for the
  five directive kinds); rejections stay unlabeled - ambiguous between
  wrong kind and bad directive.

Gates: 8973 unit tests, ruff, xenon B/A/A.
@github-actions

Copy link
Copy Markdown

DevOps image toolchain smoke results

DevOps toolchain smoke check (agent-devops)

docker-cli 28.4.0

@github-actions

Copy link
Copy Markdown

Agent Image Smoke (Playwright) results

Playwright image size delta (real docker inspect sizes)

image before after delta
agent-qa-fe 2117MB 2117MB +0MB
agent-ux 2028MB 2028MB +0MB

Headless chromium launch smoke check

PLAYWRIGHT_SMOKE_OK
PLAYWRIGHT_SMOKE_OK

Playwright MCP server smoke check

Version 0.0.78
Version 0.0.78

Headless browser verification screenshot (ux-qa image)

Screenshot of a live panel page (/login), taken from inside agent-ux:after. See the ux-qa-panel-screenshot build artifact.

…orrection grading

The last gradeable task-trajectory surfaces, per the spec 12.1 map:

- stranded_response: the blocked batch's fate grades the lane decision.
  Every task cleared after the decision = wait was right; human-cancelled
  or still-stuck past the rot horizon = escalate was warranted; respawn
  is never auto-derived. Mixed fates stay unlabeled.
- collision_edge: per-pair question outcomes from the delivered tasks'
  recorded commit files - intersecting files confirm the semantic edge,
  disjoint files disprove it; pairs with an undelivered side wait.
- release_worthy + release_readiness correction capture: the CEO's
  approve/reject at the proposal decision points grades the urgency and
  risk calls (approve = pressing + acceptable risks; reject-with-changes
  = the opposite, soft golds). Session keys recomputed exactly as the
  pilots built them (capped summaries, commit_count, open_gap_count).

Honest corrections to the earlier achievable-ten list, now documented in
the map: tool_spotlight needs agent/task attribution on verb traces,
transcript_notes + the lesson family need the note-to-lesson identity,
and secretary-assignee/intake have no correction path in their flows.

Gates: 8981 unit tests, ruff, xenon B/A/A.
@github-actions

Copy link
Copy Markdown

DevOps image toolchain smoke results

DevOps toolchain smoke check (agent-devops)

docker-cli 28.4.0

@github-actions

Copy link
Copy Markdown

Agent Image Smoke (Playwright) results

Playwright image size delta (real docker inspect sizes)

image before after delta
agent-qa-fe 2117MB 2117MB +0MB
agent-ux 2028MB 2028MB +0MB

Headless chromium launch smoke check

PLAYWRIGHT_SMOKE_OK
PLAYWRIGHT_SMOKE_OK

Playwright MCP server smoke check

Version 0.0.78
Version 0.0.78

Headless browser verification screenshot (ux-qa image)

Screenshot of a live panel page (/login), taken from inside agent-ux:after. See the ux-qa-panel-screenshot build artifact.

…grading

The last buildable family from the spec 12.1 map. migration 109 adds
memory_links: a decision row -> knowledge item link that closes the
identity gap between what laya decided and what the memory system did
with it afterwards.

- memory_distill_gate graded from the distilled learning's later
  retrievals: retrieved after the decision = the persist-worthy claim
  held; no retrieval past the rot horizon = it did not. The distill
  persist site links the decision to the learning's content-addressed
  source URI (roboco://learnings/lrn-{md5}, recomputed deterministically
  via learning_source_uri - no shared state needed).
- vault_prefilter graded from the board-review draft's fate: approved
  and started proves the note warranted a task; a cancelled draft
  proves it did not; still-pending waits.
- the memory_distill_session_key/state helpers expose the gate's
  content-addressed session key to the persist site, so the link lands
  on the right decision row without guessing.

Honest corrections to the map: transcript_notes' journal entries never
enter the retrieval corpus (needs journal indexing - a product decision,
documented); lesson_prune's applicability is unobservable even with
identity (its lessons are retrieved by definition).

Labeled pilots: 18 of 50 (16 + these two), 2 partial, 30 input-only.
Gates: 8985 unit tests, ruff, xenon B/A/A.
@github-actions

Copy link
Copy Markdown

DevOps image toolchain smoke results

DevOps toolchain smoke check (agent-devops)

docker-cli 28.4.0

@github-actions

Copy link
Copy Markdown

Agent Image Smoke (Playwright) results

Playwright image size delta (real docker inspect sizes)

image before after delta
agent-qa-fe 2117MB 2117MB +0MB
agent-ux 2028MB 2028MB +0MB

Headless chromium launch smoke check

PLAYWRIGHT_SMOKE_OK
PLAYWRIGHT_SMOKE_OK

Playwright MCP server smoke check

Version 0.0.78
Version 0.0.78

Headless browser verification screenshot (ux-qa image)

Screenshot of a live panel page (/login), taken from inside agent-ux:after. See the ux-qa-panel-screenshot build artifact.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: agents Touches agents/ (prompts, role config) area: alembic Touches alembic/ (database migrations) area: api Touches roboco/api/ (FastAPI routes, schemas, app) area: db Touches roboco/db/ or roboco/models/ area: gateway Touches roboco/services/gateway/ (Choreographer, verb surface) area: mcp Touches roboco/mcp/ (MCP server entry points) area: orchestrator Touches roboco/runtime/ (agent spawner, dispatch loops) area: panel Touches panel/ (Next.js control panel) area: services Touches roboco/services/ (business logic, side effects) build Makefile / Docker / Docker Compose / packaging ci GitHub Actions and CI configuration dependencies pyproject.toml / uv.lock or panel package manifests tests Test suite changes

Projects

Status: In review

Development

Successfully merging this pull request may close these issues.

1 participant