Skip to content

Fix the runbook's Workflows doctor step; document today's behavior - #52

Merged
jacobprall merged 4 commits into
render-examples:mainfrom
jacobprall:docs-runbook-refresh
Oct 25, 2026
Merged

jacobprall merged 4 commits into
render-examples:mainfrom
jacobprall:docs-runbook-refresh

Conversation

@jacobprall

Copy link
Copy Markdown
Collaborator

Summary

A review of every doc and the runbook against the current code.

Corrections

  • Workflows doctor: the runbook asked for doctor in the factory-workflows Render Shell, but a Workflows service has none (Render starts a short-lived instance per task run, and render ssh doesn't recognize workflow IDs). The runbook, operations guide, quickstart and AGENTS.md now run the Workflows check locally with that service's values and its deployed slug, which can carry a suffix.
  • Dependency graph: AGENTS.md no longer says the gateway imports events.
  • Troubleshooting: GitHub's Recent Deliveries is now the only record of why a webhook started nothing, so docs/operations.md explains the gateway's responses and ignore reasons, plus its 422 and 502 log lines.
  • Invariants: the Sandbox invariant in architecture.md now matches AGENTS.md.

Newly documented

  • How to read why an agent's submission was rejected, from the agent.finished log line.
  • How to refresh a snapshot by hand, needed after runner or sandbox/ changes. configuration.md links to it.
  • doctor's expected "no webhook deliveries yet" warning before the smoke test, and an optional telemetry check on the smoke test.
  • The strict output schema rule, in CONTRIBUTING.md and AGENTS.md.
  • Two residual risks: telemetry in logs, and unrecorded builder sandboxes. The code map now lists claiming-task.ts.

Code

  • /factory cancel now clears a status comment left on "running" by a run that already stopped, and terminates any Sandbox it left, instead of replying that nothing is running.
  • tests/agents-catalog.test.ts enforces strict output schemas for every agent. I checked that it fails when one schema is loosened.

Verification

npm run check: lint, typecheck, and 110 tests.

- The runbook asked for doctor in the factory-workflows Render Shell, but a
  Workflows service has no Shell: Render starts a short-lived instance per
  task run. The runbook, operations guide, quickstart, and AGENTS.md now run
  the Workflows check locally with that service's values and deployed slug.
- Operations: GitHub's Recent Deliveries is now the record of why a webhook
  started nothing, so the troubleshooting guide explains the gateway's
  responses and ignore reasons, its 422 and 502 log lines, how to read why an
  agent's submission was rejected, how to refresh a snapshot by hand, and how
  to clear a status comment stuck on "running".
- The runbook expects doctor's "no webhook deliveries yet" warning before the
  smoke test, and can confirm telemetry from the smoke test's log line.
- AGENTS.md: the gateway no longer depends on events; agent output objects
  must be strict.
- Architecture: the Sandbox invariant matches AGENTS.md, two residual risks
  are added, and the code map lists claiming-task.ts.
- /factory cancel now clears a status comment left on "running" by a run that
  already stopped, instead of replying that nothing is running.
- tests/agents-catalog.test.ts enforces strict output schemas for every agent.
Use the repository's own spelling for the product name in docs, the runbook, the Blueprint comment, and the GitHub App description that setup:app sends. Agent prompts keep "an unattended software factory", which describes the concept rather than naming the product.
render.yaml now creates factory-gateway and factory-workflows in the production environment of a software-factory project, so a deployment shows up as one project in the Render Dashboard. The docs that describe what the Blueprint creates, and the add-a-database recipe, follow the new layout.
@jacobprall
jacobprall merged commit e9d9280 into render-examples:main Oct 25, 2026
2 checks passed
@jacobprall
jacobprall deleted the docs-runbook-refresh branch October 25, 2026 00:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant