fix(runtime): seal authority selectors for prompts rendered at runtime

[mrthankyou]

Fixes #1234.

Problem

A planned task whose prompt waits on a dynamic group is rendered at runtime, not at plan time. Authority selectors were computed only from plan-time renders (promptArtifactAuthoritySelectorsFor), so these tasks were compiled with none. materializePromptArtifactAuthority then returned early, and the authority file the prompt names (.ultrafuzz/authorities/<attempt>.json) was never written.

On the stock default topology this affects:

• dedupe-findings, which names 3 authorities;
• aggregate-test-files, which names 1;
• final-report, which names 2.

The goal-group templates name no authorities, so generated children are not affected. Smoke-profile runs have no dynamic groups and were unaffected.

Change

• smithers.ts: a selector depends only on its placeholder (a contract ID, or a path list and its selector ID), not on the expansion. For a task with a deferred prompt, promptArtifactAuthoritySelectorsForTemplate therefore derives the selectors at compile time from the sealed template copy that renderReadyRuntimePrompts reads. It uses extractPromptVariables and keys selectors exactly as a rendered prompt does.
  • The sealed manifest stays the single source of selectors. Nothing changes at runtime, so the publish, verify and derive modes of the dynamic runtime still agree.
• CHANGELOG entry.

Not in this PR

#1237's prompt refresh still publishes runtime prompts without checking their authorities against the compiled selectors (Greptile's declined P1 there). Those tasks now have a compiled set, so that check can be added as a follow-up.

Verification

• New test (dynamic-lifecycle.test.ts): a planned join behind the fanout group uses {{ancestor_artifact_path_authority:plan.json}}. The test asserts that its runtime-rendered prompt names the authority file and that the task carries selectors.
  • It fails on unstable (reproduced on Linux and macOS) and passes here.
• Stock-topology test extended (runtime.test.ts, resume applies an edited stock review prompt that names artifact authorities once its groups expanded): it now asserts 3, 1 and 2 selectors on dedupe-findings, aggregate-test-files and final-report, and its Write the prompt input index from the live task plan; delete the sealed prompt-artifact authority binding #1234 comment is corrected.
• Linux runs (node:24-bookworm, Bun 1.3.14), 0 failures in both:
  • dynamic-lifecycle, dynamic-workflow and dynamic-expansion test files: 48 of 51 passed;
  • runtime.test.ts tests matching authorit|prompt|refresh|dynamic|selector|deferred: 37 of 40 passed, 3 skipped.
• Clean: ESLint and Prettier on the changed files.
• Not done:
  • a real default-profile campaign to watch the authority file appear in a task worktree;
  • the full runtime.test.ts, which is left to CI.

🤖 Generated with Claude Code

Confidence Score: 4/5

The PR appears safe to merge; an end-to-end authority-file assertion would strengthen the regression coverage.

Findings

1.  Authority file remains untested ▶

Fix with agent prompt

### Issue 1
packages/runtime/test/dynamic-lifecycle.test.ts:1085-1089
The test confirms that the rendered prompt names an authority file and that the task has a selector, but the fixture never runs the join task that writes the file. A selector for the wrong output, or a failure to resolve the expanded group’s artifacts, could still pass this test. Please exercise that path and check that the task-local authority JSON contains the selected `plan.json` output.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

The PR compiles authority selectors from sealed templates for prompts deferred until dynamic groups expand, and adds regression assertions for a dynamic join and the stock topology.

• The new checks establish that selectors are present, but do not verify the task-local authority file that motivated the fix.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Sealed deferred template] --> B[Compile authority selectors]
  B --> C[Expand dynamic group]
  C --> D[Render join prompt]
  C --> E[Publish tasks with child dependencies]
  D --> F[Execute join]
  E --> F
  F --> G[Derive and write task-local authority JSON]

Loading

_{Reviews (1) · Last reviewed commit: "fix(runtime): seal authority selectors f..."}

[greptile-apps]

Authority file remains untested The test confirms that the rendered prompt names an authority file and that the task has a selector, but the fixture never runs the join task that writes the file. A selector for the wrong output, or a failure to resolve the expanded group’s artifacts, could still pass this test. Please exercise that path and check that the task-local authority JSON contains the selected plan.json output.

Knowledge Base Used: Runtime orchestration

Prompt To Fix With AI

This is a comment left during a code review.
Path: packages/runtime/test/dynamic-lifecycle.test.ts
Line: 1085-1089

Comment:
**Authority file remains untested** The test confirms that the rendered prompt names an authority file and that the task has a selector, but the fixture never runs the join task that writes the file. A selector for the wrong output, or a failure to resolve the expanded group’s artifacts, could still pass this test. Please exercise that path and check that the task-local authority JSON contains the selected `plan.json` output.

**Knowledge Base Used:** [Runtime orchestration](https://app.greptile.com/monad-foudnation/-/custom-context/knowledge-base/monad-developers/ultrafuzz/-/docs/runtime-orchestration.md)

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

[aviggiano]

Thanks. The root cause is right: deferred-join tasks get no authority selectors, so the authority file their prompt names is never written. Deriving the selectors from the sealed template copy also looks correct. But merging this as-is makes the default profile worse, so we're leaving it open for now. Here's the problem.

What breaks. Once dedupe-findings (which has dynamicDependencies) gets selectors, materializePromptArtifactAuthority derives authority against <executionSnapshotRoot>/controls/tasks.json (workflow.tsx ~491-494). That manifest was sealed at launch, before any dynamic group expanded. So it doesn't list the goal-child artifact directories that admission lets in later. indexAdmittedDependencies then throws on every attempt:

prompt artifact authority admitted dependency is outside the sealed ancestor closure: "artifacts/dynamic-fanout-…"

(prompt-artifact-authority.ts:478)

Repro. In the dynamic-lifecycle fixture, with a join prompt that uses {{ancestor_artifact_path_authority:plan.json}}, call derivePromptArtifactAuthority for the join task twice, with the same admitted dependency dirs:

• against the execution-snapshot controls/tasks.json, it throws (above);
• against the live smithers/tasks.json, it is OK (["planner"]).

Impact. On a default-profile run where any goal child succeeds, dedupe-findings fails in preparation on every retry. Triage, severity classification, aggregate-test-files and final-report never run, so the run produces no report. Today the same run produces a report and only the authority file is missing. CI doesn't catch it for two reasons: the e2e topology has no dynamic groups, and the new tests only check selector counts and prompt text, never derivation through the snapshot path (Greptile's P2).

What a fix needs. Authority for tasks with dynamic dependencies has to be derived from a manifest that includes the runtime closure. One option is the dynamic task plan re-derived from the sealed base controls plus the digest-bound expansion manifests. The launch snapshot alone isn't enough. Until then, those tasks shouldn't get selectors. Related, and pre-existing: aggregate-test-files' sealed closure leaves out generated goal children, because dynamic lowering only extends direct dependents (#1176).

[mrthankyou]

Agreed on the regression; holding this open. I confirmed the path on this branch. A snapshot-launched engine resolves taskManifestPath to <executionSnapshotRoot>/controls/tasks.json (workflow.tsx, hydrateTaskSpec). That manifest is sealed before any dynamic group expands, so once dedupe-findings has selectors, derivePromptArtifactAuthority rejects the admitted goal-child directories (prompt-artifact-authority.ts:478). Before this PR the same tasks returned early because they had no selectors, which is why the default profile still produces a report today.

Proposed rework, not started yet:

1. Write a failing test first. In the dynamic-lifecycle fixture, run the join task's preparation through the execution-snapshot path, with admitted goal-child artifact directories, and assert that the task-local authority JSON is written and selects the expected plan.json output. This reproduces the regression above, covers Greptile's "authority file remains untested" comment, and would fail on the current head.
2. Derive authority for tasks with dynamicDependencies from the runtime closure, not the launch snapshot. The workflow already rebuilds and verifies the dynamic task plan from the sealed base controls (runtime-base-tasks.json) plus the digest-bound expansion manifests, to render these tasks' prompts. Deriving authority against that same verified plan would make the authority describe exactly the closure the prompt was rendered against. Tasks without dynamic dependencies keep the launch-snapshot manifest unchanged.
3. Keep the selectors compiled at plan time (this PR's change). The sealed manifest stays the single source of selectors. Only the closure they are resolved against changes.

Not proposed: leaving selectors off tasks with dynamic dependencies until (2) lands. That would make this PR a no-op for dedupe-findings, aggregate-test-files and final-report, the tasks it targets.

The aggregate-test-files closure gap (#1176, where dynamic lowering extends only direct dependents) predates this PR. With (2) in place it should show up as a failing case for aggregate-test-files in the new test, to be tracked under #1176 rather than worked around here.

[aviggiano]

Thanks for the detailed follow-up. Your diagnosis is right. I'm closing this, though: we've decided to remove the sealing machinery, and both this PR and the proposed rework build on it (selectors resolved against the sealed base controls and digest-bound expansion manifests).

I've rescoped #1234 to a simpler fix: write the prompt input index from the live task plan for every task, and delete the authority seal checks. Thanks for the careful work here.

[aviggiano]

Superseded by #1285, which implements the rescoped #1234: the prompt input index is written for every task from the live task plan, and the sealed authority selectors are deleted. Your reproduction and diagnosis here shaped it, and its commits credit you as co-author (Co-authored-by: mrthankyou). Thanks!