Feat: workflow handoff (external code-generation)

[jeromevdl]

Important

This PR replaces #407 It is based directly on current main and preserves the reviewed external-handoff work without the previous branch-history issues.

Summary

This PR adds an additional native workspace export mode for one Construction unit: code-generation only, with workflow continuation in Collaborative AI-DLC.

When a non-autonomous unit reaches the existing validation gate immediately before code-generation, the developer can either:

• approve and continue with managed code generation; or
• approve and develop the unit externally in an IDE, CLI, hosted workspace, remote agent, or
  another preferred environment.

The second option parks only that unit lane. Independent lanes and the overall intent remain running. The developer downloads a native AI-DLC workspace scoped to the selected unit, pushes changes to the existing unit branch, then submits the native code-generation-plan and code-summary. Collaborative AI-DLC validates and imports the result, completes the parked code-generation stage, and resumes the normal cloud workflow.

What changed

Construction gate and handoff

• Honor existing per-stage validation gates in parallel Construction lanes.
• Keep the walking skeleton gated and preserve the existing AI-DLC autonomy decision: autonomous remaining lanes continue without optional gates, while gated lanes stop before code generation.
• Add Approve and develop externally at the validation immediately before code-generation.

• Reuse the existing HUMAN#, STAGE#, and UNIT# lifecycle instead of adding a separate handoff state machine.
• Add Cancel external development to return only that unit to managed collaborative AIDLC generation.

Unit-scoped native export

• Add a unit-handoff projection mode on top of the exporter from feat: export native AI-DLC workspaces #403.
• Select one running unit and verify that its code-generation stage is parked on the current external-development task.
• Export the unit's existing repository branches and recorded base SHAs.
• Restrict native execution to the selected unit's code-generation stage. Later stages remain cloud-owned and run after the handoff is accepted.
• Reuse the existing workspace export confirmation dialog with handoff-specific messaging: code-generation only, assigned branches

• Extend the post-download setup dialog with copyable extraction, repository setup, harness, document lookup, interactive source staging, commit, and exact-branch push commands.
• Keep normal workflow-continuation exports unchanged.

Submission and workflow continuation

• Accept exactly the semantic plan and summary document slots and canonicalize them to code-generation-plan.md and code-summary.md.
• Read current unit-branch heads directly from GitHub, GitLab, or Bitbucket; the client does not provide trusted SHAs.
• Validate repository assignments, branch heads, and ancestry from the recorded base revision.
• Validate independent repositories concurrently and freeze the complete ordered revision vector.
• Fetch and check out every exact submitted revision in the existing lane workspace.
• Import both native documents with task, stage, unit, content-hash, and revision provenance.
• Complete the existing human task and durable callback only after import succeeds.
• Preserve validation findings on the pending task so the developer can correct and resubmit.

Relationship to #324

This PR implements a practical first slice of #324 using the workflow and exporter infrastructure that already exists. It intentionally does not implement the RFC's full session platform.

Implemented

• A developer can take a unit out of managed code generation at the existing pre-code-generation gate, including for the walking skeleton.
• The claim is scoped to one unit, one stage attempt, and the developer who selected external development.
• Independent unit lanes continue running.
• Existing unit branches are used for external source changes.
• The server observes branch heads, verifies ancestry, freezes exact revisions, and checks out those revisions itself.
• Multi-repository revisions are accepted atomically.
• Stale submissions are fenced by the existing task ownership, pending task pointer, stage state, and stage attempt.
• The developer can abandon external development and return the unit to managed code generation.
• After acceptance, Collaborative AI-DLC resumes its existing downstream testing, review, reconciliation, PR, and merge flow.

Not implemented in this PR

• Lease heartbeats, expiry, renewal, or administrator revocation APIs.
• A remote MCP gateway for external agents, including remote questions, answers, comments, or short-lived capability tokens.
• IDE-specific launch integrations such as Codespaces, Dev Containers, VS Code Remote, or JetBrains.
• A client-submitted commit SHA. This implementation deliberately observes and freezes the current assigned branch heads on submission.
• Online creation of the canonical plan before development and online generation of the summary after source submission. This slice imports the two native documents produced by the external code-generation workflow.
• A new shared hosted/external finalizer abstraction.
• Additional changed-path or protected-file policy enforcement beyond branch assignment, exact revision checkout, and base-ancestry validation.

For these reasons, this PR only references #324 (but does not close it).

Verification

• Projector and exporter regression tests cover the new projection mode and preserve the existing workflow-continuation mode.
• Orchestrator tests cover walking-skeleton gates, gated and autonomous lanes, external development, cancellation back to managed execution, stale-attempt fencing, and replay.
• Submission tests cover document validation, concurrent multi-repository validation, exact revision checkout, artifact import, callback resumption, and rejected resubmission.
• Frontend tests cover the pre-code-generation choice, workspace download, document submission, mode-specific confirmation and setup guidance, commit/push commands, validation findings, and cancellation.
• Focused backend and frontend tests, type checking, formatting, lint, Terraform formatting, and diff checks pass.

Related to #324.

[JWThewes]

The overall approach is sensible: reusing the existing HUMAN/STAGE/UNIT lifecycle and native exporter keeps the feature's scope manageable. I am requesting changes for the five reproduced correctness and lifecycle issues in the inline comments: submission races, runtime routing, parallel export isolation, retry identity, and retained construction capacity.

Validation against a7823eb: 727 existing tests passed across focused backend/frontend suites, including exporter and durable replay checks; frontend type checking passed. Five additional local reproduction tests fail on the reported issues. These were local checks, not a deployed end-to-end run.

[JWThewes]

[P1] Claim submission ownership before importing

The acceptance CAS runs after import-handoff-artifacts has already changed the lane checkout and canonical artifacts. Two submissions can both pass updateExternalDevelopment while the task is pending. If submission A pauses before import, B can import and be accepted, then A can overwrite B's artifacts before its own acceptance returns 409. Downstream execution has already been resumed with B's accepted metadata.

I reproduced this by holding A's import until B completed: B returned 200, A returned 409, and the live code-summary hash matched A while the accepted task recorded B. Please claim exclusive submission ownership before any checkout/artifact mutation and enforce that ownership throughout import and acceptance, including against cancellation.

[JWThewes]

[P1] Use the intent's saved runtime target for handoff imports

This invocation uses the fallback runtime ARN and omits the saved environment endpoint. Normal stage execution resolves meta.environment through the shared runtime-target helper. For an intent using a managed environment, the import can therefore update a different runtime's checkout while downstream stages resume in the original environment with its existing checkout.

A reproduction with a saved managed runtime and revision_r_7 endpoint still invoked the default ARN without a qualifier. Please use runtimeTargetInput(meta, AGENTCORE_RUNTIME_ARN()) here and base the configuration check on that resolved target as well.

[JWThewes]

[P2] Scope pending gates to the selected handoff unit

This filter removes the selected task but retains pending tasks from sibling units, then collapses them to the shared stage ID. When two independent units are parked for external code generation, the sibling task adds code-generation to awaitingGateStageIds. The selected unit's stage is consequently marked ?, and the later requirement that its marker be - rejects the export with "handoff code-generation stage is not active". Both handoffs can block each other's downloads.

I reproduced this with two pending unit handoffs; removing only the sibling task made the same export succeed. Please scope gate projection by the selected unit/stage instance and section.

[JWThewes]

[P2] Give a fresh handoff a distinct identity on lane retry

The task ID contains the persisted stage attempt but no lane retry identity. The existing halt-and-ask retry creates a new lane context while preserving that attempt. After an external handoff succeeds and a later stage or merge fails, choosing external development again therefore reuses the already-answered task. Task creation fails and is swallowed, then setGateCallbackId rejects the answered row and the lane fails with gate_callback_conflict.

I reproduced a successful handoff followed by a second handoff at the preserved attempt: the second failed at callback binding. Please distinguish fresh handoffs across lane retries while keeping replay of the same handoff idempotent.

[JWThewes]

[P2] Release construction capacity while external development is parked

Stopping the runtime session does not release the semaphore permit acquired by runLaneBody. The lane holds that permit throughout this potentially long external-development wait. With maxParallelUnits = 1, an independent unit in the same wave cannot even initialize until the external handoff finishes; with larger limits, queued independent work is blocked once all permits are occupied by handoffs. This contradicts the intended unit-local pause and the existing release of construction capacity during PR review waits.

I reproduced this with a completed skeleton and two independent remaining units: the second unit started only after the first handoff was accepted. Please release the permit while parked and reacquire it before managed execution resumes.