Repository navigation
✨🧑💻 - Improve bounded drafting and companion recovery for large SDDs #763
Description
Is there an existing issue or feature request for this?
- I have searched the existing issues and feature requests.
Related closed work: #695 (deterministic case budgets/traceability), #714 (safe blocked-SDD diagnostics) and #541 (provider drafting failures). Reuse their contracts and existing fixes; this follow-up concerns proportional large-document planning and recovery. #704 is the concrete reproducer, not the place to implement this improvement.
What area does this improvement affect?
Other
What actions does this improvement affect?
Issue
Description of the idea or improvement
Make preparation of large, complete SDDs predictable without reducing product scope or weakening the pre-branch gate. A legitimate existing-owner change can exhaust its bounded appendix generation twice, despite an explicit request for a compact appendix, and only proceed after a maintainer manually selects the companion path.
Investigate the exact generation/assembly/context cause before choosing an implementation. The rejected candidates were not published, so the observed error alone does not prove that the model repeated the entire owner, that the validator is wrong, or which intermediate representation exceeded its limit.
Desired behavior:
- Use immutable owner/inventory evidence when selecting an update or companion. A document omitted from bounded model context must not be described as nonexistent without verification.
- Plan an appropriately sized change: reference the existing contract rather than unnecessarily repeating it. Preserve every resolved requirement, explicit case, acceptance budget and reciprocal mapping; “compact” must never mean dropping scope.
- Account for derived-table expansion and the final assembled owner document when assessing feasibility. Keep draft, appendix, assembled document and provider-output limits distinct, with units stated accurately.
- If an appendix cannot be prepared within the supported bounds, offer a concrete companion proposal under the same catalog capability, with path, reason, retained baseline and required maintainer decision. Do not repeatedly ask the maintainer to guess a recovery sentence or invent another implementation branch.
- Reuse 🐛🧑💻 - Expose blocked pre branch SDD outcomes and safe recoverable draft diagnostics #714's safe structured diagnostics and durable card. Revalidate what is already fixed on develop before claiming another diagnostic defect. Show available closed stage/count/limit facts and the next supported action; do not expose rejected provider output, secrets or private source.
- Preserve the existing bounded generation/repair policy. A transition requiring an ownership decision must remain explicit and must not become an unbounded hidden regeneration loop. A linked branch's established SDD ownership cannot silently change.
Current limitations or challenges
Observed on 2026-10-28 while starting extension H3 #704:
- The gate pinned develop
4f69a8f9761568eac17120e9fe819a2c08ee0703. Its first question card initially asserted that the owning specifications were absent. The existing ownerspecs/github-browser-extension-control-center-and-setup.mdwas present at that base (blob8d6a96236111162085a7b1b194b9c1a86587b487; 41,477 UTF-8 bytes, not a measured rejected-draft length). - Run 40715722878 exhausted the one automatic size repair: failure comment, reference
947c9691-d3e0-415f-9f0b-13616843928e. - A fresh clarification explicitly requested an H3-only appendix near 6,000 characters, below the 18,000-character update bound, preserving the existing owner and full product/case scope by reference. Run 40716701933 failed identically after its bounded repair: failure comment, reference
0f69002d-2407-4612-9138-989cf26f12c0. - After an explicit companion decision, run 40717544625 succeeded. The Action published the companion at its first commit and created the linked branch normally. The original owning document remained unchanged.
The workaround unblocked #704. There is no need to retry those failed invocations or change H3's scope now. The runtime in the failed jobs was production master 39679e26a22b40a5a4a40e8f167f46273157182e; newer develop diagnostics must be evaluated separately.
Expected impact
Large, comprehensive specifications can reach a validated Action-owned branch through an understandable bounded flow. Maintainers retain the full contract and know whether to refine the appendix, approve a companion or fix a prerequisite.
Acceptance should include deterministic fixtures for: a large existing owner with a small valid appendix; oversized raw output; expansion of derived tables; total composed-owner overflow; malformed output that must not be misclassified as a size problem; correct existing-file evidence despite omitted model context; explicit companion recovery under the same owner; stale base/issue or changed linked branch; case collisions and complete budget/traceability preservation; retained answers and immutable first publication; and safe localized/idempotent recovery content. A later successful live reproducer supplements these regressions rather than replacing them.
Alternatives considered
The explicit companion recovery above is the working current alternative. Better detection and guidance may be sufficient; investigate before introducing automatic splitting. Do not simply raise limits, truncate content, lower the test budget, relax validation, expose raw rejected drafts or bypass branch creation/publication proof.
Version of copilot
Production Action 3.10.5, exact runtime 39679e26a22b40a5a4a40e8f167f46273157182e; Codex CLI 0.165.0 in the observed jobs. Immutable consumer baseline: 4f69a8f9761568eac17120e9fe819a2c08ee0703.
Additional context or comments
This follow-up was deferred when filed. The maintainer has now requested the normal start for #763; this does not reopen #704 or initiate a release, hotfix or deployment.
When deliberately started, follow the normal Action-managed issue/SDD/branch flow. Relevant implementation: src/application/usecases/sdd/pre_branch_sdd_gate_use_case.ts, src/application/usecases/sdd/pre_branch_sdd_draft_workflow.ts, src/domain/pre_branch_sdd.ts, src/domain/sdd_draft_consistency.ts and src/infrastructure/pre_branch_sdd_workspace_adapter.ts; owning contract specs/pre-branch-sdd-gate.md and user guide docs/issues/pre-branch-sdds.mdx. Keep SDD/catalog evidence, regression tests and user recovery guidance synchronized.
Start context: verified route for #763
Use action=companion, existing catalog capability issue-start-and-sdd-readiness, and the new path specs/sdd-bounded-drafting-and-companion-recovery.md. On reviewed develop a859c9e9e, that path is absent and unregistered, and the principal owner specs/pre-branch-sdd-gate.md is 187,073 UTF-16 code units (187,176 UTF-8 bytes), already above the 70,000-code-unit validation limit for a composed update. Preserve the existing owner documents and their case budgets by reference. Recheck the route against the actual selected immutable base. This selects the route for this issue; the feature must still require an explicit maintainer decision before switching any other issue to a companion.
Evidence from the discussion
- Domus #17, #20 and #21, #25, #27, and #23 document response-stage 18,000-limit rejections after two attempts on the released runtime. The rejected content and exact counts are unavailable. The valid independent proposals for Pasive Usage #20 (16,142 raw / 16,772 assembled / 60,969 composed) and 🐛🧑💻 - Deploy key to compile #27 (16,421 / 17,504 / 58,906) prove a proportional delta can fit. 🐛🧑💻 - Deploy key to compile #27 also supplies the near-capacity 66,994-character owner control. These measured examples do not establish what the model returned. Native same-capability companion publication was verified for Pasive Usage #20, #21 and 🐛🧑💻 - Deploy key to compile #27. The Test new close-merge checks #23 companion instead reached a separate section 15 path-operation rejection.
- ha-ui #7 size rejection, input correction, assembly rejection and verified publication show distinct size, input-delivery, and mapping outcomes. The approved companion ultimately retained the byte-identical foundation and all 64 baseline cases, adding six distinct cases for three delivery requirements. The issue body and authorized numbered answers reach preparation; ordinary comments are evidence for humans, not drafting inputs. Document the actual supported decision surface.
- Domus #30 and its follow-up reproduce failures even with twelve explicit cases. The locally validated companion measured 11,816 assembled characters but native generation failed path validation, including a missing canonical Path/Change table in one run. A locally validated 11,765-character appendix retained the 28,218-character baseline; the native update still hit the 18,000 response bound. This supports evaluating a strictly validated, authorized seed or more targeted repair as alternatives; it does not establish either as the required design. Keep 🐛 - SDD drafting omits the required hyphenated requirement ID syntax #905's requirement ID grammar separate.
What to verify before implementation
- Reuse the current fixes for deterministic tables (🐛🧑💻 - SDD draft generation repeatedly disagrees with its own case budget and traceability #695), closed diagnostics and the durable card (🐛🧑💻 - Expose blocked pre branch SDD outcomes and safe recoverable draft diagnostics #714), targeted structural feedback (🐛🧑💻 - Pre branch SDD assembly rejects an undefined requirement after the bounded repair on a resolved UI feature #833), immutable source binding (🐛🧑💻 - Pre branch SDD analysis reads the default checkout instead of its selected immutable development base #836), ownership continuity (🐛🧑💻 - SDD ownership analysis proposes registered path companion for an explicit existing update request #856), and draft grounding (🐛🧑💻 - Published consumer SDD requires a nonexistent repository verification script #858). Keep provider token/output failures (🐛🧑💻 - bug pre branch SDD drafting fails after successful analysis with an unclassified agent exit #541) distinct from Markdown limits. Compare deployed pins and current develop; do not claim any historical hidden output or infer a size failure from malformed data.
- On develop, the response schema counts Unicode code points; the subsequent raw, rendered and composed Markdown checks use JavaScript UTF-16 length. The update prompt can inline the full owner, and exact composition is only checked after assembly/consistency. These are inspectable mechanisms, not a diagnosis of the historical runs. Verify UTF-8 byte counts, character units, advisory size estimates, and provider token budgets separately.
- Use deterministic fixtures for large and near-capacity owners, compact valid deltas, raw and derived expansion limits, exact composed capacity, absent versus uninspected files, malformed and provider responses, same-capability recovery with explicit authorization, stale input/branch, collisions and reciprocal mapping, immutable first publication, and localized/idempotent safe diagnostics. Preserve the full requirement and case inventory and the existing one-repair ceiling. Passing local fixtures or publishing a Draft is distinct from successful live end-to-end acceptance.
Metadata
Metadata
Assignees
Labels
Type
Projects
- Status
Activity
[-]Improve bounded drafting and companion recovery for large SDDs[/-][+]✨ - Improve bounded drafting and companion recovery for large SDDs[/+]vypbot commentedon Oct 28, 2026
Hi! I’m @vypbot, the Copilot assistant for this repository.
I can answer questions, explain the codebase, propose implementation and test plans, review issues and pull requests for potential bugs or security problems, and help authorized maintainers apply changes.
Try
/copilot helpto see the available commands, or mention @vypbot with your question.efraespada commentedon Nov 1, 2026
New released-SDK consumer occurrence on 2026-11-01: Domus recurrence issue #17, existing-owner update plan, run https://github.com/vypdev/domus/actions/runs/41338286158 . Safe blocked card: vypdev/domus#17 (comment) ; application-error reply 6562318884. Immutable SDK 3e7ceea, Codex CLI 0.166.0, selected base 63d1159f83f51ae0cd3c41f32341d9e9e36bb6da. Reference 5ef0e6bb-b231-462e-aa73-dd1d857dc5b0.
The card exposes reason=size, phase=response, draftAttempts=2, expectedCount=18000. No rejected candidate length/text is retained, so this does not establish the generated/assembled length or which representation caused it. The card labels this a complete-draft limit; unchanged original domain constants at this SDK distinguish SDD_MAX_UPDATE_CHARACTERS=18000 from SDD_MAX_DRAFT_CHARACTERS=70000. The actual plan was update, so the numerical update bound is consistent; please preserve stage/unit precision in recovery wording. We did not weaken either bound.
The preceding questions were ordinary source/policy/migration clarifications. The consumer supplied the complete independently verified public immutable source archive: 1,049,723 bytes, SHA256 3a89c9b8a60197015375e95f2d6ca3a83ec5daf774fe652ec8fc330b0b03a203, all 430 tracked blobs match tree f1e3aa8f9e8408076ca6c9740ace5391f8ac8c2b. Questions and source-view mismatch are separately tracked in #836; no claim is made that model tool access to that archive was observed.
We are applying the supported explicit maintainer recovery: approve specs/recurrence-execution-foundation.md as a compact companion under the existing recurrence-calendar owner, preserving the unchanged owner, original 37 cases/budgets and all launch requirements by reference. Target 10000..14000 draft characters before derived tables; no repeated owner content, new branch invented by a contributor, configuration change, provider-output exposure, limit increase or dropped scope. This is a new ordinary clarification, not an automatic rerun of the failed invocation. Companion publication success remains unproven until the Action verifies it.
Please retain this occurrence alongside the original reproducer and distinguish the observed size failure from its unproven generation root cause. The ordinary issue-assignment diagnostic failure in another run 41337490802 is separately reported as #849; it is not the SDD-size failure.
efraespada commentedon Nov 1, 2026
New released-SDK recurrence of the bounded update-drafting limitation, with precise safe diagnostics and consumer recovery input. This belongs to the existing proportional-appendix investigation; no request to weaken limits or bypass admission.
Consumer: vypdev/domus#20 . Immutable development base: 3131810082760a98ee4af66f11bb65eaac49cc3b; tree c18c57b60e20ccbc83d61464ab5fe623c8ed1713. The existing recurrence companion retains 52 cases and eight RCE requirements, alongside its original 37-case owner. Continuation follows explicitly partial foundation merges; actual schema remains 7 and full implementation remains required.
Observed terminal failures:
Runtime: vypdev/copilot@v3 resolved to 3e7ceea. Strict rejection and bounded repair worked. Unlike earlier diagnostic reports, the current safe response-stage/limit/attempt facts are useful and not claimed missing. The hidden candidate does not prove owner repetition, a validator defect or a particular model root cause.
Consumer recovery: after read-only inspection confirmed composeSddUpdate appends and reserves existing owner case IDs, we prepared a proper bounded 20-section delta outside the checkout. Original seven-module Git blob identities were reverified; unchanged original render/format/consistency/composition/catalog functions passed. All 17 owners remain unchanged. Raw delta 16142 characters; rendered 16772; composed owner 60969; four new independent storage boundaries, no reused owner case IDs; rendered SHA256 58d2994b7de4bcc2796246691fdf4c36a4437156d399bed23ad7a5b5a3045022. This proves a prospective proportional appendix fits, not that the model generated the same result or that Action publication succeeded. We supply that complete raw delta as substantive same-issue review input, then make one normal supported retry. Its outcome is still unproven.
A preliminary consumer-authored full replacement was correctly rejected locally for reserved case IDs. That was our wrong update wrapper, not an upstream defect; it is excluded from this report's failure claims. No Copilot source/resource/configuration, credential, limit, case budget or branch lifecycle was changed.
Please retain the existing investigation's distinct raw/assembled/composed limits, immutable owner reservation and explicit companion recovery. A deterministic regression should use this large retained owner plus a small valid delta; distinguish it from a duplicate full-owner replacement, truly oversized output and final composed-owner overflow. Do not accept our local validation as runtime or feature acceptance. The consumer continues through the ordinary gate with the complete remaining scope.
Update after that substantive recovery input: run 41349377261 attempt 2 also completed/failure, job 124979877519, with unchanged SDK pin and reference 87b09310-efea-44a4-8dd4-e6ca6d3b7b91. Draft query 21:11:49..21:12:34 UTC and its one repair 21:12:34..21:13:31 ended in the same response-stage size rejection. Current card still reports expected 18000 and attempts 2, without exposing candidate text or observed size. The independently fitting consumer input does not establish what the model actually generated.
The consumer now explicitly approves the existing supported companion recovery at specs/recurrence-storage-continuation.md under the same recurrence-calendar capability, preserving both current owners and all remaining scope. The proposed companion has original format/consistency checks passing against all 17 immutable owners, raw 16359/rendered 16989 characters under the unchanged 70000 new-document limit, four additional storage cases, SHA256 83b397e2c71b395167c95125b92c4a90ea5c8a47812354cd922ee1f2c5981428. Its actual Action publication is still unproven; no workaround branch or SDK modification is used. This is new ownership/route input, not another blind retry of the failed appendix.
Publication outcome: the next normal issue-edit run https://github.com/vypdev/domus/actions/runs/41351649126 completed successfully. The Action published specs/recurrence-storage-continuation.md at 0806a38f8a2de1f102a017bff25ad6856fb17bcd from exact development base 3131810082760a98ee4af66f11bb65eaac49cc3b, verified its linked branch, and applied branched/state:working. Native card: vypdev/domus#20 (comment) . The published companion is 36214 characters; implementation remains incomplete and its Draft status is retained.
The ordinary draft prompt includes the issue body and authorized numbered answers, not arbitrary proposed-draft comments. We verified the original gate file Git blob b64b3e70b4fc7df034f090a1f35a4a3ac801d2f9 at the same runtime pin. Therefore our earlier complete proposal in an ordinary comment was review input that was not actually included in that prompt. That is a consumer input-delivery mistake, not proof that the model ignored a supplied in-prompt draft. A prepared complete-body alternative was never applied because the notation clarification already succeeded. The actual hidden causes of rejected drafts remain unproven; the independent notation controls and observed bounded failures keep their stated scope. No SDK/resource/configuration/limit was changed. Full product/runtime acceptance is not inferred from publication.
New bounded-update failure on Domus administration continuation#21: run41449269130/job125291050392, pin3e7ceea08e6131360e026937d1c0cc1b120472e0, again exhausts one automatic size repair before any draft/branch publication. Diagnostic reference3d522f84-7e59-4462-8823-93dcdc402d48, validation.invalid-input, reason=size, phase=response, draftAttempts=2, expected count18,000; no actual rejected-candidate size or downloadable candidate is exposed. The owned status card6577800753 is updated in place, so its latest text may describe later states; the original size-card observation and terminal job log are privately retained.
The requested existing owner is specs/household-administration.md under household-identity at actual selected base4b71969d3b3f6df5f28d617ab0da4e86c9be5884. Its original file is57,838 UTF-8 bytes with72 existing case rows; that source-file byte count is not the rejected model-response size or the18,000-character limit. No claim is made that the hidden model copied the owner or which internal representation exceeded its bound.
The consumer clarified action=update and a proportional additive appendix, then answered the new source-snapshot question using exact immutable GitHub URLs/blob/SHA256 in comment6578016626. The answer requests an appendix near4,000..6,000 characters, preserving the original owner and all72 cases within both18,000 update and70,000 composed limits. No retained scope, numeric denominator or publication check is weakened, and arbitrary proposed-draft comments are not described as included in the prompt. The selected-base versus bootstrap-checkout question is tracked separately under#836; the initial registered-path companion decision under#856. The latest normal clarified invocation reached a valid ownership/question card, but actual SDD/branch publication is still unverified. No healthy queued job is restarted and no Copilot source/configuration/bound is changed.
Second actual Domus#21 update attempt after the explicit numbered snapshot/compact-appendix answer also fails: run41451007669/job125295917443, reference0b0a50bd-da34-434a-9ada-15341356d68d, validation.invalid-input, reason=size, phase=response, draftAttempts=2, expected18,000. The answer6578016626 was a substantive1897-character source/constraint input, including a requested appendix near4,000..6,000 characters and the70,000-character composed bound. This was not an unchanged-input rerun. The rejected text and actual response length remain unavailable, so no specific duplication/assembly cause is claimed.
Supported recovery is now explicitly selected in the same issue body: action=companion, capabilityId=household-identity, new unregistered path specs/household-administration-integration.md. Preserve the existing57,838-byte administration owner and identity owner verbatim, all eight HADM requirements and all72 original cases; do not duplicate/redefine IDs or count overlapping integration refinements as new independent product journeys. Target10,000..14,000 draft characters before derived tables within the unchanged70,000 bound. This is a real new ownership input after terminal failure, not a validator/bound change or fabricated branch.
The original pinned parseSddPlan control accepts this new companion tuple under the actual4b71969 catalog, while still rejecting companion creation at the already-registered original path; all7 original module bytes remain verified. That local control is not native publication/readiness. Normal events remain queued; no healthy run is restarted, no Copilot/consumer workflow is edited and no scope is dropped. New companion publication remains to verify.
efraespada commentedon Nov 2, 2026
Recovery outcome for the issue21 observations above: the Action has now published the new companion and catalogs at a9c4b9723c66868884476743940102ada65d2db0 on the exact managed feature/21-complete-the-verified-household-administration-workspace-after-the-profile-model-foundation branch, with reviewed parent4b71969d3b3f6df5f28d617ab0da4e86c9be5884. The native published gate and branched readiness were verified before implementation. The original registered owner and72 case rows remain byte-identical. This recovered through the supported new-companion route after providing a closed complete proposal in the issue body; it does not establish a correction to the earlier update-size, ownership-route or selected-snapshot defects. No Copilot code, policy, limits, workflow or credentials changed. The intervening assembly mapping diagnostic is recorded in #833. Product implementation/acceptance remains outstanding.
efraespada commentedon Nov 2, 2026
Additional terminal size rejection on a new coherent implementation continuation, using the same released runtime rather than a newer upstream fix.
Supported recovery is being prepared on this same issue: a new unregistered bounded workspace companion under existing household-identity, preserving all owner documents/cases and schema9, with a closed complete proposal checked against the unchanged seven original SDK policy modules. No Copilot source/resource/configuration/workflow/credential/size bound will change, no feature inventory will shrink, and no contributor branch will be invented. Healthy queued runs41463524353/41463683825 are not restarted or canceled. Local proposal checks remain separate from native publication and branch readiness.
47 remaining items
Merge pull request #906 from vypdev/feature/763-improve-bounded-draft…