Repository navigation
🐛🧑💻 - Progress assessment rejects a 39 file comparison at a fixed evidence budget without capacity or continuation facts #851
Description
Is there an existing issue for this?
- I have searched the existing issues.
Searches for validation.evidence-too-large and the exact fixed-budget message found closed #773 (complete large comparison recovery) and closed #547 (incorrect outage classification). This is a new released-runtime occurrence on a smaller comparison after those changes. Please consolidate with the appropriate owner without losing the immutable source identity. The current diagnostic correctly rejects incomplete evidence and says retryable No; the outage-classification issue is not claimed recurring.
Which actions are affected?
Commits
Which platforms are affected?
Linux
Description
Progress assessment of a coherent 39-file recurrence storage/CI increment stops at a fixed analysis budget with validation.evidence-too-large. The native message asks the maintainer to inspect the reported capacity, but the visible failure supplies no actual/maximum count, exceeded dimension, uncovered-path manifest or completed multipart facts. Existing progress is correctly preserved. Please investigate the applicable capacity and supported complete continuation; do not raise unchecked limits, truncate scope or publish a partial assessment as complete.
Consumer vypdev/domus#22 , issue vypdev/domus#20 . Exact head f835775b2f230db956750a224f8ebd6cc8bbbb5f. The push before SHA is 0806a38f8a2de1f102a017bff25ad6856fb17bcd; native comparison metadata separately pins the actual assessment base to develop 3131810082760a98ee4af66f11bb65eaac49cc3b. Do not substitute the event before SHA for that assessment base.
The comparison contains the admitted companion/catalog and scoped SQLite storage, tests, migration documentation and current-source image checks. Native Bugbot metadata independently reports 39 changed files and 2536 changed lines. These review counts do not establish progress evidence coverage or identify the dimension that exceeded its limit.
Reproducing the issue
- Inspect https://github.com/vypdev/domus/actions/runs/41357362161/job/125000203225 . Runtime resolves vypdev/copilot@v3 to 3e7ceea.
- At 22:30:09.287 UTC the log records a pull-request comparison for PR 22, base 3131810 and source head f835775. At 22:30:09.767 it selects a bounded GitHub comparison; at22:30:09.769 it rejects a fixed analysis budget. No rejected provider output or native progress part manifest is available in this job output; no specific model cause, file-recovery exhaustion count or attempted multipart count is inferred.
- Native Copilot / Verification 125000453447 binds to the exact head and concludes failure: lifecycle blocked, validation.evidence-too-large, reference 3a6e91b7-6ebb-4327-9538-cc3b9c169975, Retryable No. Existing labels/assessments are retained. The safe summary asks to inspect the reported capacity and says supported multipart recovery continues automatically, but no capacity values or continuation handle are shown.
- Separate current-source CI run 41357396927 proves backend tests and actual amd64/aarch64 images (23 executions each) pass. Overall CI is not claimed green: Python tooling found our stale old-image contract assertion, which is being corrected locally (148 tests/100% coverage now pass). That consumer test failure is not blamed on Copilot and does not explain the independent comparison-budget rejection.
- Local exact-source backend verification passes 3500 executions, 122 authored modules, 8706 statements/2482 branches at 100% globally/per module. The full retained 52 companion/37 owner cases remain requirements; no completed full recurrence feature is claimed.
copilot Version
vypdev/copilot@v3 resolved to immutable 3e7ceea in the observed job. No unverified package version is inferred. The failure occurs before any progress model result is logged. Separate Bugbot review uses codex-cli 0.166.0 / gpt-6-luna smart and completes 4/4 code partitions for 39 assigned files, but is partial for a human-conversation dimension (#843); that is not progress evidence or a fully complete review.
Relevant Log Output
2026-11-01T22:30:09.2872580Z Progress comparison source (native metadata): baseSha 3131810082760a98ee4af66f11bb65eaac49cc3b, sourceHeadSha f835775b2f230db956750a224f8ebd6cc8bbbb5f, PR 22
2026-11-01T22:30:09.7674621Z Progress will be assessed from a bounded GitHub comparison pinned to the base and source heads.
2026-11-01T22:30:09.7698822Z Progress evidence exceeds a fixed analysis budget. Existing progress was retained; retrying the same comparison cannot resolve this limit.
Error code: validation.evidence-too-large
Reference: 3a6e91b7-6ebb-4327-9538-cc3b9c169975
The first log line above is a compact field transcription; the original native JSON remains in the linked job. No secrets, source body or rejected candidate is disclosed.
Additional context and comments
The consumer is continuing the full implementation and fixing its own test independently. No identical over-limit recheck is requested. No Copilot source, installed resources/configuration, credential, bound, model or managed branch lifecycle is modified. Future changed-head runs are normal contributor verification, not a claimed capacity workaround.
Please preserve fail-closed complete evidence and add safe closed dimension/actual/maximum/phase facts when available, immutable required/reviewed-path identity and a supported resumable continuation or explicit bounded incapacity result. Cover both successful complete multipart assessment and unavailable/corrupt/moved-head/over-budget rejection. This is an observed native failure, not a deterministic proof of a particular internal cause or a request to reopen a closed issue automatically.
Metadata
Metadata
Assignees
Labels
Type
Projects
- Status
Activity
[-]Progress assessment rejects a 39-file comparison at a fixed evidence budget without capacity or continuation facts[/-][+]🐛 - Progress assessment rejects a 39 file comparison at a fixed evidence budget without capacity or continuation facts[/+]vypbot commentedon Nov 1, 2026
Hi! I’m @vypbot, the Copilot assistant for this repository.
I can answer questions, explain the codebase, propose implementation and test plans, review issues and pull requests for potential bugs or security problems, and help authorized maintainers apply changes.
Try
/copilot helpto see the available commands, or mention @vypbot with your question.efraespada commentedon Nov 1, 2026
A second normal source push reproduces the same complete-comparison progress rejection on Domus PR 22, head 71c54ab92cc099d65fa6681c1a09ec61fc5656c1, without an unchanged-input rerun.
This push fixes our stale tooling assertion so CI tests the actual checked-out image as well as the frozen schema-7 control; no Copilot source, scope, policy, bound or credential changed. Python tooling and backend checks pass, as do both current-source CPU image jobs. Product frontend CI separately exposed a real storage_busy response in the simultaneous-setup browser test; we are checking its bounded contention and exact-request/current-authority recovery rather than treating that product test failure as a Copilot defect.
Native review at this head completed 4/4 code partitions and all 40 assigned files with zero candidate/published findings, while the separately reported human-conversation dimension remains partial under issue 843. That is not a successful progress assessment or a claim of complete product delivery. The comparison error still supplies no actual budget dimension/count/limit or durable multipart/continuation facts in its public diagnostic. We retain the real inputs and continue the unfinished feature; no evidence is trimmed to obtain a passing assessment.
Follow-up after the next normal implementation push, head 9c12953232c0fb967663c1853e5dc48112379623: Copilot Commit run 41361524486 / job 125012715508 and Verification check 125013082112 reproduce validation.evidence-too-large, reference 894d37b9-1732-4cf7-8237-19e46946ab5e, Retryable: No. The maintained base is still 3131810082760a98ee4af66f11bb65eaac49cc3b; no Copilot setting, bound or code changed. Native review independently completes 5/5 code partitions,57 diff fragments and52 assigned files with0 new candidate/published findings and the prior real product finding fixed (not dismissed). The separate human-conversation limitation persists. The product frontend, backend and architecture/tooling checks pass; full local backend 3545 tests/100% supported coverage pass. Actual image jobs are progressing normally and are not restarted. This is another normal source revision, not an unchanged-input retry; no complete-comparison capacity/continuation facts appeared in the progress diagnostic.
Next normal source push, c5819b8ffd217bde96b440a46971e6df77867099: Commit run 41363414573 / job 125018432742, Verification check 125018761670, again rejects with validation.evidence-too-large, reference e5d87e1e-56a3-4bfb-9d4a-0bf63bb91ca3. Native PR review independently completes all 6 partitions, 63 fragments and 56 assigned files, with zero candidate/published/open findings and the prior finding fixed. The full local backend campaign passes 3,584 tests with 100% supported statement/branch coverage globally and per module. Product CI is progressing normally; no queued/running job is restarted. No Copilot bound, source, setting or evidence scope changed, and the diagnostic still provides no concrete capacity/continuation accounting.
Next normal implementation push, ae73710972c386b18406921316d6311ced0d76e6: Copilot Commit job 125033805094 / Verification check 125034168280 rejects validation.evidence-too-large, reference a4cc1a7a-431e-4887-8d81-4454f17a919f. The diagnostic reports 64 required paths and 2 source parts but still no concrete failing capacity dimension/count/limit or recoverable continuation. Native PR job 125033819503 / Review check 125034343268 completes all 7 partitions, 74 fragments and 64 assigned files with zero candidate/published/open findings; the prior finding remains fixed. Every applicable product CI Check job, including both actual CPU images, passed (aggregate125035252685). Local backend3639 tests and 100% supported global/per-module coverage passed. No unchanged-input retry, omitted evidence, scope reduction or Copilot modification was used.
Next normal source revision, 872bdba55dc7fa9ec942a9f0ff27a635b62d0169: Commit run41414939116/job125176389510 and Verification check125176958964 again reject validation.evidence-too-large, reference95884a3e-20f2-476b-972f-4c5fa710e671, Retryable:No. The diagnostic reports71 required paths,2 source parts and recovery usage null, but no concrete failing capacity dimension/count/limit or durable continuation accounting. Independent native PR run41414946003/job125176414646, Review check125177712439, completes all8 partitions,82 fragments and71 assigned files. It publishes one valid product finding (noncanonical stored JSON), with the previous finding fixed; this is a real consumer defect being corrected, not a Copilot finding defect. Local3667 tests/100% coverage, hosted backend and both current-source CPU images pass. The frontend check separately exposed a real concurrent read failure and is being corrected. We do not restart unchanged progress input or reduce scope/evidence to evade capacity. No Copilot source/configuration/bound/credential changed.
Next normal correction push, abf6403b569c39101df2001eb90ad013c40972c1: Commit run41417012935/job125183242869 and Verification check125183799672 repeat validation.evidence-too-large, reference32ac3fec-0655-4f07-8dd2-825afe7d6368, Retryable:No. It reports75 required paths and2 source parts; recovery usage remains null and the concrete failing dimension/count/limit or continuation facts are still absent. This is actual new source correcting the valid canonical-storage product finding and independently reproduced inspection lock classification; it is not an unchanged-input rerun. Local3707 tests/100% supported global and per-module coverage,49 browser journeys and52 package/architecture tests pass. Hosted frontend125183275824 also passes; backend, images and fresh native review are running normally and are not restarted. The finding remains pending native verification rather than dismissed. No Copilot resource, configuration, bound, scope or credential changed.
Native review follow-up at abf6403b569c39101df2001eb90ad013c40972c1: PR run41417022777/job125183276463 / Review check125184665794 completes8/8 partitions,86 fragments and75 assigned files with0 candidate/published/open/reopened findings, fixed=2, dismissed=0, unknown=0 and verification-required=0. Both real product findings are confirmed fixed and their threads resolved. Overall review remains partial/neutral only because of the separately reported human-conversation dimension (3 fetched/2 retained/1 omitted/limitReached). The two original proposed-SDD technical inputs and both finding-fix replies were manually inspected; they are agent-authored maintainer-account material, not newly confirmed owner product answers. Hosted backend125183275937, frontend125183275824 and aarch64 image125183275906 pass; amd64 remains a normal live job. The progress-capacity failure is independently unresolved; neither code findings nor review scope were hidden to obtain a progress pass.
Normal relative-planning push 05951170603132fa17db92b49b1174d4fefa2708: Commit run41421429379/job125197634018 and Verification125198304206 reject validation.evidence-too-large, reference18107c69-3e73-42dd-9d71-09238a19dbff, Retryable:No. Complete progress evidence now reports84 required paths and3 source parts, with recovery usage null; no concrete failing dimension/count/limit or durable continuation accounting is exposed. Native PR run41421440802/job125197670541 / Review125198991451 completes9/9 partitions,95 fragments and84 files. It publishes one valid consumer finding about cursor position/count consistency; the prior two remain fixed. This is being corrected with original-source regression, not dismissed or classified as a Copilot defect. Local3783 tests/100% supported coverage and hosted backend/frontend pass. The comparison is new actual source, not an unchanged-input retry; no scope/evidence reduction or Copilot change was used.
Normal cursor-integrity revision b2bcc693d6be5065b1be8dc378334199c927735a: Commit run41425766676/job125211869190 and Verification check125212526819 repeat validation.evidence-too-large, reference3307c393-b9fc-48af-86ab-be9ff0ff49e0, Retryable:No. Complete evidence reports86 required paths,3 source parts and recovery usage null, without a concrete failing capacity dimension/count/limit or durable continuation accounting. Native PR run41425773980/job125211895989 / Review125213593582 independently completes10/10 partitions,97 fragments and86 files; it publishes two valid UTC-UNTIL boundary findings, with the prior three fixed. These consumer defects are being corrected, not dismissed or reported as Copilot defects. Hosted backend/frontend and both actual CPU images pass; aggregate productCI separately fails on an owned E501 smoke-test line, which is also being corrected. The local3810-test100% coverage campaign preceded that late smoke edit and does not imply a passing final style check. This is new source, with no unchanged-input retry, omitted evidence, reduced scope or Copilot modification.
Normal UTC-proof correction a6ba0263551e2d19ba07574f46bdbd7c6b9b1532: Commit run41432250925/job125233427034 and Verification125234040111 again reject validation.evidence-too-large, reference79b6fa0b-7146-40aa-a5e1-3bdb9060a17f, Retryable:No. Evidence reports91 required paths,3 source parts and recovery usage null; no concrete failing dimension/count/limit or durable continuation facts are supplied. Native PR run41432261833/job125233464764 completes11/11 partitions,109 fragments and91 files with0 candidate/published/open/reopened findings,5 fixed,0 dismissed/unknown/verification-required. Both valid UTC consumer findings are confirmed fixed and all threads resolved. Local3881 tests/100% global/per-module coverage, hosted backend and both actual CPU images pass. ProductCI still fails on an independently retained Playwright CDP response-body read failure (frontend job125233464397), not on Copilot or the corrected product boundary. Its test-only actual-upstream observation repair passes49 local browser journeys and awaits a normal new-source CI run. No unchanged-input retry, scope/evidence reduction or Copilot modification is used.
Normal browser-test correction 06b3b9f38955a25be2511b68b2ea667a84a42d8f: Commit run41433381636/job125237209349 again rejects validation.evidence-too-large, reference14d74cb5-f4d2-4736-9b92-a2f870fcb033, Retryable:No. Evidence remains91 required paths,3 source parts,recovery usage null, without concrete failing capacity dimension/count/limit or continuation accounting. Every applicable productCI suite now passes, including backend125237234578, frontend125237235059, both actual CPU images125237235192/125237235352 and aggregate125239685933. Native PR run41433389688/job125237236014 completes11/11 partitions,110 fragments and91 files with one valid new historical-slot original-generation finding and the prior five fixed. This consumer defect is being corrected with real original-receipt regressions; it is not dismissed or reported as a Copilot defect. No unchanged-input rerun, scope reduction, hidden finding or Copilot modification was used.
Original-slot correction8b0aafda1a4cd8d1d63bf782b04f2709fb7db46c: Commit run41435787257/job125245097895 and Verification125245735766 still reject validation.evidence-too-large, referencebb80b30f-fb77-4df7-bd60-e5ad40b2d7e2, Retryable:No. Evidence reports93 required paths,3 source parts and recovery usage null, without the concrete failing capacity dimension/count/limit or durable continuation facts. Independent native PR run41435797183/job125245131294 / Review125246682817 completes12/12 partitions,112 fragments and93 files with0 candidate/published/open/reopened findings,6 fixed and0 dismissed/unknown/verification-required. All real product findings are confirmed fixed. Every applicable current-source productCI suite passes, including backend125245130812, frontend125245131300, actual amd64/aarch64 images125245131421/125245131521 and aggregate125247642850. Local3909 tests/100% supported global/per-module coverage,52 fresh-package/architecture tests, original SDK checks and exact image-host controls pass. No unchanged-input retry, reduced scope, concealed finding or Copilot modification was used. Progress capacity remains a distinct unresolved upstream defect; complete retained recurrence execution remains in progress.
Normal relative-storage source19ff35cb2cb2ceb4a0397901341f1089001cc619: Commit run41442803362/job125268556491 / Verification125269206057 again rejects validation.evidence-too-large, referencee818e4f7-749a-4646-9148-c8f52e646bd1, Retryable:No. Complete evidence now reports102 required paths,4 source parts and recovery usage null, without concrete failing dimension/count/limit or durable continuation facts. This is real new source, not an unchanged-input retry. All applicable productCI suites pass (run41442811569; backend125268583180, frontend125268583381, actual amd64/aarch64125268583406/125268583447, aggregate125271140255). Full local backend3987 tests/100% supported global/per-module coverage across136 modules pass.
Independent native PR run41442811542/job125268584741 / Review125270397194 completes14/14 partitions,128 fragments and102 files. It initially publishes one demonstrably false double-encoding finding, with the prior6 valid consumer findings fixed. Exact unchanged-source reproduction proves the nested value is a dict and all4 variants round-trip; the exact suggested cast breaks all4. Existing6 targeted actual codec/storage/restore cases pass. That separate review defect is reported in #855, with an evidence-based reply at vypdev/domus#22 (comment) and manual thread resolution. Native initial telemetry open=1 is not relabeled as native-confirmed fixed; no production/Copilot patch was made to satisfy it. Human preflight remains partial8 fetched/6 retained/2 omitted/0 truncated/limitReached; separate dimensions complete. No evidence/scope reduction, unchanged-input rerun or Copilot source/configuration/credential/bound change was used. Full retained issue20 implementation remains in progress.
Normal current-authorized relative-creation revision ed7cd14190d750846833cf1bb822988b628560da: Commit https://github.com/vypdev/domus/actions/runs/41445903600/job/125278785687 / Verification125279436027 again rejects validation.evidence-too-large, reference2e271fe7-5bf7-4341-bb09-55e8bbe633b9, Retryable:No. It reports105 required paths,4 source parts and recovery usage null; concrete failing capacity dimension/count/limit and durable continuation accounting remain absent. Full local4031-test/100% supported global and per137-module coverage,44 actual creation/permission/rollback/concurrency/restore controls and52 package/architecture tests pass. Every applicable current-head productCI suite passes (run41445912990; backend125278815870,frontend125278815895,actual amd64/aarch64 images125278816046/125278815943,aggregate125281291351).
Independent native PR run41445912711/job125278815180 / Review125280665009 completes15/15 partitions,133 fragments and105 files with0 candidate/published/open/reopened findings,6 fixed and1 dismissed. The one dismissal is the previously reproduced false-positive codec report#855, not dismissal of a valid defect; all6 valid prior consumer fixes remain confirmed. Its codec/task-codec/JsonDocument module bytes remain exactly equal to19ff, with no production fix. Human preflight remains partial9 fetched/7 retained/2 omitted/0 truncated/limitReached, with separate issue6/6,PR14/14,threads7/7,diff105/105 complete. All9 known agent-authored technical inputs are manually inspected with provenance retained. This is a real new feature revision, not an unchanged-input retry. No Copilot modification, evidence/scope reduction or concealed finding is used. Full retained recurrence execution remains in progress.
[-]🐛 - Progress assessment rejects a 39 file comparison at a fixed evidence budget without capacity or continuation facts[/-][+]🐛🧑💻 - Progress assessment rejects a 39 file comparison at a fixed evidence budget without capacity or continuation facts[/+]vypbot commentedon Nov 4, 2026
Implementation plan
Acceptance: A pinned comparison exceeding one analysis budget either resumes across bounded parts and produces a complete assessment covering every required path, or fails closed while retaining existing progress and reporting safe dimension, actual/maximum, phase, and uncovered-path facts when available.
Need something else? Mention the bot with a question or use
/copilot help.