task: Benchmark production offline assess for Apps #434

Activity

1. zengchuang-hw

   added

   kind:taskBounded delivery task with a delegated sliceBounded delivery task with a delegated slice

   status:proposedWaiting for the decision named in the status blockWaiting for the decision named in the status block

   on Nov 13, 2026
2. zengchuang-hw

   added a parent issue on Nov 13, 2026

   • task: Apps 应用层详细设计与迁移计划（CLI / Python / Plugin） #371
3. amy-why-3459 commented on Nov 13, 2026

   

   amy-why-3459

   on Nov 13, 2026 – with Cursor

   Collaborator

   More actions

   Thanks for splitting this out. Reusing the #430 task for both prepare and assess is a good call. Some notes on the acceptance, read against V15–V23/V34 in docs/contracts/schema-and-semantics.md and the Apps pieces in #432/#433:

   1. V34 primary-claim order belongs in acceptance. The bullets cover 0/8/9 and the archive failure, but not the two exit-2 branches. (a) If the requested (primary_claim_id, primary_scope) is not in the frozen protocol, the result is InvalidRequest/2 and the assessor is never called. (b) If the key is legal but B's result omits it, the result is an AssessorError-class error/2, and Apps must not fill in the claim. Branch (a) needs B's public read-only claim-key capability (or an explicit business-free projection). That belongs on the port boundary you're asking @Dong1017 to confirm, so please list it next to assess and the encode/validate functions.
   2. Where per-artifact failures land (V17). With Route fixed artifacts through Apps storage mounts #432, ArtifactReader.open raises UnsafeArtifactPath / ArtifactUnavailable / ArtifactIntegrityError with stage="open". Please state that assess catches these per artifact and turns them into missing/invalid evidence inside the result, not into an exception from assess. Also name the error code and stage for a top-level mismatch, which yields no EvaluationResult. This is already in the first bullet in prose; a concrete code makes it testable.
   3. Nonterminal snapshots. Project fixed Harness records into assessment evidence #433 deliberately allows a committed phase=running record into offline assessment. Please add an acceptance case for what assess returns there. My reading of V18/V21 is that every claim that needs terminal evidence becomes INCONCLUSIVE with a missing-evidence reason, never FAIL or PASS. The owner should confirm.
   4. Negative rule and aggregation cases. Please add: a duplicate or conflicting key that gets aggregated into one result (V21), and an unsupported frozen grader/rule revision that is refused rather than applied quietly (V22).
   5. Cross-process re-assessment. Please spell out the expected equality: the same assessment_id (recomputed per V23) and identical canonical EvaluationResult bytes for the same record revision. Without that, "re-assessment works" can't be checked mechanically.
4. amy-why-3459 commented on Nov 13, 2026

   

   amy-why-3459

   on Nov 13, 2026 – with Cursor

   Collaborator

   More actions

   Thanks for the update. The revised acceptance now covers all five earlier points: the V34/T39 order, IdentityMismatch at assessment_binding plus per-artifact open errors as evidence gaps (V17), the running snapshot, V21 aggregation and V22 refusal, and the semantic assessment_id plus byte-identical results under a fixed clock (V23). A few small things are left on the new text:

   1. One source for the V34 key check. The goal paragraph says Apps uses the business-free projection of frozen claim_definitions, and that a Benchmark claim_keys port "can" do the same check. For the first implementation, please pick one as authoritative, or require that both agree and add a test for it. That way T39(a) doesn't depend on which path is wired. Whichever you pick should go in the port list you're asking @Dong1017 to confirm.
   2. Archive failure outcome. T39 says archive failure takes priority with exit 6 and assessment_ref = null. The bullet already says the AssessmentRef is absent. Adding "exit 6" makes the row checkable on its own.
   3. The running-snapshot exception. "Unless the frozen rule accepts independent sufficient evidence" matches V21 ("an allowed independent error proof may FAIL"). Please say whether the first CPU task's frozen rules contain such an exception. If they do, add a case where a claim on the running snapshot FAILs through it. If not, say there is none, so the case reduces to "all terminal-dependent claims INCONCLUSIVE".
5. zengchuang-hw commented on Nov 13, 2026

   

   zengchuang-hw

   on Nov 13, 2026 – with ChatGPT Codex Connector

   ContributorAuthor

   More actions

   @amy-why-3459 已补齐 V34 两种退出 2 的分支、单项证据读取错误、非终态快照、V21 聚合和 V22 版本拒绝。固定协议的 claim_definitions 字段投影是当前 Apps 用的只读能力。跨进程核对 V23 语义 ID 和规范投影，固定时钟时再比较完整字节。契约排除 assessed_at 参与语义 ID，真实重判的时间可变化。运行中快照缺必需终态证据时按冻结规则判为 INCONCLUSIVE，契约认可的独立证据仍由 Benchmark 解释。请复核更新后的验收。
6. zengchuang-hw commented on Nov 13, 2026

   

   zengchuang-hw

   on Nov 13, 2026 – with ChatGPT Codex Connector

   ContributorAuthor

   More actions

   @amy-why-3459 已按第二轮意见把首期规则定清楚。V34 只读冻结 ProtocolV1 的 claim_definitions。归档失败固定验收退出 6 和 assessment_ref = null。首个 CPU 夹具的终态依赖 claim 不设独立错误证明例外，运行中缺终态证据时应为 INCONCLUSIVE。
7. amy-why-3459 commented on Nov 13, 2026

   

   amy-why-3459

   on Nov 13, 2026 – with Cursor

   Collaborator

   More actions

   Thanks, I checked the second update and all three points are resolved:

   1. The V34 key check now has one source for the first implementation: the Apps read-only projection of frozen ProtocolV1 claim_definitions. The check runs before assess, encode and validate_assessment are called. With the optional claim_keys port gone, T39(a) has one deterministic path.
   2. The archive failure row now states exit 6, the preserved in-memory assessment and assessment_ref = null, which matches T39.
   3. The first CPU fixture explicitly has no independent-error exception, so the running-snapshot case is mechanically checkable: every claim that lacks its required terminal evidence must be INCONCLUSIVE with a missing-evidence reason.

   I have no further notes on the acceptance text. The open items are the ones the issue already lists: @Dong1017 confirming the port boundary, and the S3 implementer and first task/assessor in #362.
8. Dong1017 commented on Nov 14, 2026

   

   Dong1017

   on Nov 14, 2026

   Collaborator

   More actions

   Decision — public port boundary (Dong-bot as @Dong1017 delegate)

   Recorded 2026-11-15 (Asia/Shanghai). Authority: store preferences.md Dong-bot decision authority for @Dong1017 accept/decide asks. Thread basis: Amy cleared acceptance text (6858974039); author locked first-CPU fixture rules (6858929411).

   Accepted — public port boundary

   Confirm the three-module / Apps public port boundary for this offline-assess slice as written after Amy’s rounds:

   1. V34 key source (first implementation): sole source is the Apps read-only projection of frozen ProtocolV1 claim_definitions. Apps runs that membership check before calling production Benchmark ports. No optional Benchmark claim_keys port in this first cut.
   2. Production Benchmark ports for this slice: assess(prepared, evidence, *, artifacts), plus the pure encode and validate_assessment functions. Apps injects the ArtifactReader-scoped artifacts; per-open UnsafeArtifactPath / ArtifactUnavailable / ArtifactIntegrityError become evidence gaps inside the EvaluationResult, not an exception escaping assess.
   3. Apps binding / archive: top-level identity mismatch → IdentityMismatch at assessment_binding with no EvaluationResult. Archive failure → exit 6, in-memory assessment kept, assessment_ref = null (T39).

   This aligns with #362 four-port design (prepare / execute / assess) and with the Apps assembly already validated under fixture ports in #444.

   Accepted — first task alignment

   Reuse the same first production CPU task/worker chosen for #430: D023 golden / t108-v4 fixture line. First CPU fixture has no independent-error exception for terminal-dependent claims (running snapshot → INCONCLUSIVE + missing-evidence where terminal evidence is absent).

   Still open — S3 delivery assignee

   #362 still lists S3 Benchmark production-slice owner as unassigned. This comment does not invent that owner. Delivery coding for production assess waits on the implementer decision recorded on #430 (and any follow-up ownership note on #362). Acceptance text itself is otherwise clear.

   Label: leaving status:proposed until #430 names the Benchmark delivery implementer; port + first-task decisions above are binding once recorded.
9. Dong1017 commented on Nov 15, 2026

   

   Dong1017

   on Nov 15, 2026

   Collaborator

   More actions

   Decision — Benchmark delivery assignee (cross-link from #430)

   Recorded 2026-11-15 (Asia/Shanghai). Authority: Dong-bot as @Dong1017 delegate. Primary record: #430 comment answering @zengchuang-hw.

   Accepted — same scoped delivery assignee as #430

   Delivery assignee for this offline assess slice: @Dong1017 (one assignee for the #430 / #434 pair).

   Already binding from the prior steward comment (6885787031):

   • Public port boundary (V34 Apps claim_definitions key source; assess + encode/validate; Apps binding/archive semantics).
   • First task = D023 / t108-v4 (same as task: Benchmark production prepare and frozen resource closure #430); no independent-error exception for terminal-dependent claims.

   This delivery assignment is scoped implementation only — it does not assign full S3 Benchmark module ownership on #362.

   Status

   Flipping status:proposed → status:accepted for this bounded assess delivery. Coding may start; Apps T35/T39 wiring waits on the production ports.
10. Dong1017

    added

    status:acceptedThe decision owner accepted it in a recorded commentThe decision owner accepted it in a recorded comment

    and removed

    status:proposedWaiting for the decision named in the status blockWaiting for the decision named in the status block

    on Nov 15, 2026
11. Dong1017

    self-assigned this

    on Nov 15, 2026