You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Which trust measures can this repository compute today? #15668
Which trust measures can this repository compute today from its own data: revert rate, the share of pull requests merged with no human touch, and corrections converted to checks?
From the talk
Chapter 1 and chapter 9, slides 3 and 34-38: quality and quantity together; her results shown as a scoreboard. Her own numbers are not targets.
Today and the gap
Pieces exist (/work-items:measure-delivery, /work-items:morning-brief); no single view, and no one has checked which of the three are computable.
What resolves it
For each measure: the data source, whether it is computable now, and the definition used. Evidence from this repository's history and the delivery-metrics literature. Basis: menu priority P1.
Done would add
A finding that sharpens the map's fog on trust measures, and, if computable, a view in an existing skill. Dogfood: the measures computed for this repository.
Size: Medium
Complexity: Medium
Uncertainty: Knowable
Flags: none
Blocked by: none
Understood: a research question asking which of three trust measures (revert rate, share of PRs merged with no human touch, corrections converted to checks) this repository can compute from its own data. For each measure the finding should give the data source, whether it is computable now, and the definition used, with evidence from this repository's history and the delivery-metrics literature. If a measure is computable, an existing skill gains a view.
The scope is clear enough to start. Reading the repository history and the existing measure-delivery and morning-brief skills should settle it.
Related, not a duplicate: #15650 covers counting corrections that became checks, which is only the third measure here.
Not admitted to the issue lane yet: no trusted-human acceptance criteria: a listed human must put the confirmed - [ ] criteria in the body or post them as their own comment, or, on an issue a listed human filed, intake's newest refinement comment must hold them unedited; intake's refinement of this body is due and may supply them.
Resolved: of the three trust measures, revert rate is computable today, the no-human-touch merge share is computable only as a lower bound, and corrections converted to checks is not computable because nothing tracked records the conversion. Basis: the research below. A fresh verifier confirmed the definitions and the arithmetic and lowered the confidence of the computed values; the caveats are stated per measure.
From the talk
Slide 37 (chapter 9, Results) shows revert rate as "% of commits reverted" per month; the speaker says the number of pull requests that get reverted is steady at about half a percent. Her numbers are not targets (the map's Out-of-scope).
Slides 3 and 33 to 38 show pull request volume, release counts, contributors per month, revert rate and app performance. None shows a share of pull requests merged with no human touch; that measure is ours, not hers.
Slide 23 (chapter 6, Corrections) shows a rising step line headed "whenever you correct your agent", with the footer "lint against it or categorically eliminate the problem" and no axis label. It is the talk's only picture of corrections becoming checks, and it gives no definition.
These three points were read from our digest of the talk (slide descriptions and transcript), not from a fresh viewing of the video.
Per measure
Measure
Data source
Computable now
Definition used
Value for this repository
Revert rate
git history of main, through /work-items:measure-delivery
Yes
Landings whose subject is a revert, over all landings
13 of 7652 landings from 2026-08-20 to 2026-11-11, 0.17%
No-human-touch merge share
GitHub pull request data (author, merger, reviews, comments)
Lower bound only
Merged pull requests with no human review, comment, approval, merge or commit, over merged pull requests
About 61% over a 2.3-day window: 305 of 500 by one read, 315 of 511 by a second
Corrections converted to checks
None tracked
No
Proven /corrections:codify proposals whose check was later installed, over all proposals
None
Revert rate
DORA defines change fail rate as the ratio of deployments that require immediate intervention following a deployment, and deployment rework rate as the ratio of deployments that are unplanned but happen as a result of an incident in production (page last updated January 5, 2026).
This repository has no incident record, so the value is a revert-based stand-in for DORA's rework rate, not the DORA measure. plugins/work-items/skills/measure-delivery/SKILL.md:46-69 states that the skill counts each first-parent commit on the default branch as a deployment and detects reverts from git history alone.
Confidence: medium. The 13 reverts were matched one for one between git and GitHub's merged pull requests; the 7652 denominator comes from git alone.
Not settled: the skill's script resolves 6 of the 13 reverts to an earlier landing and leaves 7 unresolved (one measurement only).
No-human-touch merge share
The nearest published definition is Jellyfish's agentic-merged pull requests: merged pull requests whose only review activity came from AI tools. It covers review only and says nothing about who merged, so the wider definition in the table is ours.
It is a lower bound because agent sessions act under the owner's login: plugins/work-items/skills/morning-brief/SKILL.md:244 states that its owner-touch count includes any agent session holding that login. A pull request the owner's login touched cannot be split into human and agent action by login alone.
An indication, not a join: in the same window 198 of 510 commits on main carry a Claude co-author trailer, against 191 of the 500 merged pull requests authored by the owner's login.
What a full measure needs: one per-pull-request signal that separates the owner's own actions from an agent session's (the co-author trailer, a session trailer or a label).
Corrections converted to checks
plugins/corrections/skills/codify/SKILL.md:153-164,181 states that codify writes proposals and notes under the gitignored .work/codify directory; it records no tracked link from a correction to an installed check.
On the one machine read, the corrections findings buffer returned no rows and the checkout has no .work/codify directory.
No published ratio for this measure was found (checked: one web search, which returned a conference talk page, an arXiv paper measuring rule recurrence and a pattern-book page on first-pass acceptance; not checked: the analytics docs of DX, Jellyfish and LinearB).
Revert rate needs no new work: /work-items:measure-delivery already reports it.
The other two cannot be shown honestly until their missing record exists, so a single trust view is not worth building yet. The two records are the open questions: the per-pull-request human-or-agent signal, and the correction-to-check link (already owned by the work item above).
Verification
A fresh verifier re-fetched the DORA and Jellyfish pages and the three skill files and confirmed each quoted definition and line. After one rework round it still failed three rows:
the revert-rate value rests on two independent stores, not the three a high-confidence value needs, so it is stated here as medium;
the three statements about what our skills do are confirmed as what their SKILL.md files state, not as observed run-time behavior, and are worded that way here;
the research index cited the Jellyfish definition for the whole share; this comment limits it to review activity.
The full research files and the raw command output go on the map's dossier branch.
Done (2026-11-12): Resolved; see the resolution comment: revert rate is computable now (13 of 7652 landings, 0.17%), the no-human-touch share only as a lower bound, corrections converted to checks not until a tracked record exists.
Owner decision (2026-11-12, "Let's go with recommendations"): the per-pull-request signal that separates the owner's own actions from an agent session's is the Claude co-author trailer on commits, which sessions already write. Basis: in the measured window 198 of 510 commits on main carry it, against 191 merged pull requests authored by the owner's login; that is an indication, not a per-pull-request join, so the first step of any view is that join.
Activity
github-actions commentedon Nov 12, 2026
Size: Medium
Complexity: Medium
Uncertainty: Knowable
Flags: none
Blocked by: none
Understood: a research question asking which of three trust measures (revert rate, share of PRs merged with no human touch, corrections converted to checks) this repository can compute from its own data. For each measure the finding should give the data source, whether it is computable now, and the definition used, with evidence from this repository's history and the delivery-metrics literature. If a measure is computable, an existing skill gains a view.
The scope is clear enough to start. Reading the repository history and the existing measure-delivery and morning-brief skills should settle it.
Related, not a duplicate: #15650 covers counting corrections that became checks, which is only the third measure here.
Automated triage. A maintainer reviews it.
melodic-automation-lanes commentedon Nov 12, 2026
Not admitted to the issue lane yet: no trusted-human acceptance criteria: a listed human must put the confirmed
- [ ]criteria in the body or post them as their own comment, or, on an issue a listed human filed, intake's newest refinement comment must hold them unedited; intake's refinement of this body is due and may supply them.kyle-sexton commentedon Nov 12, 2026
kyle-sexton commentedon Nov 12, 2026
Resolved: of the three trust measures, revert rate is computable today, the no-human-touch merge share is computable only as a lower bound, and corrections converted to checks is not computable because nothing tracked records the conversion. Basis: the research below. A fresh verifier confirmed the definitions and the arithmetic and lowered the confidence of the computed values; the caveats are stated per measure.
From the talk
Per measure
main, through/work-items:measure-delivery/corrections:codifyproposals whose check was later installed, over all proposalsRevert rate
plugins/work-items/skills/measure-delivery/SKILL.md:46-69states that the skill counts each first-parent commit on the default branch as a deployment and detects reverts from git history alone.No-human-touch merge share
plugins/work-items/skills/morning-brief/SKILL.md:244states that its owner-touch count includes any agent session holding that login. A pull request the owner's login touched cannot be split into human and agent action by login alone.maincarry a Claude co-author trailer, against 191 of the 500 merged pull requests authored by the owner's login.Corrections converted to checks
plugins/corrections/skills/codify/SKILL.md:153-164,181states that codify writes proposals and notes under the gitignored.work/codifydirectory; it records no tracked link from a correction to an installed check..work/codifydirectory.What this means for a view
/work-items:measure-deliveryalready reports it.Verification
A fresh verifier re-fetched the DORA and Jellyfish pages and the three skill files and confirmed each quoted definition and line. After one rework round it still failed three rows:
SKILL.mdfiles state, not as observed run-time behavior, and are worded that way here;The full research files and the raw command output go on the map's dossier branch.
kyle-sexton commentedon Nov 12, 2026
Done (2026-11-12): Resolved; see the resolution comment: revert rate is computable now (13 of 7652 landings, 0.17%), the no-human-touch share only as a lower bound, corrections converted to checks not until a tracked record exists.
kyle-sexton commentedon Nov 12, 2026
Owner decision (2026-11-12, "Let's go with recommendations"): the per-pull-request signal that separates the owner's own actions from an agent session's is the Claude co-author trailer on commits, which sessions already write. Basis: in the measured window 198 of 510 commits on
maincarry it, against 191 merged pull requests authored by the owner's login; that is an indication, not a per-pull-request join, so the first step of any view is that join.