Skip to content

Map: integrate the 'All Code Is Slop (Except Mine)' talk into plugins and skills #15651

Closed
17 of 17 issues completed
Closed

Description

@kyle-sexton

Destination

Every topic of the talk has a decided outcome (build it, already have it, or declined with a reason), and each buildable one is a work item for a plugin or skill that includes a dogfooding step in this repository. Closes to /work-items:decompose.

Notes

  • Skill every session on this map consults: /discovery:research, before any recommendation that rests on outside evidence.
  • Method for every decision item (owner, 2026-11-12): start from the talk's own section for the item's chapter (the video frames, the transcript and the on-screen diagrams at the timestamps and slide numbers below), then weigh current research, Matt Pocock's skills as absorbed and this repository's existing plugins and setup, and converge on the best fit. A resolution that cites no slide or transcript passage is incomplete.
  • Where a research item's result lands: its resolution comment on the item is the finding's home; the full findings file goes on this map's one dossier branch, which is never merged and is deleted when the map closes: dossier/15651-integrate-all-code-is-slop-talk.
  • The talk: "All Code Is Slop (Except Mine)", Lauren Tan, AI Engineer Code Summit, 2026-11-11, video from 1:00:18 to 1:18:25. Her thesis: with agents, volume is free and trust is the constraint; buy trust cheapest and widest first (code shape, machine checks, verification, written rules, human review), turn every correction into a check, review evidence rather than diffs, and automate only a loop that already produces good work.
  • Its ten chapters:
    1. Opening (slides 1-5): slop is relative; volume; "coding machines"; trust, not a "software factory".
    2. Trust model (6-11): two loops and the inverted pyramid; deterministic layers versus the prose judgment layer.
    3. Code is memory (12-16): the codebase is the largest input; gardening; copies breed copies.
    4. Examples (17-19): banned APIs, import bans, directory structure and file names as context.
    5. Escape hatches (20-22): unsolicited markdown and comments; the 99% comment ban.
    6. Corrections (23): every correction becomes a check whose message names the fix.
    7. Verification (24-28): review verification artifacts, not diffs; runtime verification with a feature map and driver CLI; TLA+ and Lean; UI property testing.
    8. Why now (29-32): most tokens go to verification; a 2018 chart of techniques per test stage; statecharts.
    9. Results (33-38): her team's pull request volume, releases, revert rate and performance.
    10. Order of work (39-44): lint first, clean up, strengthen verification, then automate; outer-loop triggers; a release run that explores the build and compares it with the last release.
  • First region, the ecosystem-specifics and ladder Brief (locked in an interview, built separately from this map). Its outcomes:
    • Generic skills stop shipping model-read language, framework and tool choices; each skill reads the consumer's own records, then infers from the repository.
    • Executed adapters, single-tool plugins and security floors stay; an adapter names what it cannot read instead of skipping it.
    • Shipped defaults that pick a tool, command or library are retired; unattended lanes run only what the repository states or its build files imply.
    • The enforcement ladder becomes the one representation of the talk's pyramid: capability-named rungs, a verification tier in third place, a proportion rule in words, and her order of work (lint first, clean up, strengthen verification, then automate).
    • The automation-gaps audit reports how a repository's enforcement is spread across the tiers; a review rule holds new enforcement-placing skills to the ladder.
  • Constraints confirmed by the owner (C1-C11):
    • C1: No shipped language specifics, applied to content that already ships.
    • C2: /corrections:codify only picks the rung and proves the check; /implementation:implement builds the rule in the consumer's own tooling.
    • C3: /app-control:drive never judges; the release pass is a whole-map scope on the existing live verification, not a new skill (revised 2026-11-12). Property testing and flood-style fuzzing are one thing, housed in the /testing:write property guidance and not split by target.
    • C4: No issue or comment in a repository the owner does not own without the owner's permission, and the request says plainly it is not our repository.
    • C5: The owner's own posture and this repository's is no code comments; plugins ship the keep-the-why default with the no-comments posture supported.
    • C6: Quality bar is production: single responsibility, low coupling, high cohesion, things that change together live together, encapsulation, semantically accurate names.
    • C7: Decisions hold only while aligned with the talk, its facts and data, and current research from official, authoritative sources.
    • C8: Shipped skills that recommend, place or build enforcement or an instruction represent the pyramid's layers in order of preference, strongest to weakest, and say which to have most and least of.
    • C9: Shipped skills adapt to the downstream context (repository, file, organization, user): they discover, research, then choose how to implement that layer there.
    • C10: A skill reaches another only by its slash reference, presence-gated with a stated fallback, never its internals.
    • C11: The whole talk is integrated into new or existing plugins and skills, and dogfooded in this repository; nothing she covered is left out.
  • Facts already established:
    • The video digest's home is the knowledge corpus plus a docs/upstream/ record.
    • A push to the default branch is already blocked: main carries a pull-request rule here and in the dotfiles repository.
    • One skill refers to another by /plugin:skill, guarded by an "if installed" check with a documented fallback (docs/plugin-philosophy.md:101-116).
    • Five plugins carry a copy of the enforcement ladder: review, session-flow (retro), harness-config (automation-gaps audit), architecture (improve), corrections (codify) (scripts/shared-copies.txt:91-95).
    • A consumer's records already layer user-global, team and personal settings, later layers overriding per key (plugins/toolchain/reference/resolution-ladder.md:35-39).
    • Already have: the pyramid's order is the enforcement ladder (plugins/review/context/enforcement-ladder.md:21-89); "every correction becomes a check" is /corrections:codify; the feature map and driver CLI are /app-control:map and /app-control:drive; her no-comments command is /code-tidying:remove-comments; copies breeding copies is covered by /toolchain:ratchet and the code-tidying plugin.
    • Her team's numbers cannot be verified from outside, and her performance slide shows four measures that got worse.
  • Decided work items (built off this map, each needs /planning:interview before build):
  • The three dotfiles decision items (the stale multi-line commit message rule, checkable rules to hooks or permission rules, and the unproven redirect reason) are decided on this map and built in the dotfiles repository.
  • Upstream record: docs/upstream/cursor-pstack.md, her pstack plugin absorbed.
  • Spec container for the first region (the ecosystem-specifics plan, decomposed 2026-11-12): Resolve ecosystem specifics from the consumer repository and align the enforcement ladder with the talk's pyramid.
  • Related: knowledge:video-digest: digest a time range of a video (--from/--to).
  • Labels: wayfind: interview marks a decision the owner makes; wayfind: research an external-evidence question an agent can resolve; wayfind: task decision-unblocking work; needs-human keeps an item off every autonomous frontier.

Decisions-so-far

Not-yet-specified (fog)

None left. The owner settled the four remaining points on 2026-11-13:

  • Dogfooding depends on what the item produces, and the pull request body names which: a check runs in this repository's CI; a setting is set in docs/conventions/; a skill gets one recorded run against this repository through /app-control:drive.
  • A release pass in a marketplace with no application of its own treats the skills as the app: it drives every entry of this repository's control skill at each plugin release. Details are settled in the interview on the release pass.
  • No trust measure beyond the three already resolved, for now; revisited in the interview on the corrections count and rule-to-enforcer inventory.
  • "Review evidence, not diffs" for a plugin pull request is the evidence comment plus the eval compare on any change to skill text. No new item.

Out-of-scope

  • Her team's own numbers as targets: unverifiable from outside, and her performance slide shows four measures that got worse.
  • A suppression expiry key: declined by the finding-suppression contract, which reports entries older than exceptions.max_days instead (docs/contracts/finding-suppression/README.md:243-250).
  • Banning one specific API (her useEffect example) in a shared plugin: that is a consumer's rule, reached through /corrections:codify and /implementation:implement in the consumer's own linter.
  • TLA+ or Lean as a default step: only as an opt-in tool chosen by the question.

Activity

  1. added theissue type on Nov 12, 2026
  2. added
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Nov 12, 2026
  3. github-actions commented on Nov 12, 2026

    @github-actions
    Contributor

    Size: Large
    Complexity: Medium
    Uncertainty: Knowable
    Flags: none
    Blocked by: none

    I read this as a work-map tracking issue. Its goal is that every topic in the talk is decided (build, already have, or declined with a reason). Each buildable topic then becomes a work item that includes a dogfooding step in this repository. The map already lists four decided work items (#15647 to #15650) and several open sub-issues. The remaining fog (what "dogfooded" means per item, and how a release-exploring run applies to a marketplace) is by design and is being charted in those sub-issues, so I'm not raising a separate question here. No duplicate or merged fix found. The work is too big for one change and needs splitting, which the map already does.

    Automated triage. A maintainer reviews it.

  4. removed
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Nov 12, 2026
  5. melodic-automation-lanes commented on Nov 12, 2026

    @melodic-automation-lanes
    Contributor

    Not admitted to the issue lane yet: sub-issue #15657 is open; sub-issue #15658 is open; sub-issue #15659 is open; sub-issue #15660 is open; sub-issue #15661 is open; sub-issue #15662 is open; sub-issue #15663 is open; sub-issue #15664 is open; sub-issue #15665 is open; sub-issue #15666 is open.

  6. 6 remaining items

  7. added
    wayfind: mapWayfind map: the /planning:wayfind issue whose sub-issues are its decision items.
    on Nov 13, 2026
  8. kyle-sexton commented on Nov 13, 2026

    @kyle-sexton
    MemberAuthor

    Map closed: every decision, in full

    All 17 decision items are closed and the frontier is empty. Each decision's home is its item's resolution comment; this comment states them together for /work-items:decompose.

    Decisions

    1. What did the interview on the talk already settle? The interview's answered questions and the ecosystem-specifics Brief's acceptance criteria. Built separately under the first-region spec container.
    2. Which test-stage chart techniques apply to what consumers of a plugin marketplace build? No technique-coverage lens. 26 of the 42 techniques apply in a consumer repository and 25 are covered at least partly. /testing:plan gains a fuzz-target row and a consumer-driven contract-test row.
    3. Which trust measures can this repository compute today? Revert rate is computable now, the no-human-touch share only as a lower bound (signal: the Claude co-author trailer), corrections converted to checks not at all.
    4. Should a check's output have to name the fix, and be tested for it? Yes. Every finding names a root-cause remedy first, covered checks are tested for it, this repository's 68 check scripts are held by a ratchet to zero, and a suppression-only script gets a real remedy or is deleted.
    5. Should a plugin flag a new markdown file the task did not ask for, at write time? Declined. A hook cannot tell whether the task asked for a file, and every one of the 1,073 markdown files added in the last 300 commits landed in a product or record path. Reopen when an unrequested file is found merged.
    6. Should the release exploring run keep a per-release baseline and report only new findings? Yes, inside the release pass itself: its report marks each finding new, pre-existing or gone against the last release, keyed by feature-map entry. No separate skill.
    7. Should each verification check record the files it covers, so upkeep can classify a change? Already covered for runtime checks by control-skill sources globs and refresh triage; for formal models it is a requirement on the formal-verification plugin. No new build item.
    8. Should we adopt her two one-line verification rules, and in which skills? Yes. The replay rule lives in the formal-verification plugin; "suspect the observation method first" goes once in /verification:confirm with an eval case.
    9. Should verification artifacts be shown in the pull request where a reviewer sees them? Already built as the one evidence comment posted after a confirmed head; the local ready step will post it too. No second place for evidence.
    10. Should the feature map name a "what proves it works" part? Yes. Every drive step states its observable result and each feature ends with a proof step; app-control is aligned with her verification skills in five work items. Four rejections on record stay.
    11. Should folder-per-feature and file-suffix conventions be offered as derivable conventions? Not as shipped conventions. A style-agnostic layout lens on /architecture:improve checks whether files that change together live together.
    12. Which of five smaller pstack review and verification ideas should we take? Each idea, and each gap found by re-reading her whole plugin, has a disposition validated by an independent review: two reshaped takes need interviews, the small takes are filed, the rest are skipped with reasons.
    13. Should the retro consider a check before a prose rule? Yes. The retro's catalog asks "could a check decide this?" first, lists its targets strongest first, and admits a prose rule only when no check can decide it, with one eval case.
    14. Which of this repository's own checkable prose rules should get a check? Five, ranked by evidence, plus aligning the force-push prose with its hook. Four rules with no violations and no fix history are skipped until a first violation.
    15. Should the dotfiles instructions drop the stale multi-line commit message rule? Yes, the whole sentence. The hook stopped blocking multi-line messages the day the rule was written, and its message names the fix.
    16. Which low-friction checkable dotfiles rules should become hooks or permission rules? Stale prose fixed first; then the bypass-mode setting, a plugin hook on foreground waits, two deny rules and a Haiku spawn hook. Three high-friction rules stay prose; the temp-worktree rule is on hold.
    17. Should the unproven "redirects under the work directory are blocked" reason be verified or deleted? Verified false: the guard was removed the day the line was written. The line is deleted and two stale guardrails comments are fixed.

    Also decided on this map: the formal-verification plugin in three slices, the release pass as a whole-map scope on the existing live verification, property testing and flood-style fuzzing as one guidance, and the cheat sheet extension. Their items are listed in the map's Notes.

    Build items already filed

    Listed in the map's Notes under "Decided work items", plus the first-region spec container with its 30 sub-items. Do not file these again.

    Build items still to file (from decisions 13, 14, 16 and 17)

    Each is one vertical slice with its own pull request, names its dogfooding step in this repository in the pull request body, and gives every new check a message that names the fix. A change to skill text opens with do-not-merge until its eval compare lands.

    1. Retro catalog, strongest first (session-flow; decision 13). In plugins/session-flow/skills/retro/reference/ecosystem-improvement-catalog.md: a "could a check decide this?" step before the scope tree, routing to /corrections:codify when installed, else the Hooks section and the retro's ladder copy; target sections reordered Hooks and Settings, CLAUDE.md / Rules, Memory; "no rung above guidance can decide it" as the entry condition of CLAUDE.md / Rules; one eval case where a mechanical finding comes back as a check proposal. Settle whether the catalog's check thresholds duplicate codify's recurrence.min_occurrences. Agent-ready.
    2. Run the rules-index check in CI (decision 14). Wire plugins/instruction-placement/git-hooks/index-check.sh into the hygiene checks. Agent-ready.
    3. Check that an as-of stamp carries a recheck trigger (decision 14). In skill and agent files, a paragraph with as of YYYY-MM-DD must hold a recheck trigger; shrink-only baseline for the 16 current hits. Agent-ready.
    4. Check the Pre-existing (advisory): marker on follow-ups (decision 14). AGENTS.md first says whether a non-advisory follow-up is allowed; then the pull-request contract step checks bullets under ## Follow-ups. Settle whether the check lives here or in melodic-software/ci-workflows. Agent-ready.
    5. Flag a docs-page pointer with no section anchor (decision 14). Advisory first, with a baseline for the 246 current hits. Agent-ready.
    6. Flag price and savings figures (decision 14). A pattern check with an exemption list for the two exceptions .claude/rules/cost-claims.md states. Agent-ready.
    7. Align the force-push prose and the guardrails hook (decision 14). plugins/guardrails/hooks/block-dangerous-git.sh:245 accepts --force-with-lease --force-if-includes; AGENTS.md allows only the pinned lease. Make the two agree. Agent-ready.
    8. Block foreground until and --watch waits with a plugin hook (decision 16). Read the hooks reference first for whether a PreToolUse hook sees the Bash call's background flag. Agent-ready.
    9. Fix two stale comments in the secret-pattern hook (decision 17). plugins/guardrails/hooks/secret-pattern-detection.sh:336,403 name block-hook-bypass, which no longer exists. Comment-only. Agent-ready.

    The dotfiles changes from decisions 15, 16 and 17 are built in the dotfiles repository and recorded on dotfiles #1556; they are not items here.

  9. kyle-sexton commented on Nov 13, 2026

    @kyle-sexton
    MemberAuthor

    Done (2026-11-13): Destination coherent, handed to /work-items:decompose.

  10. kyle-sexton commented on Nov 13, 2026

    @kyle-sexton
    MemberAuthor

    Reopened. All 17 decision items are closed and the closing comment above stands, but the map's Not-yet-specified section still holds four uncertainties the owner has not been asked about: what "dogfooded in this repository" means per item, how a release pass applies to a marketplace with no application of its own, what else counts as a trust measure, and whether "review evidence, not diffs" has an equivalent for plugin pull requests. An earlier edit to the map body today marked them covered by existing items without the owner's answer; that edit is reverted. The map closes again once the owner has settled each one.

  11. kyle-sexton commented on Nov 13, 2026

    @kyle-sexton
    MemberAuthor

    Done (2026-11-13): The owner settled the four Not-yet-specified points, now recorded in the map body. With the 17 decisions in the closing comment above, the destination is coherent and handed to /work-items:decompose. The nine build items from the last five decisions are filed: 15935, 15933, 15936, 15937, 15938, 15939, 15940, 15941, 15942.

  12. kyle-sexton commented on Nov 15, 2026

    @kyle-sexton
    MemberAuthor

    Recorded from #15650 (owner, 2026-11-15): the trust measures stay at three. Covered: pull-request volume by deployment frequency in /work-items:measure-delivery, and app performance by /performance:protect. Skipped: release cadence, because it follows the release bot's schedule, and contributors per month, because it is a per-person count. The count of corrections that became checks is #15650, and the share of merges no human touched is its own item.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    wayfind: mapWayfind map: the /planning:wayfind issue whose sub-issues are its decision items.

    Type

    Fields

    Work status

    ready

    Priority

    medium

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions