Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 8 additions & 9 deletions .github/agents/extraction-pipeline.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ Phase 1: Collect Phase 2: Enrich Phase 3: Generate Phase 5: V
1.3 Collect PR context → comment_analysis (raw) → final artifacts
→ pr_contexts 2.2b Deduplicate 3.2 Anonymize
1.4 Reconcile paths → pr_rule_votes → *.md (anon) Phase 4 is NOT a pipeline
→ user_comments 2.3 Synthesize 3.3 Anthropic guide step — it defines the
→ user_comments 2.3 Synthesize 3.3 Skill guidance step — it defines the
(paths updated) → dimensions.json → *.md (polished) review workflow EMBEDDED
1.5 Backup → principles.json 3.4 Deduplicate in the generated agent.
→ JSON files → dim_evidence → *.md (deduped)
Expand All @@ -35,7 +35,7 @@ This pipeline processes **thousands** of GitHub items (typically 3,000–10,000+

**Context management:** Store all intermediate results in **SQLite** (queryable) and **JSON backup files** (recoverable). Sub-agents write results to files; the orchestrator imports into SQLite and dispatches the next phase. Never pass large datasets through agent context — use the filesystem.

**Model selection:** Use GPT-5.6 Sol for classification and synthesis sub-agents. Fast/cheap models produce shallow rules. Collection sub-agents can use standard models. Use background mode so agents run in parallel.
**Model selection:** Use a high-capability reasoning model for classification and synthesis sub-agents, respecting the user's model selection. Collection sub-agents can use standard models. Use background mode so agents run in parallel.

**Reliability:** After each batch of sub-agents completes, validate output files: >500 bytes, parseable JSON, contains entries for all assigned items. Re-dispatch incomplete outputs up to 3 times. Keep batch assignments to ≤5 batches per agent — agents given too much work produce placeholders or give up.

Expand Down Expand Up @@ -182,7 +182,7 @@ Store feature areas in SQLite: `CREATE TABLE feature_areas (area_name TEXT, fold
> **Output:** SQLite `comment_analysis` table, `taxonomy.json`
> **Context per sub-agent:** taxonomy + CI summary + 15 PR packets (all comments on each PR)

For each collected comment, classify using a GPT-5.6 Sol sub-agent. **Do not use a hardcoded category list** — derive categories from the data:
For each collected comment, classify using a sub-agent. **Do not use a hardcoded category list** — derive categories from the data:

1. **Bootstrap pass**: Take a stratified sample of ~300 comments: proportional by year, at least 5 per major feature area from §2.1, and at least 20 each of review_comments, pr_descriptions, and issue_comments. Ask a sub-agent to read them and propose a category taxonomy. The agent should identify recurring themes, name them, and define each in one sentence. Expect 15–40 categories to emerge. After deriving the taxonomy, cross-check it against the feature area table — if any area representing >10% of the codebase has zero categories, re-sample with enforced coverage.

Expand Down Expand Up @@ -223,7 +223,7 @@ This ensures a PR with 50 comments gets weight=1, same as a PR with 1 comment. T

### 2.3 Clustering

> **Sub-agents:** 1 (GPT-5.6 Sol, synthesis)
> **Sub-agents:** 1 (synthesis)
> **Input:** `pr_rule_votes` table, `taxonomy.json`, `feature_areas` table, `ci_summary.txt`
> **NOT available:** raw `user_comments`, JSON backups — synthesis works only with classified, deduplicated data
> **Output:** `dimensions.json`, `principles.json`, `folder_hotspots.json`, SQLite `dimension_evidence` table
Expand Down Expand Up @@ -298,12 +298,12 @@ Remove all personal names, comment counts, PR number references, evidence statis

**Commit** after anonymization.

### 3.3 Improve per Anthropic guide
### 3.3 Apply skill-authoring guidance

> **Input:** `*.md` (anonymized)
> **Output:** `*.md` (polished)

Apply https://platform.claude.com/docs/en/agents-and-tools/agent-skills/best-practices:
Apply these skill-authoring practices:
- `name`: gerund form, lowercase+hyphens
- `description`: third person, specific triggers, ≤1024 chars
- Concise — only add what the model doesn't already know
Expand Down Expand Up @@ -387,7 +387,7 @@ For each non-LGTM finding, actively prove or disprove it:
- **Write and run tests for claims**: Write a minimal test that demonstrates the claimed issue. Run it against the PR branch. If the test fails as predicted → confirmed. If it passes → disputed.
- **Proof-of-concept snippet**: When a full test is too complex to run inline, write pseudocode or partial code demonstrating the issue. Include in PR feedback as evidence — enough for another engineer to implement.
- **Scenario simulation**: For complex issues (concurrency, state machines, protocol interactions), write a step-by-step execution trace showing how the bug manifests.
- **Multi-model consensus**: For borderline findings, validate with 3 models (Opus, Codex, Gemini). Keep findings confirmed by ≥2/3.
- **Multi-model consensus**: For borderline findings, validate with 3 distinct available model families. Keep findings confirmed by ≥2/3.

A finding is confirmed only with concrete evidence. Never validate against `main` — PR code only exists in the PR branch.

Expand Down Expand Up @@ -436,7 +436,7 @@ All `[x]` → APPROVE. Any BLOCKING → REQUEST_CHANGES. Otherwise → COMMENT.

## Phase 5: Final Quality Gate

### 5.1 Anthropic guide compliance
### 5.1 Skill-authoring compliance

Verify all artifacts against best practices:
- YAML frontmatter: name (gerund, ≤64), description (third person, ≤1024, triggers)
Expand Down Expand Up @@ -508,4 +508,3 @@ Final check on the complete artifact set:
- No rules referencing specific function names or line numbers unless those functions are long-lived stable APIs (verified by grep in 5.6)
- Every CHECK item is phrased as a generalizable principle, not a transcription of one PR's feedback
- Dimension frequency was counted by PRs, not by comments — a PR with 50 comments counts the same as one with 1 comment

4 changes: 2 additions & 2 deletions .github/skills/code-compaction/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,8 +61,8 @@ routing); steps 3–4 are per-mode prep once a mode is picked:

Dispatch **3–5 background reviewers in parallel** — never sequentially.
**Cross-model > cross-angle**: three same-model agents with different
prompts ≠ three different models (Opus high/xhigh, newer Opus, GPT). Each
agent picks one angle from the mode's template. Record
prompts ≠ three different models. Use distinct available model families.
Each agent picks one angle from the mode's template. Record
`model × angle × LOC × tests × reuse × verdict`. Cross-model agreement is
signal; lone-model claims need code citations before promotion.

Expand Down
51 changes: 29 additions & 22 deletions .github/workflows/agentic-state-machine.lock.yml

Large diffs are not rendered by default.

5 changes: 4 additions & 1 deletion .github/workflows/agentic-state-machine.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ description: |
state machine they define, renders Mermaid diagrams + tables in
.github/docs/state-machine.md. Weekly. Opens PR if changed.

imports:
- shared/model-defaults.md

on:
schedule: every 7d
workflow_dispatch:
Expand Down Expand Up @@ -261,7 +264,7 @@ You are a workflow-automation documentor. You read all workflow files in `.githu
38. **`always()` in guard expressions MUST be preserved.** When a job `if:` uses `always() && <condition>`, the `always()` modifier is semantically significant — it means the job evaluates even when predecessors fail/are skipped. Document the FULL expression including `always()`. Dropping `always()` changes the semantics and is HIGH error.

39. **gh-aw `safe-outputs:` — signature, not enumeration.** For each gh-aw `.md` workflow, document its safe-output **signature**: which action verbs it can emit (`create-pull-request`, `add-comment`, `push-to-pull-request-branch`, `add-labels`, etc.) and the distinguishing config per verb. **Do NOT exhaustively list every leaf key.** Universal defaults are suppressed: `target: "*"`, `noop.report-as-issue: false`, `draft: false`. Per-workflow blocks list only OVERRIDES + behaviorally distinguishing fields: `max`, `title-prefix`, `labels`/`allowed`, `allowed-files`, `protected-files`, `reviewers`, `auto-merge`, `hide-older-comments` (when true), `base`.
**Format — PREFER PER-WORKFLOW MINI-TABLES** with columns `| Workflow | Output | Max | Key Constraints |`. Tables scan faster than run-on prose for any workflow with ≥3 actions or any action with ≥3 distinguishing fields. Multi-reviewer feedback (Sonnet + GPT-5.4 + Gemini, average 2.67/5 on first pass) ranked run-on safe-output prose as the #1 readability failure mode. Example:
**Format — PREFER PER-WORKFLOW MINI-TABLES** with columns `| Workflow | Output | Max | Key Constraints |`. Tables scan faster than run-on prose for any workflow with ≥3 actions or any action with ≥3 distinguishing fields. Example:
```
| Workflow | Output | Max | Key Constraints |
|---|---|---|---|
Expand Down
Loading
Loading