Skip to content

Pattern miner: Add backlog-drip archetype for the All You Can Eat scheduling pattern #297

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
pelikhan merged 1 commit into from
Sep 29, 2026
Merged

Pattern miner: Add backlog-drip archetype for the All You Can Eat scheduling pattern #297

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/copilot-instructions.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ You are an expert GitHub Agentic Workflow generator. You help users create produ

## Your Knowledge

Use the committed pattern library as the source of truth: `patterns/manifest.json` plus `patterns/archetypes/*.json` generated on 2026-08-31 from 223 source repos, 175 active workflows, and 671 total workflows scanned. The current wizard manifest lists 30 user-facing archetypes; the `custom` archetype exists as a supporting pattern file and is intentionally not exposed as a HOW-step archetype.
Use the committed pattern library as the source of truth: `patterns/manifest.json` plus `patterns/archetypes/*.json` generated on 2026-08-31 from 223 source repos, 175 active workflows, and 671 total workflows scanned. The current wizard manifest lists 31 user-facing archetypes; the `custom` archetype exists as a supporting pattern file and is intentionally not exposed as a HOW-step archetype.

### Key Data Points

Expand All @@ -23,7 +23,7 @@ Use the committed pattern library as the source of truth: `patterns/manifest.jso
- `custom` is hidden from the wizard archetype cards but retained for matching and profile data. Best observed custom profiles are schedule + create-pull-request + noop at 95.2% (n=21), schedule + create-issue + noop + threat-detection at 83.9%, and schedule + create-issue + noop at 80.0%.

**Curated archetypes without empirical runs yet (`count: 0`):**
- accessibility-expert, agent-cost-tracker, batched-ci-doctor, ci-failure-triage, code-health-auditor, community-digest, contribution-guidelines-checker, issue-hierarchy-manager, link-checker, linter-applier, linter-miner, linter-refiner, linter-workflows, nitpick-reviewer, performance-nut, pr-fix-assistant, pr-iteration-loop, repo-qa-assistant, security-scanner, skill-pr-reviewer, user-simulator.
- accessibility-expert, agent-cost-tracker, backlog-drip, batched-ci-doctor, ci-failure-triage, code-health-auditor, community-digest, contribution-guidelines-checker, issue-hierarchy-manager, link-checker, linter-applier, linter-miner, linter-refiner, linter-workflows, nitpick-reviewer, performance-nut, pr-fix-assistant, pr-iteration-loop, repo-qa-assistant, security-scanner, skill-pr-reviewer, user-simulator.
- Keep these archetypes available. They are newer curated patterns and should not be removed simply because they have no measured success rate.

**Trigger combo risk:**
Expand Down
36 changes: 36 additions & 0 deletions patterns/archetypes/backlog-drip.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
{
"id": "backlog-drip",
"label": "Backlog Drip",
"description": "Serve one open-ended backlog item at a time, activating again only after the previous item is closed",
"success_rate": null,
"count": 0,
"recommended_triggers": [
{
"type": "schedule",
"config": {}
}
],
"recommended_safe_outputs": [
"issues"
],
"recommended_tools": [
"create-issue"
],
"timeout_minutes": 15,
"prompt_style": "role-steps",
"size_range_bytes": [
2000,
6000
],
"top_repos": [],
"tips": [
"Wake up frequently (e.g. every 30 minutes) but cap the safe output at max: 1 and set skip-if-match to a query that matches the workflow's own still-open item, so a new one is never produced while the last is unconsumed",
"Match on a stable identity marker (the hidden gh-aw-workflow-id footer, or a consistent title-prefix) rather than free text, since humans rename titles",
"Set expires on the created issue or pull request so an abandoned, never-closed item eventually unblocks the schedule instead of stalling it forever",
"Before proposing a new item, search the last few closed items from this workflow and read the close reason and any review feedback \u2014 treat 'not planned' or a rejecting comment as a negative signal and avoid re-proposing the same idea",
"Persist a compact accept/reject history across runs in cache-memory (or repo-memory if it must survive cache eviction) so the backlog improves over time instead of repeating rejected proposals",
"Use this pattern for open-ended, non-event-driven backlogs (improvement ideas, maintenance chores, research notes) \u2014 prefer the community-digest archetype instead when the output is a periodic report tied to a time window",
"Call noop when nothing in the backlog clears the quality bar left by past rejections"
],
"anti_patterns": []
}
3 changes: 2 additions & 1 deletion patterns/manifest.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,8 @@
"link-checker",
"repo-qa-assistant",
"nitpick-reviewer",
"pr-fix-assistant"
"pr-fix-assistant",
"backlog-drip"
],
"workflow_generation": "workflow-generation.json",
"anti_patterns": [
Expand Down
26 changes: 26 additions & 0 deletions patterns/workflow-generation.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1251,6 +1251,32 @@
"- **DO NOT** post more than one comment per invocation.",
"- **DO NOT** assert claims you could not verify from the repository or trusted sources."
]
},
"backlog-drip": {
"icon": "checklist",
"capabilities": {
"github_toolsets": true
},
"body": [
"# {{label}}",
"",
"You are a **backlog drip** agent for this repository, serving one open-ended backlog item at a time so the next one only appears after the previous item is closed.",
"",
"## Process",
"",
"1. Load the accept/reject history from cache-memory",
"2. Search the last few closed items produced by this workflow (same marker or title prefix), newest first",
"3. Read each closing reason, closing comment, labels, and any review feedback; treat 'not planned' or a rejecting comment as a negative signal, and completed/merged as a positive signal for similar work",
"4. Identify the next backlog candidate that clears the quality bar set by that history",
"5. Create the item, or call `noop` when nothing clears the bar",
"6. Update the accept/reject history in cache-memory before finishing",
"",
"## Constraints",
"",
"- **DO NOT** create a new item while a previous item from this workflow is still open — the schedule guard already enforces this, but never bypass it.",
"- **DO NOT** re-propose an idea already recorded as rejected in cache-memory without new evidence.",
"- **DO NOT** produce more than one item per run."
]
}
}
}
2 changes: 1 addition & 1 deletion test/copilot-instructions.test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
it('describes the committed pattern-library corpus, not stale scan data', () => {
const generatedDate = manifest.metadata.generated_at.slice(0, 10);

expect(instructions).toContain(`generated on ${generatedDate}`);

Check failure on line 12 in test/copilot-instructions.test.js

View workflow job for this annotation

GitHub Actions / build

test/copilot-instructions.test.js > copilot instructions pattern guidance > describes the committed pattern-library corpus, not stale scan data 

AssertionError: expected '# Agentic Workflow Generator — Copilo…' to contain 'generated on 2026-09-21'

- Expected
+ Received

- generated on 2026-09-21
+ # Agentic Workflow Generator — Copilot Instructions
+
+ You are an expert GitHub Agentic Workflow generator. You help users create production-ready `.md` workflow files for [GitHub Agentic Workflows (gh-aw)](https://github.github.com/gh-aw/).
+
+ ## Your Knowledge
+
+ Use the committed pattern library as the source of truth: `patterns/manifest.json` plus `patterns/archetypes/*.json` generated on 2026-08-31 from 223 source repos, 175 active workflows, and 671 total workflows scanned. The current wizard manifest lists 31 user-facing archetypes; the `custom` archetype exists as a supporting pattern file and is intentionally not exposed as a HOW-step archetype.
+
+ ### Key Data Points
+
+ **Archetypes with empirical data:**
+ - `daily-test-improver`: 100% success (n=3). Best trigger shape: permissions + reaction + schedule. Safe outputs: pull-requests.
+ - `documentation-updater`: 68% success (n=9). Best trigger shape: schedule + skip-if-match + permissions. Safe outputs: pull-requests.
+ - `issue-triage`: 52% success (n=72). Best trigger shape: issues + roles + reaction. Safe outputs: issues.
+ - `dependency-monitor`: 50% success (n=48). Best trigger shape: schedule + permissions + reaction. Safe outputs: issues, pull-requests.
+ - `code-improvement`: 46% success (n=73). Best trigger shape: schedule + reaction + permissions. Safe outputs: pull-requests.
+ - `pr-review`: 42% success (n=63). Best trigger shape: pull_request + roles + pull_request_target. Safe outputs: pull-requests.
+ - `status-report`: 38% success (n=36). Best trigger shape: schedule + skip-if-match + permissions. Safe outputs: issues.
+ - `repo-maintainer`: 33% success (n=8). Best trigger shape: reaction + slash_command + schedule. Safe outputs: issues, pull-requests.
+ - `content-moderation`: 0% success (n=3). Best trigger shape: issue_comment + issues + pull_request. Safe outputs: issues, pull-requests.
+
+ **Supporting empirical profile:**
+ - `custom` is hidden from the wizard archetype cards but retained for matching and profile data. Best observed custom profiles are schedule + create-pull-request + noop at 95.2% (n=21), schedule + create-issue + noop + threat-detection at 83.9%, and schedule + create-issue + noop at 80.0%.
+
+ **Curated archetypes without empirical runs yet (`count: 0`):**
+ - accessibility-expert, agent-cost-tracker, backlog-drip, batched-ci-doctor, ci-failure-triage, code-health-auditor, community-digest, contribution-guidelines-checker, issue-hierarchy-manager, link-checker, linter-applier, linter-miner, linter-refiner, linter-workflows, nitpick-reviewer, performance-nut, pr-fix-assistant, pr-iteration-loop, repo-qa-assistant, security-scanner, skill-pr-reviewer, user-simulator.
+ - Keep these archetypes available. They are newer curated patterns and should not be removed simply because they have no measured success rate.
+
+ **Trigger combo risk:**
+ - The manifest's curated `trigger_combos` list contains only high performers: 13 of 15 tracked combos are 90–100% successful and all are marked Recommended.
+ - Lone `reaction` is very reliable at 99% success (n=90).
+ - `bots+roles+schedule+stale-check` is the softest Recommended tracked combo at 90% success (n=20).
+ - workflow_run chaining has 13-16% success rate. Use pre-steps or schedule instead. Only use workflow_run when the archetype is explicitly about scoped workflow-run analysis.
+ - Slash commands act as dispatchers that route conversational commands to target workflows through `workflow_dispatch`; retain slash-command profiles even when measured performance is low.
+
+ **Configuration profiles and anti-patterns:**
+ - Trigger choice alone does not guarantee success: `code-improvement` schedule + skip-if-match -> create-pull-request measured 0% across 23 runs, and the workflow_run variant also measured 0%.
+ - `issue-triage` with add-comment + add-labels + assign-to-agent underperformed at 12.2% (n=82); prefer s
expect(instructions).toContain(`${manifest.metadata.source_repos} source repos`);
expect(instructions).toContain(`${manifest.metadata.active_workflows} active workflows`);
expect(instructions).toContain(`${manifest.metadata.total_workflows} total workflows scanned`);
Expand All @@ -28,7 +28,7 @@
}

expect(empirical).toHaveLength(9);
expect(curated).toHaveLength(21);
expect(curated).toHaveLength(22);
for (const id of empirical) expect(instructions).toContain(`\`${id}\``);
for (const id of curated) expect(instructions).toContain(id);
expect(instructions).toContain('`custom` is hidden from the wizard archetype cards');
Expand Down
Loading