[copilot-token-optimizer] Token Optimization: Glossary Maintainer — search_repositories overuse (342 calls/run)

[github-actions]

🔍 Optimization Target: Glossary Maintainer

Selected because: 3rd-highest token consumer in 7-day window; not recently optimized; doubled tokens in 24 hours (1.79M → 3.86M)
Analysis period: 2026-04-06 to 2026-04-07
Runs analyzed: 2 runs (1 in pre-downloaded detail, 1 in historical snapshot)

---

📊 Token Usage Profile

Metric	2026-04-07	2026-04-06	Change
Total tokens	3,855,042	1,786,466	+116% ⚠️
Input tokens	3,838,637	—	—
Output tokens	16,405	—	—
Cache read tokens	3,696,415	—	—
Cache efficiency	49%	—	—
Turns	49	26	+88%
Duration	18.7 min	12 min	+56%
Write actions	2	—	—
Agentic fraction	4%	—	—
Model	claude-sonnet-4.6	—	—

Input:Output ratio = 234:1 — the agent is reading enormous amounts of context to produce ~16K tokens of actual output.

---

🔧 Recommendations

1. Remove search_repositories from the toolset — Est. savings: ~1.5–2M tokens/run

The agent called search_repositories 342 times in a single run, with 76 calls in a 10-second burst (parallel fan-out). This tool searches across all of GitHub — not just the local repository — and is completely unnecessary for a glossary update task that only needs to scan local file changes.

The current config uses toolsets: [default] which includes search_repositories. This should be replaced with a scoped set.

Evidence: 342/588 total tool calls (58%) were search_repositories — by far the top consumer.

Action: Replace toolsets: [default] with explicit tools:

tools:
  github:
    toolsets: [repos]
    # Only needs: list_commits, get_commit, get_file_contents, search_pull_requests, pull_request_read

Or add a prompt guard: "Do not use search_repositories — it searches GitHub globally and is not relevant to this task."

2. Pre-compute recent commits in a bash step — Est. savings: ~500K–1M tokens/run

The system assessment flags agentic_fraction = 0.04, meaning 95% of 49 turns are pure data-gathering with no writes. Only 2 write actions occurred across the entire run. Git history can be fetched deterministically before the agent starts.

Action: Add a pre-step to the workflow frontmatter:

steps:
  pre:
    - name: fetch-recent-changes
      run: |
        git log --since='24 hours ago' --oneline > /tmp/gh-aw/agent/recent-commits.txt
        git log --since='24 hours ago' --name-only --format="%H %s" -- 'docs/**/*.md' 'docs/**/*.mdx' > /tmp/gh-aw/agent/doc-changes.txt
        echo "Changes fetched:" && wc -l /tmp/gh-aw/agent/recent-commits.txt

Then reference in the prompt: "Recent commits are in /tmp/gh-aw/agent/recent-commits.txt — read this file instead of using GitHub tools for commit history."

Expected impact: Reduces data-gathering turns from ~45 to ~10.

3. Restrict issue_read — Est. savings: ~300–500K tokens/run

The agent called issue_read 232 times. The workflow's task is to update a glossary based on code changes, not issue discussions. Reading 232 issues for terminology research is well outside scope.

Action: Add to the prompt: "Do not read issues. Terminology should come from recent commits, PRs, and documentation files — not issue discussions."

Alternatively, remove issues from the allowed toolset entirely.

4. Consider model downgrade — Est. savings: varies by pricing

The system's agentic assessments explicitly flag model_downgrade_available:

"This Repo Maintenance run may not need a frontier model. A smaller model (e.g. gpt-4.1-mini, claude-haiku-4-5) could handle the task at lower cost."

Glossary maintenance is a well-scoped, structured task (read docs → add terms → write PR). A smaller model can handle this reliably.

Action: Add to frontmatter:

engine:
  id: copilot
  agent: technical-doc-writer
  model: gpt-4.1-mini
```

#### 5. Add explicit turn budget and scope limits — Est. savings: **~200–400K tokens/run**

The prompt currently has no hard limits on exploration depth. Adding explicit constraints prevents runaway behavior.

**Action**: Add to the prompt:
```
## Constraints
- Analyze at most 20 commits (oldest first if more exist)
- Do not read more than 10 pull requests
- Do not use search_repositories
- Do not read issues
- The only repository that matters is the current one

---

Tool Usage Matrix (2026-04-07 run)

Tools observed during the run window (2026-04-07T10:29–10:50 UTC):

Tool	Server	Calls	% of Total	Needed?
search_repositories	github	342	58%	❌ Never — remove from toolset
issue_read	github	232	39%	❌ Out of scope — restrict
upload_asset	safeoutputs	5	<1%	✅ Used for assets
list_issues	github	1	<1%	⚠️ Probably not needed
add_comment	safeoutputs	2	<1%	✅ Fine
update_issue	safeoutputs	2	<1%	✅ Fine
create_discussion	safeoutputs	1	<1%	✅ Fine
create_pull_request	safeoutputs	1	<1%	✅ Core output
noop	safeoutputs	1	<1%	✅ Fine
push_repo_memory	safeoutputs	1	<1%	✅ Fine

Total tool calls in run window: 588

Agentic Assessments (system-generated)

The run triggered 4 system-level flags:

1. resource_heavy_for_domain (HIGH): "This Repo Maintenance run consumed a heavy execution profile for its task shape. turns=49 tool_types=0 duration=18m42s write_actions=2"

2. poor_agentic_control (MEDIUM): "The run showed signs of broad or weakly controlled agentic behavior. friction=0 execution=exploratory actuation=selective_write"

3. partially_reducible (MEDIUM): "About 95% of this run's turns appear to be data-gathering that could move to deterministic steps. agentic_fraction=0.04 turns=49 — Move data-fetching work to frontmatter steps."

4. model_downgrade_available (LOW): "This Repo Maintenance run may not need a frontier model. domain=repo_maintenance resource_profile=heavy actuation=selective_write"

Run Detail

Field	Value
Run URL	§24076814565
Date	2026-04-07T10:29:53Z
Conclusion	success
Duration	18.7 min
Turns	49
Cache efficiency	49%
Effective tokens	4,273,899
Input tokens	3,838,637
Output tokens	16,405
Cache read	3,696,415
GitHub API calls	6
Behavior	exploratory / selective_write
Agentic fraction	4%

Historical comparison (from snapshot):

Date	Tokens	Turns	Duration
2026-04-06	1,786,466	26	12 min
2026-04-07	3,855,042	49	18.7 min

---

⚠️ Caveats

• Analysis based on 2 runs over 2 days — patterns are consistent but sample size is small
• The Apr 7 run may have been triggered on a Monday (full weekly scan), which could explain higher turn count vs Apr 6 daily scan — however 342 search_repositories calls is anomalous regardless of scan type
• Verify recommendations in a test run before applying permanently
• The search_repositories overuse may be linked to the search tool described in the prompt being misinterpreted by the agent as GitHub-wide search

---

References:

• §24076814565 — Apr 7 run (3.86M tokens)
• §24028498926 — Apr 6 run (1.79M tokens)

Generated by Copilot Token Usage Optimizer · ● 1.7M · ◷

• expires on Apr 14, 2026, 3:21 PM UTC