docs: extend AGENTS.md with conventions mined from PR reviews

[julian-risch]

Related Issues

• none

Proposed Changes:

Our AGENTS.md documents the mechanics — hatch per integration, tests, mypy, versioning — but none of the judgement reviewers apply across 100+ integration packages. That knowledge lives only in review comments.

This adds 129 rules mined with pydantic/braindump (see Scaling open source with AI) from 2,772 PR review comments written by deepset reviewers between 2025-07-01 and 2026-08-14. The hand-written sections of AGENTS.md are untouched; the mined rules follow underneath.

Layout. Everything lives in one root AGENTS.md (129 rules, ~5.1k tokens), with CLAUDE.md reduced to a one-line @AGENTS.md import.

An earlier revision split this across integrations/AGENTS.md plus two agent_docs/*.md topic guides, which would have cut the always-loaded size to ~2.1k. That was reverted for two reasons:

• Topic guides are not memory files. The root AGENTS.md only links to them, so nothing loads them automatically — an agent has to choose to follow the link. Verified with canary fixtures: with file tools blocked the topic content is absent 3/3 while the root file is present. With tools available the agent did follow the link 3/3, but that test named docstrings explicitly and so matched the link's "when to check" text almost word for word; a vaguer real task matches less well.
• The split bought little here. Nearly all work in this repo happens under integrations/, so the lazily-loaded file would have loaded almost always anyway.

The cost is ~5.1k tokens in every session rather than ~2.1k. The benefit is that no rule can be missed. The companion core PR keeps a nested layout, where rules divide cleanly by directory (test/, releasenotes/notes/, docs-website/) and group produced no topic files.

Claude Code reads CLAUDE.md, not AGENTS.md (docs), so the import is what puts the guidance in context. Codex and Cursor read AGENTS.md directly, so it stays the single source of truth.

Verified with canary fixtures (claude -p, file tools disabled so the model can only answer from loaded context):

Setup	Canary in context?
CLAUDE.md = @AGENTS.md	yes
AGENTS.md alone, no CLAUDE.md	no — Claude Code does not read AGENTS.md on its own
CLAUDE.md = the old prose, tools disabled	no — nothing is preloaded
nested sub/CLAUDE.md = @AGENTS.md, cwd at root, after reading sub/thing.py	yes — loads lazily, which is what the split relies on
nested, cwd inside sub/	yes

For fairness: with Read enabled the old prose form did work — the model went and read AGENTS.md in 3/3 runs. The import's advantage is that it is deterministic and costs no tool-call round-trip, not that the old form was broken.

2,627 candidate generalisations collapsed into 379 deduplicated rules, of which 129 survived scoring and review. The clustering does useful work in a monorepo: conventions that recur across integrations (serialisation symmetry with __init__, warm_up() lifecycle, Secret resolution, filter policy, sync/async parity, protocol parameter order) generalise and survive, while per-integration one-offs score too low to appear. place assigned most rules to the root or to integrations/ rather than to individual packages, which is the outcome we want.

Handling the 3.0 boundary. The corpus mostly predates Haystack 3.0 (2026-07-20), so rules were filtered against the current tree before inclusion. This repo is the interesting case: it removed almost nothing of its own — it gained the ~45 component classes that moved out of core. So its staleness set is inherited from core rather than computed locally: guidance about APIs that left haystack and landed here stays valid, guidance about APIs that disappeared entirely does not. Verdicts: 343 current, 34 rewritten, 2 stale. A recency weighting demotes rules whose only supporting evidence is old.

How did you test it?

Markdown-only change, so no test suite applies. Verified instead:

• No stale rule leaked. Every rule joined back to its verdict: 119 current, 11 rewritten, 0 stale, 0 unknown ids.
• Symbol sanity. All 165 backticked identifiers in the block resolve against the current integrations/*/src/ tree — zero removed or relocated names.
• Stale bucket read by hand (2 rules).
• Extraction quality gated by a 50-comment smoke test before the full run.

Notes for the reviewer

• What to actually review: the rules themselves. A rule you would not enforce in review should not be here. The files carry no provenance metadata: judge each rule on whether it reads as a convention this project actually holds, not on where it came from.
• Five rules were generalised on review because clustering had pinned them to whichever integration the source comments discussed: WATSONX_API_KEY became "default each Secret from the provider's conventional env var" (it is one of ~10 such variables here), Elasticsearch bulk-write handling became "preserve documented bulk write/delete error behaviour" (DocumentStoreError is used by 17 stores), and the RagasEvaluator/Google GenAI example list became "refresh docs when model names or provider APIs change". One rule scoped entirely to integrations/amazon_bedrock/tests/ was dropped. Please flag any others that still read as too narrow for a repo-wide file.
• The rules are weighted toward the highest-volume reviewers, so they reflect their emphases more than the team average.
• Verification limitation worth knowing: every recent PR is inside the corpus, so I could not test against held-out reviews.
• Companion PR against deepset-ai/haystack does the same for core.

Checklist

• I have read the contributors guidelines and the code of conduct
• I have updated the related issue with new insights and changes
• I added unit tests and updated the docstrings — documentation-only change.
• I've used one of the conventional commit types for my PR title: docs:.

🤖 Generated with Claude Code