fix(diff --application): an unobserved agent is not "no change" (#876)

CHANGELOG.md

@@ -8,6 +8,16 @@
 - `diff --application` no longer reads another library's `Agent` or `function_tool` as the OpenAI Agents SDK's, and no longer reports an agent it stopped observing as removed. On speechmatics/speechmatics-academy#142, a LiveKit voice agent (`from livekit.agents import Agent, function_tool`) moved its tools into an `Agent` subclass that passes them through `super().__init__`. The comparison printed `compared` with four `REMOVED agent → …` rows although the same four tools were still bound. It now prints `not_established`: no supported application agent on either side. (#871 follow-up; related #580, #864, #865)
   - **Framework identity.** Discovery and the SDK reader count `function_tool` and `Agent` as the SDK's only when the name is imported from the absolute `agents`/`openai_agents` package, or is not imported at all (the existing reading, unless a wildcard import from another module could supply it). The name is resolved in the scope that uses it, as Python resolves it, so an import in a sibling function never decides it, and a parameter or local assignment of that name is not the SDK's. A name imported from anywhere else — `livekit.agents`, a relative `.agents` package — is another library's. A LiveKit file is no longer an SDK candidate, and a file using both reads only the SDK's symbols. A relative import is no longer any framework's import signal. `scan` of a declared SDK source reads the same way.
   - **Unobserved is not removed.** One side may observe an agent while the other side's same file still assigns or imports its name (`from factory import agent`), or passes it as `name=`, through a construction the reader does not support: an `Agent` subclass, a factory, `Agent[Context](...)` or `.clone()`. That side then records a coverage gap for the agent. The result is `partial`, and its rows are `not_established` with `candidate_change` `removed` or `added`. An agent referenced only in another agent's `handoffs` is not an observed construction. An agent whose name is gone from the file is still an established removal, and rows for other agents are unaffected. No schema changes.
+- `diff --application` no longer reports `compared` with no changes while an agent it could not see gained a tool. On kkmiecik-coder/CRM#5, the quoting agent `Wycena`, built by `return Agent(name="Wycena", tools=NARZEDZIA_WYCENY)` inside a function, gained `wyslij_obraz`; the only agent read was a test double, so the result said `compared` and printed nothing. (#876)
+  - **Every SDK construction is observed or named.** The OpenAI Agents SDK reader read only `name = Agent(...)`. It now also reads `return Agent(...)`, `self.agent = Agent(...)`, agents inline in a list, `Agent("name")` and `Agent[Context](...)`, identified by the literal `name`. An agent assigned to a plain name keeps that name as its identity, so renaming its `name=` or moving it between scopes changes nothing; only a variable name assigned in more than one function or class body (two builders' local `agent`) gives way to each agent's literal `name`, so those are two agents, and a handoff to such a variable reaches its own. What it cannot read is a named limit on that agent, so other agents' rows stand: one identity constructed at two sites that bind different tools (identical constructions are one agent), including a Google ADK agent name; `**` or extra positional arguments; a copy (`clone`, `dataclasses.replace`, `copy.replace`) passing its own `tools`/`handoffs`/`mcp_servers`, or only `**` on a value known to be an agent; and an agent built from a same-module `Agent` subclass, including one made by `type("X", (Agent,), {})`. A construction without a literal name is a limit on its file.
+  - **Changes after construction.** A change to an agent's tools, handoffs or MCP servers — assigning, extending or slicing `x.tools`, a list method on it, `setattr`/`delattr` by name, or a handle `t = x.tools` that is itself changed later (a handle only read, like `len(request.tools)`, is nothing) — is read in the scope where it happens: on an agent the reader constructed (directly, through `self.agent`, or through a module function that returns it) it is a limit on that agent, and a change in place is one on every agent built from the same list object; on a value proven not to be an agent (`Settings()`, a literal, `type(...)()`, the instance's own `self.tools`) it is nothing; on anything else it is a limit on the file. In a module that is not read as an SDK source — a Google ADK source included, whose reader does not follow a change after construction either — a copy is a named limit on it, and so is a change to any object's tools once the module imports anything from the scope, or is itself a Google ADK source — through an alias, a loop, a parameter or a call's result, as in an SDK file — unless the object is plainly not an agent (a literal, or an instance of a class the scope defines that is not an `Agent` subclass); a module that imports nothing from the scope has another library's `.tools`. A module that imports an SDK `Agent` subclass from the scope, directly or through a package that re-exports it (`from app.core import *`), is limited too, never one that only spells its name in a string or imports a vendor class of the same name.
+  - **Google ADK agent subclasses.** A class deriving from an ADK agent class (`class Helper(LlmAgent)`, or a class deriving from that one) is a named limit where it is used: on the module that defines it when that module builds, passes or decorates it, and on every other module in the scope that imports it. An agent built from it was never read, so a tool it gained could leave the result `compared` with no rows. `scan` no longer reports the defining module's tool surface as enumerated when that module uses the class. A subclass nothing uses is not a limit. (Carried over from #880.)
+  - **Rows and verdicts that change.** A construction that used to be unread and silently absent is now either read (rows appear) or named (the result, and a `scan` of the same code, becomes `partial` / `insufficient_evidence` instead of complete). A Google ADK agent name constructed at two sites in one file with differing tools or handoffs is `insufficient_evidence` for `scan` too, where it could be `passed` or `review_required`; constructions binding the same definitions and handoffs, each read cleanly, are one agent and keep their rows and verdict. A copy passing no capabilities of its own, and a subclass nothing uses, are not limits.
+  - **No agent source, no census.** The census of copies, changes after construction and subclasses runs only when some side's discovery found an OpenAI Agents SDK or Google ADK source. Without one no agent is read, so `--scope src` of this repository stays `not_established` instead of turning `partial` over its own `artifacts.tools.append(record)`; every module is still parsed, and a parse failure is still a gap.
+  - **Test files are not the application.** Test files (the discovery convention, relative to the scope: `test`/`tests` directories, `test_*.py`, `*_test.py`, `conftest.py`, `test.py`, `tests.py`) are listed per side in `excluded_tests`, printed in the text output, and not read as agent sources, so a test double cannot establish a scope and a test file's own defects are not application gaps.
+  - **A duplicate tool definition limits one name.** A file defining one tool name twice refused the whole comparison with exit 2, and the message asked for the test file to be edited. It is now a named limit on that name in that file: no agent binds either definition, and the file's other tools and every other file are still compared. Seen on alliance-genome/agr_ai_curation#842 and usestrix/strix#1103.
+  - **One identity built twice keeps what both sites bind.** When one agent name is constructed at two sites (tensorflow#128063's root agent and builder), a tool both sites bind to the same callable is still a row of that agent, beside the limit that names the constructions; only a tool they bind differently is an ambiguous identity.
+  - Additive on the unreleased `0.1` advisory result (`excluded_tests` per side); no schema or contract bump.
 - `diff --application` no longer refuses a repository over a link or submodule its application reader never opens, and reads a Google ADK agent imported from the package root. (#871 follow-up, measured for #868)
   - **The problem.** On 2026-09-25, against open third-party pull requests that edit SDK/ADK tool wiring, 15 of the first 27 runs at the default root scope exited 2 with `Git tree contains unsupported external binding`. The path named was never application source: `CLAUDE.md -> AGENTS.md` (dlt-hub/dlt#4417, jaegertracing/jaeger#9636, omnigent-ai/omnigent#6611, wandb/weave#7948), a linked `.claude/skills/…` or `.agents/skills/…` directory (asterinas#3834, vllm#57322, kagent-dev/kagent#2788), `agent/VERSION -> ../../VERSION` (TencentCloud/CubeSandbox#1508), `.pylintrc` (tensorflow#128063), and a vendored submodule (temporalio/sdk-python#1868). The root scope took the unscoped archive route, which refuses every link and gitlink. `from google.adk import Agent` was not read as an agent, so CubeSandbox#1508's `root_agent` gave `not_established` ("No supported application agents were established").
   - **Links.** The root scope now uses the scoped materializer a named scope already used, which recreates each link as a link. A link is never read through. Every link under the scope is censused, a dangling one included, and gapped only where it can hide application source: a `*.py` link whose target is not a Python input the scope already reads (an alias of one is compared at the target's path; reading it too made one agent two ambiguous ones and hid the real file's change, as a named scope did on main); a link to a directory outside the scope that holds Python; and a link resolving to nothing in the repository where the other side reads source at or beneath its path. Replacing `agent.py` with a dangling link, or a source directory with an absolute link, is therefore `not_established`, never a removal. A link Python discovery would not read changes nothing, and a scope that is itself a link is still refused. The host-configuration census, which counts every link that could conceal a *host* path, no longer adds application coverage gaps.

docs/application-comparison.md

@@ -68,13 +68,121 @@ artifact from the existing host diff JSON and verifier receipt.
 - `not_established`: neither side established a supported application agent.
 - Exit 2: refs/materialization/input could not be read. This is not no change.
 
+Every OpenAI Agents SDK `Agent(...)` construction in a file the scope reads is
+either an observed agent or a named limit. An agent assigned to a plain name
+is identified by that name (`assistant = Agent(...)` is `assistant`), as
+before, so renaming its `name=` or moving it into a builder changes nothing.
+Only a variable name assigned in more than one function or class body — two
+builders' local `agent` — gives way to each agent's literal `name`, and a handoff to such a
+variable is named by the agent it holds. Any other construction is identified
+by its literal `name`: `return Agent(name="Quote", ...)` inside a factory
+function, `self.agent = Agent(name="Held", ...)`, an agent inline in a list,
+or `Agent("Positional")`. `Agent[Context](...)` is the same construction.
+
+What the reader cannot read is a named limit on the agent it concerns, so other
+agents' rows in the file stand:
+
+- one identity constructed at more than one site in a file with different
+  tools or handoffs (two `return Agent(name="Quote", ...)` branches, or a
+  literal `name` equal to another agent's variable) — the binding graph would
+  merge them, so which construction binds which tool is not established;
+  constructions that bind exactly the same tools and handoffs are one agent.
+  A tool every such construction binds identically is still a row of that
+  agent, beside the limit. The same holds for a Google ADK agent name: its
+  constructions are one agent only when each binds the same tool definitions
+  and handoffs and was read cleanly (no toolset, no `**`, no warning, no
+  unresolved or non-literal `sub_agents`), as `root_agent` and a builder
+  returning its twin do;
+- a construction with `**` keyword unpacking or positional arguments after its
+  name, which can carry `tools` or `handoffs`;
+- an agent whose `tools`, `handoffs` or `mcp_servers` are changed after
+  construction: assigned, extended or sliced (`agent.tools.append(...)`,
+  `agent.tools = [...]`, `agent.tools[:] = ...`), `setattr`/`delattr` by name,
+  or a handle taken on them (`t = agent.tools`) that is itself changed later —
+  a handle only read, like `len(request.tools)`, is nothing. The changed value is read in
+  the scope of the change (a change in place reaches every agent built from the
+  same list object): an agent the reader constructed — directly, through
+  `self.agent`, or through a module function that returns it — carries the
+  limit; a value proven not to be an agent (`Settings()`, a literal,
+  `type(...)()`, the instance's own `self.tools`) carries nothing; anything
+  else is a limit on the file;
+- a copy that passes its own `tools`, `handoffs` or `mcp_servers`
+  (`agent.clone(tools=...)`, `dataclasses.replace(agent, tools=...)`,
+  `copy.replace(...)`), wherever the original came from, unless the original
+  is proven not to be an agent. A copy passing only `**` is a limit only on a
+  value known to be an agent. A copy that passes none of them keeps the
+  original's tools, which the original's rows already compare, and is not a
+  limit;
+- an agent built from a subclass of the SDK's `Agent` defined in the same
+  module, whose tools arrive through its constructor, including one made with
+  `type("X", (Agent,), {})`.
+
+A construction whose `name` is not a literal is a limit on its file.
+
+A module that is not read as an SDK source is still read for what can change
+an agent without it: a capability-passing copy, and — once the module imports
+anything from the scope, or is a Google ADK source, whose reader does not
+follow a change after construction — a change to any object's tools, handoffs, MCP
+servers or sub-agents, reached by an alias, a loop, a parameter or a call's
+result as in an SDK file, unless the object is plainly not an agent (a literal,
+or an instance of a class the scope defines that is not an `Agent` subclass).
+A module that imports nothing from the scope has another library's `.tools`.
+In any module, importing an SDK `Agent` subclass from the scope — `from core
+import Assistant`, `core.Assistant` after `import core`, or through a package
+that re-exports it with `from app.core import *` — is a limit; a subclass
+nothing imports, a vendor class of the same name, or a name that only appears
+in a string is not. Each is a limit on the module where it appears. The scope's
+own package path counts as the scope (`from svc.app.x import …` under
+`--scope svc/app`).
+
+The Google ADK reader reads each `Agent(...)` / `LlmAgent(...)` call, not a
+subclass's constructor. A class deriving from an ADK agent class (`class
+Helper(LlmAgent)`, its base imported from `google.adk`, or a class deriving
+from that one) is therefore a named limit where it is used: on the module that
+defines it when that module names it again — a call, `functools.partial`, any
+other reference, or a decorator that could build it — and that module's tool
+surface is then not reported as enumerated to `scan` either; and on every other
+module in the scope that imports it, as for an SDK subclass. A subclass nothing
+uses, even one another unused subclass derives from, is not a limit. `scan`
+reads one declared module, so a subclass used only in another module does not
+affect it.
+
+The census of copies, changes and subclasses runs only when some side's
+discovery found an OpenAI Agents SDK or Google ADK source. Without one, no
+agent is read on either side, and the result stays `not_established` rather
+than turning `partial` over the repository's own `.tools` (a report builder's
+`artifacts.tools.append(record)`). Every module is still parsed, and one that
+cannot be is still a gap.
+
+Not read at all: an `Agent` re-exported through a project module,
+`functools.partial(Agent, ...)`, or a subclass defined outside the scope.
+
 An agent one side observes is absent from the other only when that side's
 file no longer names it. If the file still assigns or imports the agent's name
-(`from factory import agent`), or passes it as `name=`, through a construction the reader does not support (an `Agent`
-subclass passing `tools` through `super().__init__`, a factory,
-`Agent[Context](...)`, `.clone()`), that side records a gap for the agent and
-its rows are `not_established`, never `removed` or `added`. An agent referenced
-only in another agent's `handoffs` is not an observed construction.
+(`from factory import agent`, `agent = build()`), or passes it as `name=`,
+through a construction the reader does not read (an `Agent` subclass, a
+`.clone()`), that side records a gap for the agent and its rows are
+`not_established`, never `removed` or `added`. A factory's own `return
+Agent(name="assistant", ...)` is read under that literal name in the module
+that builds it; it is a construction site of its own, not evidence about the
+name the factory's result is assigned to. An agent referenced only in another
+agent's `handoffs` is not an observed construction.
+
+Test files never establish the application. A file under a `test` or `tests`
+directory, a `test_*.py` or `*_test.py` module, `conftest.py`, `test.py` or
+`tests.py` — matched case-sensitively and relative to the selected scope, the
+convention discovery uses — is listed in each side's `excluded_tests`, printed
+in the text output, and not read as an agent source: a test double's agent is
+not the application's agent, so it cannot make a scope count as established,
+and a test file's own defects (a tool defined twice, an unsupported framework,
+a parse failure) are not the application's gaps. A product module that only
+looks like a test is excluded too, which the printed list makes visible; the
+import resolver still follows a tool the application imports from such a file.
+
+A file that defines one tool name twice is a named limit on that name in that
+file: no agent binds either definition, so every binding of the name there is
+`not_established`, while the file's other tools, and every other file in the
+scope, are still compared.
 
 Framework identity follows the import, not the spelling. `Agent` and
 `function_tool` are the OpenAI Agents SDK's only when imported from the absolute