Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,12 @@
- `diff --application` no longer reads another library's `Agent` or `function_tool` as the OpenAI Agents SDK's, and no longer reports an agent it stopped observing as removed. On speechmatics/speechmatics-academy#142, a LiveKit voice agent (`from livekit.agents import Agent, function_tool`) moved its tools into an `Agent` subclass that passes them through `super().__init__`. The comparison printed `compared` with four `REMOVED agent → …` rows although the same four tools were still bound. It now prints `not_established`: no supported application agent on either side. (#871 follow-up; related #580, #864, #865)
- **Framework identity.** Discovery and the SDK reader count `function_tool` and `Agent` as the SDK's only when the name is imported from the absolute `agents`/`openai_agents` package, or is not imported at all (the existing reading, unless a wildcard import from another module could supply it). The name is resolved in the scope that uses it, as Python resolves it, so an import in a sibling function never decides it, and a parameter or local assignment of that name is not the SDK's. A name imported from anywhere else — `livekit.agents`, a relative `.agents` package — is another library's. A LiveKit file is no longer an SDK candidate, and a file using both reads only the SDK's symbols. A relative import is no longer any framework's import signal. `scan` of a declared SDK source reads the same way.
- **Unobserved is not removed.** One side may observe an agent while the other side's same file still assigns or imports its name (`from factory import agent`), or passes it as `name=`, through a construction the reader does not support: an `Agent` subclass, a factory, `Agent[Context](...)` or `.clone()`. That side then records a coverage gap for the agent. The result is `partial`, and its rows are `not_established` with `candidate_change` `removed` or `added`. An agent referenced only in another agent's `handoffs` is not an observed construction. An agent whose name is gone from the file is still an established removal, and rows for other agents are unaffected. No schema changes.
- `diff --application` no longer refuses a repository over a link or submodule its application reader never opens, and reads a Google ADK agent imported from the package root. (#871 follow-up, measured for #868)
- **The problem.** On 2026-09-25, against open third-party pull requests that edit SDK/ADK tool wiring, 15 of the first 27 runs at the default root scope exited 2 with `Git tree contains unsupported external binding`. The path named was never application source: `CLAUDE.md -> AGENTS.md` (dlt-hub/dlt#4417, jaegertracing/jaeger#9636, omnigent-ai/omnigent#6611, wandb/weave#7948), a linked `.claude/skills/…` or `.agents/skills/…` directory (asterinas#3834, vllm#57322, kagent-dev/kagent#2788), `agent/VERSION -> ../../VERSION` (TencentCloud/CubeSandbox#1508), `.pylintrc` (tensorflow#128063), and a vendored submodule (temporalio/sdk-python#1868). The root scope took the unscoped archive route, which refuses every link and gitlink. `from google.adk import Agent` was not read as an agent, so CubeSandbox#1508's `root_agent` gave `not_established` ("No supported application agents were established").
- **Links.** The root scope now uses the scoped materializer a named scope already used, which recreates each link as a link. A link is never read through. Every link under the scope is censused, a dangling one included, and gapped only where it can hide application source: a `*.py` link whose target is not a Python input the scope already reads (an alias of one is compared at the target's path; reading it too made one agent two ambiguous ones and hid the real file's change, as a named scope did on main); a link to a directory outside the scope that holds Python; and a link resolving to nothing in the repository where the other side reads source at or beneath its path. Replacing `agent.py` with a dangling link, or a source directory with an absolute link, is therefore `not_established`, never a removal. A link Python discovery would not read changes nothing, and a scope that is itself a link is still refused. The host-configuration census, which counts every link that could conceal a *host* path, no longer adds application coverage gaps.
- **Submodules.** A gitlink under the scope is materialized as the empty directory a checkout without `--recurse-submodules` leaves; its content is never fetched. The same gitlink commit on both sides is named in `limits` as unchanged and does not make the comparison `partial`, since identical content cannot carry a change. An added, removed or moved gitlink is a coverage gap over its path on each side that has it, and over the whole scope when it is the selected scope itself, so the result is `partial`, never `compared`. A gitlink was refused (exit 2) before.
- **Google ADK.** `google.adk.Agent`, the package-root re-export of `google.adk.agents.llm_agent.Agent`, is read as an agent constructor by the ADK reader, for `from google.adk import Agent` and `import google.adk as adk` alike. CubeSandbox#1508 now establishes `cube_code_agent` and is `partial`, naming the tool it imports from a sibling module (#864), instead of `not_established`.
- **Re-measured.** At the root scope, main `a430e81a` refused CubeSandbox#1508, dlt#4417 and asterinas#3834; this change compares each, with its own limits (a Python census past `--max-python-files`, an unsupported framework, an unresolved import). No application comparison schema change; `application_comparison_schema_version` stays `"0.1"`.

### Changes

Expand Down
46 changes: 44 additions & 2 deletions docs/application-comparison.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,9 @@ requested and compared refs/tree IDs, per-side scope/coverage, rows, source
correspondence, and a deterministic `comparison_id`. This is a separate advisory
artifact from the existing host diff JSON and verifier receipt.

- `compared`: the selected supported source observations were compared.
- `compared`: the selected supported source observations were compared. An
unchanged submodule may still be named in `limits` (see below); it cannot
carry a change, so it does not make the comparison `partial`.
- `partial`: a parse/discovery/binding gap remains. Gaps identify their source
and agent where known. Only affected candidates become `change: not_established`
rows, carrying `candidate_change` and per-side `uncertainty`; independent known
Expand Down Expand Up @@ -106,7 +108,47 @@ object with that key removed, encoded as UTF-8 JSON with sorted keys,
`separators=(",", ":")` and `ensure_ascii=True`; readers can recompute it.

Each side materializes only its selected scope through the existing verified
Git materializer, which retains symlinks and containment checks. Partial clones
Git materializer, which retains symlinks and containment checks. The default
root scope goes through the same scoped materializer, so it packs the tree
rather than the history.

### Links and submodules

A symlink anywhere in the tree is recreated as the link it is, never refused
and never read through. Every link under the scope is censused, including one
that resolves to nothing, and what it can hide decides what it is:

- A link Python discovery would not read changes nothing, as in a checkout:
`CLAUDE.md -> AGENTS.md`, or a linked `.claude/skills/…` directory whose
target the scope already reads at its own path.
- A `*.py` link whose target is a Python input the scope already reads is
compared at that target's path. Any other `*.py` link — dangling, leaving
the scope or the repository, or landing on something that is not a Python
input — is a coverage gap over the link's own path
(`Linked Python input: agent.py`), so replacing `agent.py` with a dangling
link is `not_established`, never a removal.
- A link to a directory outside the scope that holds Python is a gap over the
link's path (`Linked directory holds Python outside the scope: lib`).
- A link that resolves to nothing in the repository (dangling, absolute, or
leaving the tree) is a gap only where the other side reads source at or
beneath its path (`Linked input resolves outside the tree: tools`). An
unchanged `agent/VERSION -> ../../VERSION` beside the application changes
nothing.

A selected scope that is itself a link is refused (exit 2).

A submodule's content is in another repository. The comparison never fetches
it and materializes its gitlink as the empty directory a checkout without
`--recurse-submodules` leaves. The same gitlink commit on both sides is the
same content, so it is named in both sides' `limits` —
`Submodule content is not read (unchanged commit 85b71d7ecd4f): vendor/core` —
and nothing more. An added, removed or moved gitlink is a coverage gap over its
path on each side that has it, so the comparison is `partial`, and a binding
the other side holds at that path is `not_established` rather than added or
removed. A gitlink at the selected scope itself covers the whole scope
(`…: the selected scope`).

Partial clones
with unfetched objects exit 2 with `objects_missing`, name the affected side,
and provide the existing `git fetch --refetch --no-filter <remote>` recovery.
The comparison never runs that fetch. Other configuration/materialization errors
Expand Down
2 changes: 1 addition & 1 deletion docs/distribution-surfaces.md

Large diffs are not rendered by default.

Loading
Loading