Skip to content

fix(copilot): route Claude models through Copilot's native anthropic provider - #495

Draft
larsmoan wants to merge 1 commit into
databricks:mainfrom
larsmoan:fix/copilot-anthropic-prompt-caching
Draft

fix(copilot): route Claude models through Copilot's native anthropic provider#495
larsmoan wants to merge 1 commit into
databricks:mainfrom
larsmoan:fix/copilot-anthropic-prompt-caching

Conversation

@larsmoan

@larsmoan larsmoan commented Sep 4, 2026

Copy link
Copy Markdown

Fixes #494.

Claude models routed through Copilot CLI get zero Anthropic prompt-cache
hits, because ucode always sets COPILOT_PROVIDER_TYPE=openai and points
Copilot at the OpenAI-compatible MLflow gateway — a wire format with no field
for a cache_control breakpoint — even for Claude models. Every other agent
ucode configures (claude, opencode, pi) already gets Claude traffic
routed to Databricks' native /ai-gateway/anthropic path; Copilot did not.

Copilot CLI documents (and its shipped runtime implements) a native
COPILOT_PROVIDER_TYPE=anthropic provider — see issue #494 for the
runtime-string evidence (prompt_cache_request.rs, attemptCacheControl,
ephemeral_5m_input_tokens, etc.) that this path actually inserts
cache_control breakpoints. This PR uses it.

Changes

  • databricks.py: build_copilot_base_url (single URL) → build_copilot_base_urls
    (dict with anthropic and openai entries), same pattern as
    build_opencode_base_urls/build_pi_base_urls. build_shared_base_urls()["copilot"]
    is now a dict, matching the existing precedent for opencode/pi.
  • agents/copilot.py: render_env_overlay now branches on whether the
    selected model is a Claude model ("claude" in model.lower(), same
    substring check databricks.py already uses elsewhere for this family):
    • Claude models → COPILOT_PROVIDER_TYPE=anthropic,
      COPILOT_PROVIDER_BASE_URL = the native anthropic gateway,
      COPILOT_PROVIDER_API_KEY (Copilot's native-Anthropic auth var, sent as
      x-api-key — same header Claude Code's apiKeyHelper already
      authenticates with against this same endpoint).
    • Everything else (codex/gpt-5) → unchanged: COPILOT_PROVIDER_TYPE=openai,
      MLflow gateway, COPILOT_PROVIDER_BEARER_TOKEN.
    • write_tool_config now clears whichever of
      COPILOT_PROVIDER_API_KEY/COPILOT_PROVIDER_BEARER_TOKEN doesn't apply
      before writing, so switching the default model between families doesn't
      leave a stale auth key in ~/.copilot/ucode.env.
  • Tests updated/added in test_agent_copilot.py and test_databricks.py.

Not yet verified

I don't have a live workspace handy to confirm system.ai_gateway.usage
actually shows cache_read_input_tokens > 0 for Copilot traffic after this
change — the auth-header compatibility (x-api-key against
/ai-gateway/anthropic) is inferred from how claude.py already
authenticates against the same endpoint, not directly tested against
Copilot's client. Opening as a draft for that reason — happy to run the e2e
suite (tests/test_e2e.py::...test_launch_copilot_per_model) against a real
workspace before marking ready.

Test plan

  • uv run pytest tests/test_agent_copilot.py tests/test_databricks.py -q
  • uv run pytest -q (full suite; one pre-existing unrelated failure in
    test_e2e_user_agent.py reproduces identically on main)
  • uv run ruff check .
  • Confirm real cache hits in system.ai_gateway.usage for a Copilot +
    Claude session against a live workspace

…provider

ucode always configured Copilot CLI with COPILOT_PROVIDER_TYPE=openai,
pointing even Claude models at the OpenAI-compatible MLflow gateway. That
wire format has no field for an Anthropic cache_control breakpoint, so
Copilot's own prompt-cache logic (confirmed present in its runtime) never ran
and gateway usage data showed zero cache hits on Claude traffic.

Claude models now get COPILOT_PROVIDER_TYPE=anthropic against the same
native /ai-gateway/anthropic path claude.py uses, authenticated via
COPILOT_PROVIDER_API_KEY (x-api-key) instead of the bearer-token var used for
the openai path. Codex (gpt-5) keeps the openai provider against the MLflow
gateway, since it has no native-dialect option on Copilot's side.

Fixes databricks#494
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Copilot CLI gets zero prompt-cache hits on Claude models — ucode forces the openai provider type instead of Copilot's native anthropic one

1 participant