Skip to content

fix(gemini): strip Google model prefix - #979

Open
kywch wants to merge 3 commits into
mainfrom
fix/975-gemini-prefix
Open

fix(gemini): strip Google model prefix#979
kywch wants to merge 3 commits into
mainfrom
fix/975-gemini-prefix

Conversation

@kywch

@kywch kywch commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

Summary

Normalizes google/gemini-* to the bare Gemini model ID only when configuring the Gemini ACP agent.

Root cause

Gemini CLI expects a bare model name. BenchFlow retained the models.dev google/ prefix, causing Gemini CLI to construct an invalid models/google/gemini-* path.

Impact

Provider-qualified Gemini model IDs work with the Gemini ACP agent; provider/model agents retain their existing prefix behavior.

Validation

  • uv run python -m pytest tests/test_acp.py -k 'TestConnectAcpModelSelection'
  • uv run ruff check src/benchflow/acp/runtime.py tests/test_acp.py
  • git diff --check

Fixes #975.

@kywch
kywch deployed to pypi-internal-preview August 12, 2026 17:18 — with GitHub Actions Active
@kywch
kywch had a problem deploying to pypi-internal-preview August 12, 2026 17:24 — with GitHub Actions Failure
@kywch
kywch deployed to pypi-internal-preview August 12, 2026 17:30 — with GitHub Actions Active
@kywch
kywch marked this pull request as ready for review August 12, 2026 17:42
@kywch
kywch deployed to pypi-internal-preview August 12, 2026 17:43 — with GitHub Actions Active
@Galius5136

Copy link
Copy Markdown

I reproduced the model-prefix behavior from #975 locally and compared the merge-base (29c64e7), the isolated Gemini fix commit (771be99), and the current PR head (bc9a1c3).

On the merge-base, gemini + google/gemini-2.5-pro stays as google/gemini-2.5-pro. With the Gemini fix, it becomes gemini-2.5-pro, and the current PR head behaves identically to the isolated fix commit.

I also checked the surrounding non-regression cases: an already-bare Gemini model stays bare, OpenCode still maps gemini-2.5-pro to google/gemini-2.5-pro, and google/gemma-* is left unchanged for Gemini.

The ACP test suite passed on my setup (94 passed, 1 skipped), and the four files touched by the PR pass both ruff check and ruff format --check.

Everything behaved as expected in the offline path I tested. I didn't run a live Gemini CLI call because I don't have a GEMINI_API_KEY, so I didn't independently verify the bare model ID against the real Gemini CLI.

@kywch

kywch commented Aug 13, 2026

Copy link
Copy Markdown
Collaborator Author

Thanks @Galius5136! I found other gemini issues as well, so I'd probably update on this thread.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Gemini ACP passes google/ provider prefixes into Gemini CLI model IDs

2 participants