Skip to content

Add hosted web search for Gemini, OpenAI, and Bedrock Mantle - #108

Open
wch wants to merge 6 commits into
mainfrom
openai-gemini-search
Open

Add hosted web search for Gemini, OpenAI, and Bedrock Mantle#108
wch wants to merge 6 commits into
mainfrom
openai-gemini-search

Conversation

@wch

@wch wch commented Sep 5, 2026

Copy link
Copy Markdown
Collaborator

This is needed for https://github.com/posit-dev/assistant/pull/2364.

This PR adds provider-executed web search to three model clients, so applications built on ai-lib can offer server-side search with streamed, clickable citations instead of routing every lookup through a local fetch tool:

  1. Google-hosted Gemini models can attach Google Search grounding. Only an explicit list of verified Gemini 3.x models advertises the capability — discovery stays optimistic about new models, but the search toggle does not, because Gemini 2.5 rejects built-in search combined with function calling (confirmed against the live API) and unverified models could fail loudly or answer without grounding.
  2. The OpenAI client can attach the hosted web_search tool on the Responses API, and the Bedrock client can attach the same tool for supported GPT models on the Bedrock Mantle Responses route, through one shared helper.
  3. The model catalog now computes the final "supports web search" flag only after configuration overrides and routing are resolved, so every host makes the same decision from the same inputs.

For Mantle, requests always carry an explicit external_web_access boolean — whether a cache miss may fetch a live external page. AWS defaults an omitted value to true, which would silently widen the egress boundary and can also trip IAM policies that grant Search and Fetch but not external access, so the new (optional, backward-compatible) request parameter defaults to false.

Requests that ask for search on a route that cannot provide it — Chat Completions, Databricks MLflow Responses, Bedrock Converse or Anthropic Messages, Mantle Chat Completions — are rejected before any network call rather than silently dropping the capability. Custom OpenAI endpoints require an explicit per-model opt-in; redirected built-in OpenAI endpoints default off. Redirected built-in Gemini endpoints and custom Gemini providers never advertise search, because the Gemini client hardcodes Google's hosted Interactions wire contract. Mantle search additionally requires a documented GPT model family, a web-search-enabled AWS region, and a known non-FIPS transport — an unknown FIPS state withholds the capability rather than advertising a request that transport resolution may later veto. A local tool that occupies the provider tool's reserved name is rejected with a diagnostic naming both tools instead of being silently overwritten.

Design decisions

  • Capability is gated on the resolved endpoint, not the registrar that discovered the model. Built-in Gemini credentials can carry a redirected base URL, so "the built-in registrar ran" is not evidence that Google's endpoint will serve the request. The same finalization seam applies OpenAI's canonical/redirected/custom rules and Bedrock's model, region, route, and FIPS gates, and an explicit opt-in can never bypass service-side gates.
  • One merge helper owns the provider-tool key reservation. Both the Anthropic web_search merge (pre-existing) and the new OpenAI and Google merges go through it, which fixes a silent-overwrite bug: previously a local tool named web_search would have been replaced without any error.
  • The verified-model list for Gemini is deliberate. Being conservative costs a one-line addition per verified model; being optimistic would hand a broken toggle to every future discovered model.
  • Provider-tool merges are computed before cancellation registration in the OpenAI and Bedrock clients, so a synchronously rejected name collision cannot leak the abort subscription.

Not included

  • Vertex and custom Gemini endpoints report no search capability; the Vertex client would need its own implementation first.
  • A failed Gemini search renders as an empty successful result: the pinned @ai-sdk/google drops the wire's is_error flag before emitting the tool result, so no consumer can detect the failure until that upstream fix lands.

Verification

Wire-level tests capture the actual request bodies: Google Search serialized as tools: [{ type: "google_search" }] on the Interactions API (search-only and search-plus-function-tool shapes, which differ in tool_choice handling), { type: "web_search" } with OpenAI defaults on the Responses API, and SigV4-signed Mantle requests with external_web_access serialized as both false and true. Capability tests cover the override, route, endpoint, model-family, region, and FIPS boundaries. The Gemini and OpenAI paths were also smoke-tested live, with grounded answers and citation payloads captured via raw HTTP logging.

wch added 6 commits September 5, 2026 13:22
SDK-level evidence that { google_search: googleSearch({}) } serializes to
tools: [{ type: 'google_search' }] on POST /v1beta/interactions, that
tool_choice is omitted on a search-only request, and that it survives as
generation_config.tool_choice when a local function tool accompanies search.
GeminiClient attaches the google_search provider tool when web search is
explicitly enabled per-request, merged through mergeProviderTools so a
local tool occupying the reserved key is rejected. The catalog advertises
supportsWebSearch only on the Google-hosted endpoint and only for an
explicit verified-model list (3.5/3.6/3.7/3.8 Flash and 3.5 Flash-Lite);
custom Gemini endpoints, Vertex, Gemma, Gemini 2.5, and unverified future
IDs stay false.
OpenAIClient and BedrockClient attach the OpenAI Responses web_search
provider tool through a shared helper layered on mergeProviderTools, with
enabled search rejected on Chat Completions and non-Mantle Bedrock routes
before any request. BedrockClient serializes an explicit
external_web_access boolean (default false) so Mantle's egress boundary
never depends on the AWS default.

ai-config finalizes supportsWebSearch after overrides and routing are
resolved: built-in OpenAI defaults on only at the canonical endpoint,
redirected and custom OpenAI endpoints require explicit opt-in, and
Bedrock applies the documented Mantle GPT family, Responses-route, region
allowlist, and FIPS gates so no override can bypass them. Explicit
supportsWebSearch provenance is preserved through the model-resolution
pipeline until finalization.
Add a Server-Side Web Search section to geminiInteractions.md covering
the google_search tool attachment via mergeProviderTools, the
fail-closed verified-model gate (with the live-verified 2.5 exclusion),
the tool_choice search-only difference, the observed stream shape, and
the is_error upstream limitation. Also add a Search-grounding row to the
two-clients comparison table.
Add the hosted-web-search transport seam (mergeOpenAIWebSearchTool,
per-client route vetoes, Mantle IAM actions and the external_web_access
egress boundary) to architecture.md, and the web-search capability
finalization seam (resolveWebSearchServing/finalizeWebSearchCapability,
resolveModels' fourth parameter) to aiConfig.md.
- Gemini: finalize supportsWebSearch on the resolved endpoint, not the
  registrar that discovered the model. Only the built-in gemini provider
  on Google's canonical endpoint keeps its (verified-model-gated)
  capability; redirected built-ins and custom Gemini providers resolve
  to false regardless of overrides.
- Mantle: treat an unknown AWS FIPS state as ineligible (fail closed)
  instead of advertising a capability request-time transport resolution
  may veto.
- OpenAIClient: reject webSearchEnabled on the mlflow-responses route —
  it shares the Responses wire shape but not OpenAI's hosted tools.
- OpenAIClient/BedrockClient: compute the provider-tool merge before
  registering cancellation, so a synchronously rejected name collision
  cannot leak the abort subscription.

Addresses review findings in posit-dev/assistant#2364.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant