Add hosted web search for Gemini, OpenAI, and Bedrock Mantle - #108
Open
wch wants to merge 6 commits into
Open
Conversation
SDK-level evidence that { google_search: googleSearch({}) } serializes to
tools: [{ type: 'google_search' }] on POST /v1beta/interactions, that
tool_choice is omitted on a search-only request, and that it survives as
generation_config.tool_choice when a local function tool accompanies search.
GeminiClient attaches the google_search provider tool when web search is explicitly enabled per-request, merged through mergeProviderTools so a local tool occupying the reserved key is rejected. The catalog advertises supportsWebSearch only on the Google-hosted endpoint and only for an explicit verified-model list (3.5/3.6/3.7/3.8 Flash and 3.5 Flash-Lite); custom Gemini endpoints, Vertex, Gemma, Gemini 2.5, and unverified future IDs stay false.
OpenAIClient and BedrockClient attach the OpenAI Responses web_search provider tool through a shared helper layered on mergeProviderTools, with enabled search rejected on Chat Completions and non-Mantle Bedrock routes before any request. BedrockClient serializes an explicit external_web_access boolean (default false) so Mantle's egress boundary never depends on the AWS default. ai-config finalizes supportsWebSearch after overrides and routing are resolved: built-in OpenAI defaults on only at the canonical endpoint, redirected and custom OpenAI endpoints require explicit opt-in, and Bedrock applies the documented Mantle GPT family, Responses-route, region allowlist, and FIPS gates so no override can bypass them. Explicit supportsWebSearch provenance is preserved through the model-resolution pipeline until finalization.
Add a Server-Side Web Search section to geminiInteractions.md covering the google_search tool attachment via mergeProviderTools, the fail-closed verified-model gate (with the live-verified 2.5 exclusion), the tool_choice search-only difference, the observed stream shape, and the is_error upstream limitation. Also add a Search-grounding row to the two-clients comparison table.
Add the hosted-web-search transport seam (mergeOpenAIWebSearchTool, per-client route vetoes, Mantle IAM actions and the external_web_access egress boundary) to architecture.md, and the web-search capability finalization seam (resolveWebSearchServing/finalizeWebSearchCapability, resolveModels' fourth parameter) to aiConfig.md.
- Gemini: finalize supportsWebSearch on the resolved endpoint, not the registrar that discovered the model. Only the built-in gemini provider on Google's canonical endpoint keeps its (verified-model-gated) capability; redirected built-ins and custom Gemini providers resolve to false regardless of overrides. - Mantle: treat an unknown AWS FIPS state as ineligible (fail closed) instead of advertising a capability request-time transport resolution may veto. - OpenAIClient: reject webSearchEnabled on the mlflow-responses route — it shares the Responses wire shape but not OpenAI's hosted tools. - OpenAIClient/BedrockClient: compute the provider-tool merge before registering cancellation, so a synchronously rejected name collision cannot leak the abort subscription. Addresses review findings in posit-dev/assistant#2364.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is needed for https://github.com/posit-dev/assistant/pull/2364.
This PR adds provider-executed web search to three model clients, so applications built on ai-lib can offer server-side search with streamed, clickable citations instead of routing every lookup through a local fetch tool:
web_searchtool on the Responses API, and the Bedrock client can attach the same tool for supported GPT models on the Bedrock Mantle Responses route, through one shared helper.For Mantle, requests always carry an explicit
external_web_accessboolean — whether a cache miss may fetch a live external page. AWS defaults an omitted value totrue, which would silently widen the egress boundary and can also trip IAM policies that grant Search and Fetch but not external access, so the new (optional, backward-compatible) request parameter defaults tofalse.Requests that ask for search on a route that cannot provide it — Chat Completions, Databricks MLflow Responses, Bedrock Converse or Anthropic Messages, Mantle Chat Completions — are rejected before any network call rather than silently dropping the capability. Custom OpenAI endpoints require an explicit per-model opt-in; redirected built-in OpenAI endpoints default off. Redirected built-in Gemini endpoints and custom Gemini providers never advertise search, because the Gemini client hardcodes Google's hosted Interactions wire contract. Mantle search additionally requires a documented GPT model family, a web-search-enabled AWS region, and a known non-FIPS transport — an unknown FIPS state withholds the capability rather than advertising a request that transport resolution may later veto. A local tool that occupies the provider tool's reserved name is rejected with a diagnostic naming both tools instead of being silently overwritten.
Design decisions
web_searchmerge (pre-existing) and the new OpenAI and Google merges go through it, which fixes a silent-overwrite bug: previously a local tool namedweb_searchwould have been replaced without any error.Not included
@ai-sdk/googledrops the wire'sis_errorflag before emitting the tool result, so no consumer can detect the failure until that upstream fix lands.Verification
Wire-level tests capture the actual request bodies: Google Search serialized as
tools: [{ type: "google_search" }]on the Interactions API (search-only and search-plus-function-tool shapes, which differ intool_choicehandling),{ type: "web_search" }with OpenAI defaults on the Responses API, and SigV4-signed Mantle requests withexternal_web_accessserialized as bothfalseandtrue. Capability tests cover the override, route, endpoint, model-family, region, and FIPS boundaries. The Gemini and OpenAI paths were also smoke-tested live, with grounded answers and citation payloads captured via raw HTTP logging.