diff --git a/IMPLEMENTATION_NOTES.md b/IMPLEMENTATION_NOTES.md index a900863..f8bbf90 100644 --- a/IMPLEMENTATION_NOTES.md +++ b/IMPLEMENTATION_NOTES.md @@ -309,12 +309,13 @@ Each model registration includes: segment of the deployment resource ID Model creation omits inline policies for compatibility with the live RP. The -alias `gpt-latest` routes to the full deployment at priority 0 and fails over to -the mini deployment at priority 1. Both backends explicitly set a weight of 100; -aliases can instead place multiple backends at the same priority with relative -weights for traffic splitting. The separate `gpt-mini-latest` alias addresses -the mini deployment directly. Each alias backend points to the gateway child -model resource ID, while the client sends the alias name as the OpenAI `model`. +alias `gpt-latest` routes through a three-tier failover chain: `gpt-6.1-sol` at +priority 0, `gpt-6-sol` at priority 1, and `gpt-5.4-mini` at priority 2. Each +backend explicitly sets a weight of 100; aliases can instead place multiple +backends at the same priority with relative weights for traffic splitting. The +separate `gpt-mini-latest` alias addresses the mini deployment directly. Each +alias backend points to the gateway child model resource ID, while the client +sends the alias name as the OpenAI `model`. ## Regions and model defaults @@ -341,15 +342,19 @@ The AI Gateway preview region allowlist is: - `swedencentral` - `eastus2` -The model defaults are: +The model defaults use Global Standard deployments: -| Purpose | Foundry deployment / Gateway alias | Backing model | Version | Capacity | -| --- | --- | --- | --- | --- | -| Full | `gpt-latest` | `gpt-5.6-sol` | `2026-07-09` | 20 | -| Mini | `gpt-mini-latest` | `gpt-5.4-mini` | `2026-03-17` | 200 | +| Purpose | Foundry deployment / Gateway alias | Backing model | Version | SKU | Capacity | +| --- | --- | --- | --- | --- | --- | +| Primary | `gpt-latest` | `gpt-6.1-sol` | `2026-09-29` | `GlobalStandard` | 20 | +| Secondary failover | `gpt-6-sol` | `gpt-6-sol` | `2026-09-22` | `GlobalStandard` | 20 | +| Mini safety net | `gpt-mini-latest` | `gpt-5.4-mini` | `2026-03-17` | `GlobalStandard` | 200 | -If `gpt-5.6-sol` is unavailable, set `modelName=gpt-5.5` and -`modelVersion=2026-04-24`. +If the primary deployment is unavailable or its quota is exhausted, AI Gateway +automatically tries `gpt-6-sol` in a separate quota pool, then the higher- +capacity `gpt-5.4-mini` deployment. If a model/version is unavailable in the +selected region, override its corresponding name and version parameters with a +supported catalog deployment and keep the remaining fallback tiers configured. Global Standard deployment capacity is provisioned on the Foundry model account. Mini capacity 200 accommodates requests from the GitHub Copilot app, diff --git a/README.md b/README.md index c42c9e4..4b9493c 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # Daily Repo Digest with AI Gateway -This sample runs a Microsoft Agent Framework repo-digest agent in Foundry Hosted Agents. Its model calls use AI Gateway, and Foundry Toolbox routes read-only GitHub MCP calls through AI Gateway. +This sample demonstrates a **hub-and-spoke governance architecture** for Foundry Hosted Agents using **AI Gateway** as the central hub. The Microsoft Agent Framework repo-digest agent runs in an isolated Foundry project spoke (`infra/foundry-agents/`) with zero model deployments and no direct credentials to backing models or external MCP servers. All model inference (`gpt-latest` three-tier failover across `gpt-6.1-sol` → `gpt-6-sol` → `gpt-5.4-mini`, plus `gpt-mini-latest`) and read-only GitHub MCP tool calls (via Foundry Toolbox) are routed **exclusively through AI Gateway (`infra/ai-gateway/`)** — there is no other access path from the agent to the model account (`infra/foundry-models/`) or to GitHub MCP. ## Prerequisites @@ -39,14 +39,14 @@ follow **[Set up the GitHub credential](IMPLEMENTATION_NOTES.md#tighten-the-gith ## Run locally -Create the local `.env` from the selected azd environment: +`azd up` and `azd provision` automatically generate `.env` from the active azd environment. You can also refresh `.env` manually at any time: ```bash # macOS or Linux -./scripts/create-dev-env.sh +./scripts/create-dev-env.sh --force # Windows -pwsh ./scripts/create-dev-env.ps1 +pwsh ./scripts/create-dev-env.ps1 -Force ``` Start the agent: @@ -114,4 +114,8 @@ The default is `microsoft/agent-framework`. The included scheduled routine keeps **Start with [`repo_digest_agent.py`](repo_digest_agent.py).** It connects the Agent Framework agent directly to the AI Gateway model route and uses Foundry Toolbox for the AI Gateway GitHub MCP route. [`github_mcp_middleware.py`](github_mcp_middleware.py) keeps GitHub results bounded, and [`main.py`](main.py) hosts the agent in Foundry. +- **Hub (`infra/ai-gateway/`)**: Standalone AI Gateway (`Microsoft.ApiManagement/aigateways`) that enforces central routing, token rate limits, failover (`gpt-latest`), and MCP tool brokering (`toolServers/github`). Only AI Gateway's managed identity receives cross-resource runtime access to the model account through the configured Foundry User RBAC assignment. Only AI Gateway holds the GitHub MCP token. +- **Agent Spoke (`infra/foundry-agents/`)**: Hosts the Foundry project, ACR, Storage, and Hosted Agent runtime. Its project account has no model deployments and no RBAC assignment on the separate model account, ensuring the agent cannot bypass AI Gateway to call models or GitHub MCP directly. +- **Model Spoke (`infra/foundry-models/`)**: Dedicated model-only Azure AI Services account (`allowProjectManagement: false`, `disableLocalAuth: true`) hosting `gpt-6.1-sol`, `gpt-6-sol`, and `gpt-5.4-mini`, accessible to the agent only through AI Gateway. + See [`IMPLEMENTATION_NOTES.md`](IMPLEMENTATION_NOTES.md), including the [GitHub credential boundary](IMPLEMENTATION_NOTES.md#github-credential-boundary) and [how to tighten the GitHub credential to least privilege](IMPLEMENTATION_NOTES.md#tighten-the-github-credential-to-least-privilege), for architecture, authentication, AI Gateway contracts, security boundaries, configuration, monitoring, and deployment details. diff --git a/infra/README.md b/infra/README.md new file mode 100644 index 0000000..4927157 --- /dev/null +++ b/infra/README.md @@ -0,0 +1,56 @@ +# Infrastructure + +Run the commands in this README from the `infra/` directory. + +The subscription-scoped [`main.bicep`](main.bicep) template creates three +resource groups and deploys one module into each: + +- [`foundry-models/`](foundry-models/README.md) creates a model-only Azure AI + Services account and the GPT deployments. +- [`foundry-agents/`](foundry-agents/README.md) creates the Foundry project and + hosted-agent runtime dependencies. +- [`ai-gateway/`](ai-gateway/README.md) creates AI Gateway, model registrations, + and aliases. + +The [`scripts/`](scripts/README.md) directory contains the preprovision, +postprovision, and postdown lifecycle hooks. `azd provision` runs the +preprovision and postprovision hooks from `azure.yaml`; postprovision configures +the GitHub MCP ToolServer and Foundry Toolbox after Bicep creates the Azure +resources. + +## Deploy the full stack + +`azd provision` also works from `infra/`: azd walks up to find +[`../azure.yaml`](../azure.yaml), then provisions the full stack and runs its +preprovision and postprovision hooks: + +```bash +azd provision +``` + +Alternatively, deploy the subscription-scoped Bicep template directly: + +```bash +az deployment sub create \ + --location eastus2 \ + --template-file ./main.bicep \ + --parameters environmentName=dailydigest \ + location=eastus2 \ + principalId="$(az ad signed-in-user show --query id -o tsv)" +``` + +The direct deployment creates the Azure resources but does not run azd hooks. +For GitHub MCP and Toolbox setup after a direct deployment, see +[`scripts/README.md`](scripts/README.md) and provide the deployment outputs to +the configuration script. + +## Deploy individual modules + +Each resource-group-scoped template can also be deployed independently. Create +its resource group first, then follow the module-specific parameters and +examples: + +- [Foundry model account and deployments](foundry-models/README.md) +- [Foundry project and hosted-agent runtime](foundry-agents/README.md) +- [AI Gateway and model routing](ai-gateway/README.md) +- [Lifecycle and postprovision scripts](scripts/README.md) diff --git a/infra/ai-gateway/README.md b/infra/ai-gateway/README.md new file mode 100644 index 0000000..9774bc7 --- /dev/null +++ b/infra/ai-gateway/README.md @@ -0,0 +1,139 @@ +# AI Gateway + +Run the commands in this README from the `infra/ai-gateway/` directory. + +[`main.bicep`](main.bicep) is a resource-group-scoped module for an AI Gateway +with Consumption SKU. With the default `deployApimGatewayViaBicep: true`, it +creates `Microsoft.ApiManagement/aigateways`, the +`Microsoft.Web/connectorGateways` Connector Namespace, and `apiKeys/default`. +When monitoring is enabled, it also provisions Log Analytics, Application +Insights, and the Gateway telemetry exporter. + +When `deployFoundryProvider` is enabled, the module assigns the Gateway identity +the Foundry User role on each account in `foundryResourceIds`, registers the +supplied `foundryModels` under `modelProviders/foundry`, and creates the +configured aliases. The sample routes `gpt-latest` through three tiers: +`gpt-6.1-sol` → `gpt-6-sol` → `gpt-5.4-mini` (priorities 0, 1, and 2, +respectively, with weight 100 each). `gpt-mini-latest` targets the mini model +directly. + +## Gateway without a Foundry provider + +Create a resource group, then deploy just the Gateway and Connector Namespace. +This minimal mode leaves `deployFoundryProvider` disabled, so it does not +register model catalog entries or aliases: + +```bash +az group create --name rg-ai-gateway --location eastus2 + +az deployment group create \ + --resource-group rg-ai-gateway \ + --template-file ./main.bicep \ + --parameters location=eastus2 gatewayName=aigw-example +``` + +## Connect an existing Foundry model account + +The existing account must already contain deployments named `gpt-latest`, +`gpt-6-sol`, and `gpt-mini-latest` (or update the model names below to match its +deployments). The following example gets the account ID and endpoint, creates a +parameters file, and deploys the Gateway with registrations and aliases. The +operator needs permission to create the Gateway and assign its identity the +Foundry User role on the account. + +```bash +FOUNDRY_RESOURCE_GROUP=rg-foundry-models +FOUNDRY_ACCOUNT_NAME=aimodels-example +FOUNDRY_ACCOUNT_ID="$(az cognitiveservices account show \ + --resource-group "$FOUNDRY_RESOURCE_GROUP" \ + --name "$FOUNDRY_ACCOUNT_NAME" \ + --query id --output tsv)" +FOUNDRY_ENDPOINT="$(az cognitiveservices account show \ + --resource-group "$FOUNDRY_RESOURCE_GROUP" \ + --name "$FOUNDRY_ACCOUNT_NAME" \ + --query properties.endpoint --output tsv)" + +az group create --name rg-ai-gateway --location eastus2 + +cat > /tmp/ai-gateway.parameters.json < None: - ResponsesHostServer(agent=build_agent, history_source="agent_server").run() + ResponsesHostServer(agent=build_runtime_agent, history_source="agent_server").run() if __name__ == "__main__": diff --git a/tests/test-ai-gateway-model-registration.sh b/tests/test-ai-gateway-model-registration.sh index a34270c..d33eddb 100755 --- a/tests/test-ai-gateway-model-registration.sh +++ b/tests/test-ai-gateway-model-registration.sh @@ -99,20 +99,35 @@ if [[ "$root_alias_input" != *"modelArmName"* || fi default_alias_routes="$(jq -r '.variables.effectiveModelAliasBackends' "$compiled_root_template")" -for token in \ - "parameters('modelDeploymentName')" \ - "parameters('miniModelDeploymentName')" \ - "'priority', 0" \ - "'priority', 1" \ - "'weight', 100"; do - if ! grep -Fq "$token" <<< "$default_alias_routes"; then - echo "Default alias routes must configure primary/failover priority and explicit weight." >&2 - echo "Missing: $token" >&2 +for route in \ + "createObject('modelArmName', parameters('modelDeploymentName'), 'priority', 0, 'weight', 100)" \ + "createObject('modelArmName', parameters('secondaryModelDeploymentName'), 'priority', 1, 'weight', 100)" \ + "createObject('modelArmName', parameters('miniModelDeploymentName'), 'priority', 2, 'weight', 100)"; do + if ! grep -Fq "$route" <<< "$default_alias_routes"; then + echo "Default alias routes must configure all three backends with explicit priority and weight." >&2 + echo "Missing: $route" >&2 echo "Actual: $default_alias_routes" >&2 exit 1 fi done +for setting in \ + "modelName gpt-6.1-sol" \ + "modelVersion 2026-09-29" \ + "secondaryModelName gpt-6-sol" \ + "secondaryModelVersion 2026-09-22" \ + "miniModelName gpt-5.4-mini" \ + "miniModelVersion 2026-03-17"; do + read -r parameter expected <<< "$setting" + actual="$(jq -r --arg parameter "$parameter" '.parameters[$parameter].defaultValue' "$compiled_root_template")" + if [ "$actual" != "$expected" ]; then + echo "Unexpected default for $parameter." >&2 + echo "Expected: $expected" >&2 + echo "Actual: $actual" >&2 + exit 1 + fi +done + root_model_input="$(jq -r ' .. | objects | select(has("foundryModels")) diff --git a/tests/test_agent_runtime.py b/tests/test_agent_runtime.py index 7aa3573..a9b528e 100644 --- a/tests/test_agent_runtime.py +++ b/tests/test_agent_runtime.py @@ -52,9 +52,23 @@ def test_host_server_receives_agent_factory_and_history_source(self) -> None: with patch.object(main, "ResponsesHostServer", return_value=server) as host_server: main.main() - host_server.assert_called_once_with(agent=main.build_agent, history_source="agent_server") + host_server.assert_called_once_with(agent=main.build_runtime_agent, history_source="agent_server") server.run.assert_called_once_with() + def test_build_runtime_agent_reloads_dotenv_locally(self) -> None: + sentinel = object() + with ( + patch.dict(os.environ, {}, clear=False), + patch.object(main, "load_dotenv") as reload_dotenv, + patch.object(main, "build_agent", return_value=sentinel) as build_agent, + ): + os.environ.pop("FOUNDRY_HOSTING_ENVIRONMENT", None) + result = main.build_runtime_agent() + + self.assertIs(result, sentinel) + reload_dotenv.assert_called_once_with(override=True) + build_agent.assert_called_once_with() + class _UnsupportedListen: async def __aenter__(self) -> None: