Configure tokenLimit and contentSafety policies on AI Gateway models and MCP ToolServer #28

Summary

Restore first-class AI Gateway policies (tokenLimit and contentSafety) on catalog models and contentSafety policy on the GitHub MCP ToolServer for the standalone RP (Microsoft.ApiManagement/aigateways@2025-09-01-preview).

Verified Live RP Wire Contract (Microsoft.ApiManagement/aigateways@2025-09-01-preview)

Both workspaces/default/modelProviders/foundry/models/{model} and workspaces/default/toolServers/github accept properties.policies on PUT:

1. Model Policies (Microsoft.ApiManagement/aigateways/workspaces/modelProviders/models@2025-09-01-preview)

   • tokenLimit requires counterKey as an array (['identity']), not a string ('Identity' fails with ValidationError: Invalid field 'counterKey' specified).
   • contentSafety sets hateSeverity, selfHarmSeverity, sexualSeverity, and violenceSeverity to 'Medium' (or configurable via model.?contentSafetySeverity ?? 'Medium').

   policies: model.?policies ?? [
     {
       type: 'tokenLimit'
       period: 'minute'
       count: model.?tokenLimit ?? 10000
       counterKey: [
         'identity'
       ]
     }
     {
       type: 'contentSafety'
       hateSeverity: model.?contentSafetySeverity ?? 'Medium'
       selfHarmSeverity: model.?contentSafetySeverity ?? 'Medium'
       sexualSeverity: model.?contentSafetySeverity ?? 'Medium'
       violenceSeverity: model.?contentSafetySeverity ?? 'Medium'
     }
   ]

2. Root infra/main.bicep Model Policy Inputs

   • In infra/main.bicep, pass tokenLimit: deployment.capacity * 1000 (so 20000 TPM for gpt-latest and gpt-6-sol, and 200000 TPM for gpt-mini-latest) and contentSafetySeverity: 'Medium' into each entry of foundryModels.

3. MCP ToolServer Content Safety Policy (toolServers/github)

   • Microsoft.ApiManagement/aigateways/workspaces/toolServers@2025-09-01-preview accepts properties.policies:

     "policies": [
       {
         "type": "contentSafety",
         "hateSeverity": "Medium",
         "selfHarmSeverity": "Medium",
         "sexualSeverity": "Medium",
         "violenceSeverity": "Medium"
       }
     ]

   • Because infra/scripts/configure-ai-gateway.sh and infra/scripts/configure-ai-gateway.ps1 perform a PUT on workspaces/default/toolServers/github during postprovision to inject the read-only GitHub MCP credential, include this policies array in the tool_server_body / $toolServerBody in both infra/scripts/configure-ai-gateway.sh and infra/scripts/configure-ai-gateway.ps1.

Files to Update

1. infra/ai-gateway/main.bicep:
   • Add policies default (tokenLimit with counterKey: ['identity'] and contentSafety with 'Medium' severities) on foundryCatalogModels.
2. infra/main.bicep:
   • Pass tokenLimit: deployment.capacity * 1000 on each entry in foundryModels.
3. infra/scripts/configure-ai-gateway.sh & infra/scripts/configure-ai-gateway.ps1:
   • Include "policies": [{"type": "contentSafety", "hateSeverity": "Medium", "selfHarmSeverity": "Medium", "sexualSeverity": "Medium", "violenceSeverity": "Medium"}] in the PUT payload for toolServers/github.
4. tests/test-ai-gateway-model-registration.sh & tests/test-apim-lifecycle.sh:
   • Update tests/test-ai-gateway-model-registration.sh (which previously asserted actual_model_policies == "null") to assert that foundryCatalogModels includes both tokenLimit (counterKey: ["identity"], period: "minute") and contentSafety (Medium severities), and that infra/main.bicep passes tokenLimit (mul(variables('gatewayModelDeployments')[copyIndex()].capacity, 1000)).
   • Add assertions in tests/test-apim-lifecycle.sh verifying that configure-ai-gateway.sh and configure-ai-gateway.ps1 include the contentSafety policy on toolServers/github.
5. README.md / IMPLEMENTATION_NOTES.md:
   • Ensure documentation reflects that tokenLimit (scaled to deployment capacity) and contentSafety (Medium) policies are configured on models and contentSafety (Medium) is configured on the GitHub MCP ToolServer.

Additional Requirement: Add Monthly Cost Limit (costLimit) Policy on Models

Also include a default $10.00 / month per caller identity costLimit policy on each catalog model in infra/ai-gateway/main.bicep (verified against the live Microsoft.ApiManagement/aigateways@2025-09-01-preview RP):

policies: model.?policies ?? [
  {
    type: 'tokenLimit'
    period: 'minute'
    count: model.?tokenLimit ?? 10000
    counterKey: [
      'identity'
    ]
  }
  {
    id: 'monthly-caller-budget'
    type: 'costLimit'
    period: 'month'
    amount: model.?costLimit ?? 10
    counterKey: [
      'identity'
    ]
  }
  {
    type: 'contentSafety'
    hateSeverity: model.?contentSafetySeverity ?? 'Medium'
    selfHarmSeverity: model.?contentSafetySeverity ?? 'Medium'
    sexualSeverity: model.?contentSafetySeverity ?? 'Medium'
    violenceSeverity: model.?contentSafetySeverity ?? 'Medium'
  }
]

• Note: costLimit.id is required by the live RP ("id": "monthly-caller-budget"), period is 'month', amount is 10 (or model.?costLimit ?? 10), and counterKey is ['identity'].
• Pass costLimit: 10 (or a root parameter modelMonthlyCostLimit int = 10) in infra/main.bicep for each entry in foundryModels, and assert costLimit in tests/test-ai-gateway-model-registration.sh as well.