Skip to content

Configure AI Gateway model and GitHub MCP policies - #29

Open
Paul Yuknewicz (paulyuk) with Copilot wants to merge 4 commits into
mainfrom
copilot/configure-token-limit-content-safety
Open

Paul Yuknewicz (paulyuk) with Copilot wants to merge 4 commits into
mainfrom
copilot/configure-token-limit-content-safety

Conversation

Copilot AI commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Restore first-class tokenLimit and contentSafety policies for catalog models, and contentSafety for the GitHub MCP ToolServer under the standalone AI Gateway RP contract.

  • Catalog models: Add a per-minute token limit keyed by identity and a Medium content-safety policy. Root registrations set the limit to deployment capacity × 1,000 and pass Medium severity.
    {
      type: 'tokenLimit'
      period: 'minute'
      count: model.?tokenLimit ?? 10000
      counterKey: ['identity']
    }
  • GitHub MCP ToolServer: Include the Medium content-safety policy in both Bash and PowerShell credential PUT payloads.
  • Tests and documentation: Assert the compiled model policies, capacity-scaled limit, and ToolServer policies; update implementation notes to describe the configured policies.

Co-authored-by: paulyuk <1968137+paulyuk@users.noreply.github.com>
Copilot AI changed the title [WIP] Configure tokenLimit and contentSafety policies on models and MCP ToolServer Configure AI Gateway model and GitHub MCP policies Oct 26, 2026
@paulyuk

Copy link
Copy Markdown
Member

Copilot Please also include the costLimit budget policy (0.00 / month per caller identity) on each model in infra/ai-gateway/main.bicep and infra/main.bicep (verified against the live RP—note that id is required on costLimit):

    policies: model.?policies ?? [
      {
        type: 'tokenLimit'
        period: 'minute'
        count: model.?tokenLimit ?? 10000
        counterKey: [
          'identity'
        ]
      }
      {
        id: 'monthly-caller-budget'
        type: 'costLimit'
        period: 'month'
        amount: model.?costLimit ?? 10
        counterKey: [
          'identity'
        ]
      }
      {
        type: 'contentSafety'
        hateSeverity: model.?contentSafetySeverity ?? 'Medium'
        selfHarmSeverity: model.?contentSafetySeverity ?? 'Medium'
        sexualSeverity: model.?contentSafetySeverity ?? 'Medium'
        violenceSeverity: model.?contentSafetySeverity ?? 'Medium'
      }
    ]

Please also assert costLimit in tests/test-ai-gateway-model-registration.sh and document the 0/month per-caller budget policy in README.md / IMPLEMENTATION_NOTES.md.

@paulyuk

Copy link
Copy Markdown
Member

Copilot Also make sure both the overall README.md (under How it works -> Hub (infra/ai-gateway/) or a short policy table/list) and the Bicep module READMEs (infra/ai-gateway/README.md and infra/README.md) include a short, human-readable summary of the AI Gateway policies applied by default:

  • Token rate limit (tokenLimit) on models: Scaled to each deployment's capacity (20,000 tokens/minute per caller identity on gpt-latest and gpt-6-sol; 200,000 tokens/minute per caller identity on gpt-mini-latest).
  • Cost limit (costLimit) on models: $10.00 / month per caller identity (id: 'monthly-caller-budget') on each registered model.
  • Content safety (contentSafety) on models and MCP ToolServer: Medium threshold filtering across Hate, Self-Harm, Sexual, and Violence categories on all registered models and on the toolServers/github MCP server.
  • Model failover routing (aliases): gpt-latest alias routes across gpt-6.1-sol (priority 0) -> gpt-6-sol (priority 1) -> gpt-5.4-mini (priority 2).

Co-authored-by: paulyuk <1968137+paulyuk@users.noreply.github.com>

Copilot AI commented Oct 26, 2026

Copy link
Copy Markdown
Contributor Author

Copilot Please also include the costLimit budget policy (0.00 / month per caller identity) on each model in infra/ai-gateway/main.bicep and infra/main.bicep (verified against the live RP—no...

Added the per-caller costLimit policy (monthly-caller-budget, 0 per month) to each model, asserted it in the registration test, and updated README and implementation notes. Committed as 28ce28c.

Co-authored-by: paulyuk <1968137+paulyuk@users.noreply.github.com>

Copilot AI commented Oct 26, 2026

Copy link
Copy Markdown
Contributor Author

Copilot Also make sure both the overall README.md (under How it works -> Hub (infra/ai-gateway/) or a short policy table/list) and the Bicep module READMEs (infra/ai-gateway/README.md and `...

Added default-policy summaries to the overall README and both infrastructure READMEs, including the sample token rates, $10/month caller budget, Medium content-safety categories, and alias priorities. I also aligned the root deployment input from $0 to $10/month so the documentation matches the deployed policy. Committed as db3f1e1.

@paulyuk
Paul Yuknewicz (paulyuk) marked this pull request as ready for review October 26, 2026 14:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Configure tokenLimit and contentSafety policies on AI Gateway models and MCP ToolServer

2 participants