Skip to content

[Feature]: Make models first class in the Gateway #3299

Description

@DinithHerath

Please select the area the issue is related to

AI Workspace, Gateway

Please select the aspect the issue is related to

Aspect/AI (AI/LLM integration, MCP, AI readiness), Aspect/API (API backends, definitions, contracts, interfaces, OpenAPI)

Suggested Feature

The current AI gateway doesn't treat models as first class citizens. We need to alter the Gateway architecture to identify and handle models properly. Ex:

Allow/denying models.
Forcing certain clients to certain models.
Changing models of apps without changing app code.
Multi-model routing
. . . .

Related Issues

No response

Steps to Verify

  • Design Document — A detailed design document has been created and reviewed, covering architecture, data flow, and edge cases.
  • Design Mail — A design summary email has been sent to relevant stakeholders for awareness and feedback.
  • Code Review — All code changes have been peer-reviewed and approved according to the project's review standards.
  • Testing Complete — Adequate unit, integration, and/or end-to-end tests have been written and are passing.
  • Documentation Review — User-facing and/or developer documentation has been updated to reflect the new feature and reviewed.
  • Feature Complete — The feature is fully implemented, all checklist items above are done, and it is ready for release.

Metadata

Metadata

Assignees

Labels

Area/AIGatewayAI Gateway runtime/control plane related issuesArea/AIWorkspaceIssues related to AI WorkspaceAspect/AIAI/LLM integration, MCP, AI readinessAspect/APIAPI definitions, contracts, OpenAPI, interfaces

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions