diff --git a/website/docs/reference/heimdall/plugins.mdx b/website/docs/reference/heimdall/plugins.mdx index 48f0d5e..213a4ca 100644 --- a/website/docs/reference/heimdall/plugins.mdx +++ b/website/docs/reference/heimdall/plugins.mdx @@ -64,4 +64,10 @@ Uniformly selects a candidate at random, ignoring scores entirely. ### `weighted-random-picker` Weighted random selection using scores as weights. + +## Others + +### `saturation-gate` + +Pre-dispatch admission gate: refuses a request when every endpoint in the role pool it would be scheduled onto is saturated. {/* AUTO-GENERATED:aigateway-plugins END */} diff --git a/website/docs/reference/odin/api-reference.mdx b/website/docs/reference/odin/api-reference.mdx index 21c8420..c6c7bcd 100644 --- a/website/docs/reference/odin/api-reference.mdx +++ b/website/docs/reference/odin/api-reference.mdx @@ -17,7 +17,6 @@ An `InferenceService` describes a model deployment — the inference engine, the | Field | Type | Required | Description | | --- | --- | --- | --- | | `framework` | string | no | Framework specifies the inference engine (e.g. vllm, sglang). | -| `inferencePoolRefs` | [array](#inferenceservice-inferencepoolrefs) | no | InferencePoolRefs is a list of references to InferencePools. | | `model` | [object](#inferenceservice-model) | no | Model identifies the model to serve. | | `parallelism` | [object](#inferenceservice-parallelism) | no | Parallelism defines the parallelism parameters for distributed inference. | | `replicas` | integer | no | Number of replicas for Deployments or LeaderWorkerSets. | @@ -29,12 +28,6 @@ An `InferenceService` describes a model deployment — the inference engine, the | `templateRefs` | [array](#inferenceservice-templaterefs) | no | TemplateRefs is a list of references to InferenceServiceTemplates. | | `workerTemplate` | [object](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.32/#podtemplatespec-v1-core) | no | WorkerTemplate describes the pod template for LeaderWorkerSet workers. | -### `inferencePoolRefs` {#inferenceservice-inferencepoolrefs} - -| Field | Type | Required | Description | -| --- | --- | --- | --- | -| `name` | string | no | Name of the referent. This field is effectively required, but due to backwards compatibility is allowed to be empty. Instances of this type with an empty value here are almost certainly wrong. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names | - ### `model` {#inferenceservice-model} | Field | Type | Required | Description |