Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion QUICKSTART.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@ docker compose -f docker-compose.sentinelguard.yml up --build

# Start OpenAI-compatible LLM gateway from generated config
export OPENAI_API_KEY="sk-..."
export SENTINELGUARD_GATEWAY_API_KEY="local-gateway-token"
export SENTINELGUARD_GATEWAY_API_KEY="replace-with-a-random-local-token"
sentinelguard gateway \
--config sentinelguard.yaml \
--gateway-config sentinelguard-gateway.yaml \
Expand All @@ -134,8 +134,16 @@ sentinelguard gateway --provider anthropic --port 8080

export GEMINI_API_KEY="..."
sentinelguard gateway --provider gemini --port 8080

export MOONSHOT_API_KEY="..."
sentinelguard gateway --provider kimi --port 8080
```

`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GEMINI_API_KEY`, and
`MOONSHOT_API_KEY` are upstream provider keys. `SENTINELGUARD_GATEWAY_API_KEY`
is a client-facing token you choose for apps and IDEs that call the
SentinelGuard gateway.

Point OpenAI-compatible apps or IDEs to:

```text
Expand Down
26 changes: 20 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -196,14 +196,18 @@ response, and returns the safe response.
pip install "sentinelguard[gateway,monitoring]"

export OPENAI_API_KEY="sk-..."
export SENTINELGUARD_GATEWAY_API_KEY="local-gateway-token"
export SENTINELGUARD_GATEWAY_API_KEY="replace-with-a-random-local-token"
sentinelguard init
sentinelguard gateway \
--config sentinelguard.yaml \
--gateway-config sentinelguard-gateway.yaml \
--port 8080
```

`OPENAI_API_KEY` is the upstream provider key. `SENTINELGUARD_GATEWAY_API_KEY`
is a client-facing token that you choose; apps, SDKs, and IDEs use it when they
call SentinelGuard at `http://localhost:8080/v1`.

For a quick single-provider run without generated files:

```bash
Expand All @@ -230,7 +234,7 @@ docker build -t sentinelguard-gateway .

docker run --rm -p 8080:8080 \
-e OPENAI_API_KEY="$OPENAI_API_KEY" \
-e SENTINELGUARD_GATEWAY_API_KEY="local-gateway-token" \
-e SENTINELGUARD_GATEWAY_API_KEY="replace-with-a-random-local-token" \
sentinelguard-gateway \
gateway --provider openai --client-api-key-env SENTINELGUARD_GATEWAY_API_KEY
```
Expand All @@ -242,7 +246,7 @@ sentinelguard init
cp .env.example .env
# Edit .env and set at least one upstream provider key.
export OPENAI_API_KEY="sk-..."
export SENTINELGUARD_GATEWAY_API_KEY="local-gateway-token"
export SENTINELGUARD_GATEWAY_API_KEY="replace-with-a-random-local-token"
docker compose -f docker-compose.sentinelguard.yml up --build
```

Expand Down Expand Up @@ -280,6 +284,10 @@ sentinelguard gateway --provider anthropic --port 8080
# Google Gemini
export GEMINI_API_KEY="..."
sentinelguard gateway --provider gemini --port 8080

# Kimi / Moonshot
export MOONSHOT_API_KEY="..."
sentinelguard gateway --provider kimi --port 8080
```

OpenAI-compatible providers can use the same gateway API shape. SentinelGuard
Expand All @@ -306,13 +314,17 @@ export HF_TOKEN="..."
sentinelguard gateway --provider huggingface --port 8080
```

SentinelGuard also supports private OpenAI-compatible model gateways, vLLM, TGI,
llama.cpp servers, local Ollama, and provider pools that route across multiple
models with failover.

Then point an OpenAI-compatible client at the gateway:

```python
from openai import OpenAI

client = OpenAI(
api_key="local-gateway-token",
api_key="replace-with-a-random-local-token",
base_url="http://localhost:8080/v1",
)

Expand All @@ -328,7 +340,7 @@ client-facing base URL and client-facing API key:
```bash
# In the app container or app runtime:
export OPENAI_BASE_URL="http://localhost:8080/v1"
export OPENAI_API_KEY="local-gateway-token"
export OPENAI_API_KEY="replace-with-a-random-local-token"
```

Keep the real upstream provider key on the SentinelGuard gateway process or
Expand Down Expand Up @@ -511,13 +523,15 @@ Provider defaults:
| `openai` | `https://api.openai.com/v1` | `OPENAI_API_KEY` |
| `anthropic` | `https://api.anthropic.com/v1` | `ANTHROPIC_API_KEY` |
| `gemini` | `https://generativelanguage.googleapis.com/v1beta` | `GEMINI_API_KEY` |
| `kimi` | `https://api.moonshot.ai/v1` | `MOONSHOT_API_KEY` |
| `deepseek` | `https://api.deepseek.com` | `DEEPSEEK_API_KEY` |
| `mistral` | `https://api.mistral.ai/v1` | `MISTRAL_API_KEY` |
| `minimax` | `https://api.minimaxi.com/v1` | `MINIMAX_API_KEY` |
| `ollama` | `http://localhost:11434/v1` | `OLLAMA_API_KEY` optional |
| `huggingface` | `https://router.huggingface.co/v1` | `HF_TOKEN` |

Gemini also checks `GOOGLE_API_KEY` when `GEMINI_API_KEY` is not set.
Gemini also checks `GOOGLE_API_KEY` when `GEMINI_API_KEY` is not set. Kimi also
checks `KIMI_API_KEY` when `MOONSHOT_API_KEY` is not set.
For custom OpenAI-compatible servers such as vLLM, TGI, llama.cpp servers, or
private model gateways, set `provider: openai-compatible` and provide
`upstream_url`.
Expand Down
16 changes: 16 additions & 0 deletions docs/assets/css/style.scss
Original file line number Diff line number Diff line change
Expand Up @@ -175,6 +175,22 @@
padding: 0.3rem 0.58rem;
}

.sg-provider-row {
display: flex;
flex-wrap: wrap;
gap: 0.45rem;
margin-top: 1rem;
}

.sg-provider-row span {
border-left: 3px solid var(--sg-teal);
color: var(--sg-muted);
font-size: 0.72rem;
font-weight: 750;
line-height: 1.2;
padding: 0.12rem 0 0.12rem 0.45rem;
}

.sg-terminal {
background: var(--sg-code-bg);
border: 1px solid rgba(255, 255, 255, 0.14);
Expand Down
2 changes: 2 additions & 0 deletions docs/gateway-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,8 @@ GET /gateway/provider-health
## Authentication

Runtime traffic and usage endpoints use the gateway client key when configured.
This is the token you set with `SENTINELGUARD_GATEWAY_API_KEY`; it is separate
from upstream provider keys such as `OPENAI_API_KEY` or `ANTHROPIC_API_KEY`.
Supported client headers:

```text
Expand Down
48 changes: 45 additions & 3 deletions docs/gateway.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,22 @@ SentinelGuard first.
```text
Application or IDE
-> SentinelGuard /v1/chat/completions
-> OpenAI, Anthropic, Gemini, Ollama, Mistral, DeepSeek, or another provider
-> OpenAI, Anthropic, Gemini, Kimi, Ollama, Mistral, DeepSeek, or another provider
```

## Start Locally

The example below uses two different keys:

- `OPENAI_API_KEY` is the upstream provider key used by SentinelGuard to call OpenAI.
- `SENTINELGUARD_GATEWAY_API_KEY` is a client-facing token that you choose. Your
apps, IDEs, and SDKs use this token when calling SentinelGuard at
`http://localhost:8080/v1`.

```bash
pip install "sentinelguard[gateway,monitoring]"
export OPENAI_API_KEY="sk-..."
export SENTINELGUARD_GATEWAY_API_KEY="local-gateway-token"
export SENTINELGUARD_GATEWAY_API_KEY="replace-with-a-random-local-token"

sentinelguard init
sentinelguard gateway \
Expand All @@ -24,6 +31,40 @@ sentinelguard gateway \
--port 8080
```

For local testing, the gateway token can be any random string. In shared or
production deployments, use a generated secret and keep it out of source code.

## Supported Providers

SentinelGuard can run as one gateway in front of public, private, and local
model providers:

| Provider | CLI shortcut | Key environment variable |
| --- | --- | --- |
| OpenAI | `--provider openai` | `OPENAI_API_KEY` |
| Anthropic Claude | `--provider anthropic` | `ANTHROPIC_API_KEY` |
| Google Gemini | `--provider gemini` | `GEMINI_API_KEY` or `GOOGLE_API_KEY` |
| Kimi / Moonshot | `--provider kimi` | `MOONSHOT_API_KEY` or `KIMI_API_KEY` |
| DeepSeek | `--provider deepseek` | `DEEPSEEK_API_KEY` |
| Mistral | `--provider mistral` | `MISTRAL_API_KEY` |
| MiniMax | `--provider minimax` | `MINIMAX_API_KEY` |
| Ollama | `--provider ollama` | optional `OLLAMA_API_KEY` |
| Hugging Face router | `--provider huggingface` | `HF_TOKEN` or `HUGGINGFACE_API_KEY` |
| vLLM, TGI, llama.cpp, private gateways | `--provider openai-compatible` | your configured key env |

Examples:

```bash
export ANTHROPIC_API_KEY="sk-ant-..."
sentinelguard gateway --provider anthropic --port 8080

export GEMINI_API_KEY="..."
sentinelguard gateway --provider gemini --port 8080

export MOONSHOT_API_KEY="..."
sentinelguard gateway --provider kimi --port 8080
```

## Configure Apps And IDEs

Use this OpenAI-compatible base URL:
Expand All @@ -33,7 +74,8 @@ http://localhost:8080/v1
```

If gateway client authentication is enabled, use the configured gateway token as
the client API key.
the client API key. That value is your `SENTINELGUARD_GATEWAY_API_KEY`, not the
upstream provider key.

## Change Gateway Settings

Expand Down
22 changes: 22 additions & 0 deletions docs/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,28 @@ This creates:

Use `--profile library` when you only want package-mode scanner configuration.

## Gateway Keys

Gateway mode usually has two kinds of keys:

- An upstream provider key, such as `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`,
`GEMINI_API_KEY`, or `MOONSHOT_API_KEY`. SentinelGuard uses this to call the
model provider.
- `SENTINELGUARD_GATEWAY_API_KEY`, a client-facing token that you choose.
Applications and IDEs use this token when they call SentinelGuard at
`http://localhost:8080/v1`.

For local testing, this gateway token can be any random string:

```bash
export SENTINELGUARD_GATEWAY_API_KEY="replace-with-a-random-local-token"
```

Supported gateway providers include OpenAI, Anthropic Claude, Google Gemini,
Kimi / Moonshot, DeepSeek, Mistral, MiniMax, Ollama, Hugging Face, and custom
OpenAI-compatible providers such as vLLM, TGI, llama.cpp, or private model
gateways.

## Change Configuration From The CLI

Scanner settings can be updated without opening the YAML file:
Expand Down
13 changes: 13 additions & 0 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,17 @@ description: Security-first LLM gateway and guardrails framework for AI applicat
<span>Prometheus metrics</span>
<span>Docker and Kubernetes ready</span>
</div>
<div class="sg-provider-row" aria-label="Supported providers">
<span>OpenAI</span>
<span>Anthropic Claude</span>
<span>Google Gemini</span>
<span>Kimi / Moonshot</span>
<span>DeepSeek</span>
<span>Mistral</span>
<span>MiniMax</span>
<span>Ollama</span>
<span>Hugging Face</span>
</div>
</div>

<div class="sg-terminal" aria-label="SentinelGuard gateway quick start">
Expand All @@ -52,6 +63,8 @@ description: Security-first LLM gateway and guardrails framework for AI applicat
</div>
<p class="sg-panel-title">Install and start the gateway</p>
<pre><code>pip install "sentinelguard[gateway,monitoring]"
export OPENAI_API_KEY="sk-..."
export SENTINELGUARD_GATEWAY_API_KEY="your-gateway-token"
sentinelguard init
sentinelguard gateway \
--config sentinelguard.yaml \
Expand Down
16 changes: 16 additions & 0 deletions docs/stylesheets/extra.css
Original file line number Diff line number Diff line change
Expand Up @@ -176,6 +176,22 @@
padding: 0.3rem 0.58rem;
}

.sg-provider-row {
display: flex;
flex-wrap: wrap;
gap: 0.45rem;
margin-top: 1rem;
}

.sg-provider-row span {
border-left: 3px solid var(--sg-teal);
color: var(--sg-muted);
font-size: 0.72rem;
font-weight: 750;
line-height: 1.2;
padding: 0.12rem 0 0.12rem 0.45rem;
}

.sg-terminal {
background: var(--sg-code-bg);
border: 1px solid rgba(255, 255, 255, 0.14);
Expand Down
9 changes: 9 additions & 0 deletions examples/helm/sentinelguard/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,15 @@ env:
OPENAI_API_KEY: ""
ANTHROPIC_API_KEY: ""
GEMINI_API_KEY: ""
GOOGLE_API_KEY: ""
MOONSHOT_API_KEY: ""
KIMI_API_KEY: ""
DEEPSEEK_API_KEY: ""
MISTRAL_API_KEY: ""
MINIMAX_API_KEY: ""
HF_TOKEN: ""
HUGGINGFACE_API_KEY: ""
OLLAMA_API_KEY: ""
SENTINELGUARD_GATEWAY_API_KEY: ""
SENTINELGUARD_AUDIT_SALT: ""

Expand Down
11 changes: 6 additions & 5 deletions examples/kubernetes/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,16 +38,17 @@ Create the gateway Secret:
kubectl create secret generic sentinelguard-gateway-secrets \
-n sentinelguard \
--from-literal=OPENAI_API_KEY="$OPENAI_API_KEY" \
--from-literal=SENTINELGUARD_GATEWAY_API_KEY="shared-gateway-token" \
--from-literal=SENTINELGUARD_GATEWAY_API_KEY="replace-with-shared-gateway-token" \
--from-literal=SENTINELGUARD_AUDIT_SALT="$(openssl rand -hex 32)"
```

For Anthropic, Gemini, DeepSeek, Mistral, MiniMax, or Hugging Face, add the
matching key:
For Anthropic, Gemini, Kimi/Moonshot, DeepSeek, Mistral, MiniMax, or Hugging
Face, add the matching key:

```bash
--from-literal=ANTHROPIC_API_KEY="$ANTHROPIC_API_KEY"
--from-literal=GEMINI_API_KEY="$GEMINI_API_KEY"
--from-literal=MOONSHOT_API_KEY="$MOONSHOT_API_KEY"
--from-literal=DEEPSEEK_API_KEY="$DEEPSEEK_API_KEY"
--from-literal=MISTRAL_API_KEY="$MISTRAL_API_KEY"
--from-literal=MINIMAX_API_KEY="$MINIMAX_API_KEY"
Expand Down Expand Up @@ -87,8 +88,8 @@ the OpenAI-compatible base URL to:
http://localhost:8080/v1
```

Use `shared-gateway-token` as the client API key when gateway client auth is
enabled.
Use your `SENTINELGUARD_GATEWAY_API_KEY` value as the client API key when
gateway client auth is enabled.

## Optional Ingress

Expand Down
12 changes: 12 additions & 0 deletions examples/kubernetes/deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,18 @@ spec:
name: sentinelguard-gateway-secrets
key: DEEPSEEK_API_KEY
optional: true
- name: MOONSHOT_API_KEY
valueFrom:
secretKeyRef:
name: sentinelguard-gateway-secrets
key: MOONSHOT_API_KEY
optional: true
- name: KIMI_API_KEY
valueFrom:
secretKeyRef:
name: sentinelguard-gateway-secrets
key: KIMI_API_KEY
optional: true
- name: MISTRAL_API_KEY
valueFrom:
secretKeyRef:
Expand Down
2 changes: 2 additions & 0 deletions examples/kubernetes/secret.example.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ stringData:
ANTHROPIC_API_KEY: ""
GEMINI_API_KEY: ""
GOOGLE_API_KEY: ""
MOONSHOT_API_KEY: ""
KIMI_API_KEY: ""
DEEPSEEK_API_KEY: ""
MISTRAL_API_KEY: ""
MINIMAX_API_KEY: ""
Expand Down
2 changes: 1 addition & 1 deletion examples/terraform/kubernetes/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ This example deploys the SentinelGuard Helm chart with Terraform.
terraform init
terraform apply \
-var='openai_api_key=sk-...' \
-var='gateway_api_key=local-gateway-token'
-var='gateway_api_key=replace-with-a-random-local-token'
```

The example assumes your local Kubernetes context already points at the target
Expand Down
Loading
Loading