diff --git a/docs/APIs-and-SDKs/MCP-Server/_category_.json b/docs/APIs-and-SDKs/MCP-Server/_category_.json new file mode 100644 index 00000000..1040861a --- /dev/null +++ b/docs/APIs-and-SDKs/MCP-Server/_category_.json @@ -0,0 +1,6 @@ +{ + "position": 4, + "collapsible": true, + "collapsed": true, + "label": "MCP Server" +} diff --git a/docs/APIs-and-SDKs/MCP-Server/authentication.mdx b/docs/APIs-and-SDKs/MCP-Server/authentication.mdx new file mode 100644 index 00000000..dddce0dd --- /dev/null +++ b/docs/APIs-and-SDKs/MCP-Server/authentication.mdx @@ -0,0 +1,98 @@ +--- +title: Authentication +sidebar_position: 4 +--- + +# Authentication + +The ABsmartly MCP server supports two authentication methods. Both are +configured per client — there are no server-side secrets, so credentials are +always provided by your MCP client and isolated per connection. + +| Method | Best for | How it works | +| --- | --- | --- | +| **API Key** | Programmatic access, CI/CD, headless agents | Pass the key in HTTP headers or via the URL | +| **OAuth** | Interactive use in Claude Desktop, Cursor, the web, etc. | Browser-based login via ABsmartly SAML | + +## API key + +API keys are issued from your ABsmartly instance under +**Settings → API Keys**. They authenticate every request and identify which +ABsmartly account the MCP session belongs to. + +### Where to pass the key + +Depending on the client, the API key is supplied as either: + +- An HTTP header (`Authorization: …` plus `x-absmartly-endpoint: …`), or +- A query string parameter on the connection URL + (`?api_key=YOUR_API_KEY&absmartly-endpoint=https://your-instance.absmartly.com`). + +See the [Setup](./setup) guide for the exact configuration per client. + +### Authorization header formats + +When using API key authentication, these header formats are all supported: + +| Format | Example | +| --- | --- | +| Simple API key | `Authorization: BxYKd1U2...` | +| Explicit `Api-Key` | `Authorization: Api-Key BxYKd1U2...` | +| Subdomain shorthand | `Authorization: demo-1 BxYKd1U2...` | +| Full domain | `Authorization: demo-1.absmartly.com BxYKd1U2...` | + +The **subdomain shorthand** auto-constructs the endpoint as +`https://.absmartly.com/v1`, so you don't need to send a separate +`x-absmartly-endpoint` header. + +## OAuth + +OAuth uses your existing ABsmartly login flow (SAML or username/password) and +issues a JWT scoped to your MCP session. This is the recommended method for +interactive clients like Claude Desktop, Cursor or the ChatGPT web app. + +When OAuth is configured, the MCP client opens a browser window the first time +it connects. After you sign in, the client receives the token and the MCP +connection is established. Subsequent connections reuse the cached token until +it expires. + +### Specifying the instance URL + +OAuth needs to know which ABsmartly instance to log in against. You can +provide this in either of two ways: + +- Add `?absmartly-endpoint=https://your-instance.absmartly.com` to the MCP + URL, or +- Omit it — the OAuth flow will prompt you for the URL in the browser before + login. + +## Local stdio (CLI credentials) + +When running the server locally with `npx @absmartly/mcp`, authentication uses +your ABsmartly CLI configuration and macOS Keychain credentials. Select a +profile with `--profile=` (for example, `--profile=production`). + +## Security model + +- **No server-side secrets** — credentials are always provided by the MCP + client. +- **Session isolation** — each connection has its own credentials. +- **OAuth protection** — API key sessions block OAuth discovery to prevent + authentication-method confusion. +- **Destructive action confirmation** — `start`, `stop`, `archive` and + `delete` operations require explicit confirmation before they run (see the + [Tools Reference](./tools-reference#execute_command)). +- **API error surfacing** — validation errors from the API are returned with + full detail so the assistant can correct them. + +## Checking who you are + +Once connected, you can confirm authentication and identity at any time with +the `get_auth_status` tool: + +```text +Are we authenticated? +``` + +The response includes the authenticated email, name, ABsmartly endpoint and the +authentication type in use (`API Key` or `OAuth`). diff --git a/docs/APIs-and-SDKs/MCP-Server/images/claude-desktop-add-custom-connector.png b/docs/APIs-and-SDKs/MCP-Server/images/claude-desktop-add-custom-connector.png new file mode 100644 index 00000000..bc0a170d Binary files /dev/null and b/docs/APIs-and-SDKs/MCP-Server/images/claude-desktop-add-custom-connector.png differ diff --git a/docs/APIs-and-SDKs/MCP-Server/images/claude-desktop-connectors.png b/docs/APIs-and-SDKs/MCP-Server/images/claude-desktop-connectors.png new file mode 100644 index 00000000..187f95c0 Binary files /dev/null and b/docs/APIs-and-SDKs/MCP-Server/images/claude-desktop-connectors.png differ diff --git a/docs/APIs-and-SDKs/MCP-Server/images/oauth-endpoint-prompt.png b/docs/APIs-and-SDKs/MCP-Server/images/oauth-endpoint-prompt.png new file mode 100644 index 00000000..d83867de Binary files /dev/null and b/docs/APIs-and-SDKs/MCP-Server/images/oauth-endpoint-prompt.png differ diff --git a/docs/APIs-and-SDKs/MCP-Server/overview.mdx b/docs/APIs-and-SDKs/MCP-Server/overview.mdx new file mode 100644 index 00000000..e9f24847 --- /dev/null +++ b/docs/APIs-and-SDKs/MCP-Server/overview.mdx @@ -0,0 +1,102 @@ +--- +title: Overview +sidebar_position: 1 +--- + +# ABsmartly MCP Server + +The ABsmartly [Model Context Protocol](https://modelcontextprotocol.io/) (MCP) +server lets AI assistants — Claude, Cursor, VS Code, Windsurf, Gemini, ChatGPT +and any other MCP-compatible client — operate the ABsmartly experimentation +platform directly. The AI plans experiments, creates them, starts them, reads +results and updates metrics, all through natural-language prompts. + +It exposes **230+ commands across 33 groups**: experiments, metrics, goals, +teams, applications, segments, webhooks and more. + +## Why use it + +An MCP server turns ABsmartly into a "tool" your AI assistant can call. Instead +of switching between your assistant, the ABsmartly UI and the API docs, you +work in one place and the assistant does the rest. + +Typical workflows that get faster with MCP: + +- **Drafting experiments** from a description, including templates with goals, + metrics, variants and ownership prefilled +- **Bulk operations** — "clone this experiment for the EU app, switch the unit + type to `device_id`, and update the audience" +- **Reading state** — "what experiments are running this week and what's their + status?" +- **Reviewing setups** — "find experiments without owners" or "list metrics + pending review" +- **Cross-tool agentic work** — combine ABsmartly actions with code edits in + Cursor / Claude Code, or with Slack / Jira / Notion via other MCP servers + +## When to use MCP vs the alternatives + +| Use case | Best tool | +| --- | --- | +| Conversational, ad-hoc operations | **MCP server** | +| Application code that assigns variants and tracks goals | [Client SDKs](/docs/APIs-and-SDKs/SDK-Documentation/getting-started) | +| Programmatic automation, CI/CD, custom integrations | [Web Console API](/docs/APIs-and-SDKs/Web-Console-API/backend) | +| Visual editing of pages for an A/B test | [LaunchPad browser extension](/docs/web-console-docs/launchpad-browser-extension/getting-started) | +| Manual UI work | The ABsmartly Web Console | + +MCP isn't a replacement for the SDKs — application code still assigns variants +and tracks goals via the SDK. MCP is for the **control plane**: configuring, +launching and reviewing experiments and metrics. + +## How it works + +The server exposes just **4 tools** that give AI assistants access to the full +ABsmartly CLI surface: + +| Tool | Purpose | +| --- | --- | +| `discover_commands` | Browse command groups or search by keyword | +| `get_command_docs` | Get detailed docs for a specific command | +| `execute_command` | Execute any command with typed parameters | +| `get_auth_status` | Check authentication status | + +A typical interaction looks like this: + +```text +"What experiments are running?" + → discover_commands(group: "experiments") + → execute_command(group: "experiments", command: "listExperiments", + params: { state: "running" }) +``` + +Destructive operations (`start`, `stop`, `archive`, `delete`) always require an +explicit confirmation step before they run. Experiment creation from templates +returns a preview first. + +See the [Tools Reference](./tools-reference) for the full surface. + +## Authentication at a glance + +| Method | Best for | How it works | +| --- | --- | --- | +| **OAuth** | Interactive use in Claude Desktop, Cursor, the web | Browser-based login via your ABsmartly SAML/SSO flow | +| **API Key** | Programmatic access, CI/CD, headless | Pass the key in HTTP headers or via the URL | + +Both methods are documented in detail in [Authentication](./authentication). + +## What's next + +- **[Quick Start](./quick-start)** — connect Claude Desktop with OAuth and run + your first command in under five minutes +- **[Setup](./setup)** — per-client setup for every supported MCP client +- **[Authentication](./authentication)** — API key formats, OAuth flow, + security model +- **[Usage Examples](./usage-examples)** — natural-language prompts that work + today +- **[Tools Reference](./tools-reference)** — full list of tools, command groups, + resources and prompts +- **[Troubleshooting](./troubleshooting)** — common errors and how to fix them + +## Source code + +The MCP server is open source under the MIT license: +[github.com/absmartly/mcp](https://github.com/absmartly/mcp). diff --git a/docs/APIs-and-SDKs/MCP-Server/quick-start.mdx b/docs/APIs-and-SDKs/MCP-Server/quick-start.mdx new file mode 100644 index 00000000..4cac1eed --- /dev/null +++ b/docs/APIs-and-SDKs/MCP-Server/quick-start.mdx @@ -0,0 +1,112 @@ +--- +title: Quick Start +sidebar_position: 2 +--- + +# Quick Start + +This guide gets you from zero to running your first ABsmartly command via the +MCP server in under five minutes. We'll use **Claude Desktop** with **OAuth** +— the fastest path with no API keys to manage. + +For other clients or API key setups, jump straight to [Setup](./setup). + +## Prerequisites + +- **Claude Desktop** installed + ([download](https://claude.ai/download)) +- An ABsmartly account on your instance (the URL you log into, e.g. + `https://your-company.absmartly.com`) + +## 1. Add the connector + +In Claude Desktop, open **Settings → Connectors**, scroll to the bottom, and +click **Add custom connector**. + +![Claude Desktop Connectors settings](./images/claude-desktop-connectors.png) + +In the dialog: + +- **Name:** `ABsmartly` +- **Remote MCP server URL:** + +```text +https://mcp.absmartly.com/sse?absmartly-endpoint=https://your-instance.absmartly.com +``` + +Replace `your-instance` with your ABsmartly subdomain. + +![Add custom connector dialog](./images/claude-desktop-add-custom-connector.png) + +Click **Add**, then **Connect**. + +## 2. Sign in + +Claude Desktop opens your browser. Sign in with your normal ABsmartly +credentials (SAML/SSO if your org uses it). When the browser confirms success, +the connector is live. + +:::tip +If you didn't include `absmartly-endpoint` in the URL, the OAuth flow prompts +you for it in the browser before login. + +![Connect to ABsmartly endpoint prompt](./images/oauth-endpoint-prompt.png) +::: + +## 3. Verify the connection + +Back in Claude Desktop, start a new chat and try: + +```text +Are we authenticated with ABsmartly? +``` + +The assistant calls `get_auth_status` and returns your email, name, instance +endpoint and auth type (`OAuth`). + +## 4. Run your first command + +```text +Show me the experiments that are currently running on ABsmartly +``` + +Behind the scenes the assistant calls: + +```text +execute_command(group: "experiments", command: "listExperiments", + params: { state: "running" }) +``` + +It returns the list, formatted for chat. + +## 5. Try something useful + +A few prompts that work out of the box (and don't change any state): + +```text +List all metrics that are pending review +``` + +```text +Which teams own experiments that have been running for more than 14 days? +``` + +```text +Show me the goals defined for the "checkout" application +``` + +When you're ready to **create or change** something, the assistant will pause +and show a preview / confirmation step — destructive actions like `start`, +`stop`, `archive`, and `delete` always require explicit confirmation before +they run. + +## Where to go next + +- **[Usage Examples](./usage-examples)** — more natural-language prompts that + work today +- **[Tools Reference](./tools-reference)** — the full set of commands, including + experiment creation from markdown templates +- **[Authentication](./authentication)** — switch to API key auth (for CI or + headless use), or learn about the security model +- **[Troubleshooting](./troubleshooting)** — if the connection isn't working, + start here diff --git a/docs/APIs-and-SDKs/MCP-Server/setup.mdx b/docs/APIs-and-SDKs/MCP-Server/setup.mdx new file mode 100644 index 00000000..e3c40adb --- /dev/null +++ b/docs/APIs-and-SDKs/MCP-Server/setup.mdx @@ -0,0 +1,593 @@ +--- +title: Setup +sidebar_position: 3 +--- + +import Tabs from "@theme/Tabs"; +import TabItem from "@theme/TabItem"; +import BrowserOnly from "@docusaurus/BrowserOnly"; + +export const vscodeInstallUrl = "vscode:mcp/install?" + encodeURIComponent('{"name":"absmartly","type":"sse","url":"https://mcp.absmartly.com/sse"}'); +export const vscodeInsidersInstallUrl = "vscode-insiders:mcp/install?" + encodeURIComponent('{"name":"absmartly","type":"sse","url":"https://mcp.absmartly.com/sse"}'); +export const cursorInstallUrl = "cursor://anysphere.cursor-deeplink/mcp/install?name=absmartly&config=" + btoa('{"url":"https://mcp.absmartly.com/sse"}'); + +export const VSCodeInstallLinks = () => ( + + {() => ( + + Install in VS Code{" · "} + Install in VS Code Insiders + + )} + +); + +export const CursorInstallLink = () => ( + + {() => Add to Cursor} + +); + +# Setup + +The ABsmartly MCP server can be added to any MCP-compatible client. This page +covers Claude Desktop, Claude Code, Cursor, Windsurf, VS Code (Copilot), Gemini +(CLI and Code Assist), ChatGPT and the DXT extension. + +For details on API keys, the OAuth flow and supported `Authorization` header +formats, see [Authentication](./authentication). + +## Transports + +The server is reachable on two HTTP endpoints: + +| Endpoint | Transport | Notes | +| --- | --- | --- | +| `https://mcp.absmartly.com/mcp` | **Streamable HTTP** | Modern MCP transport (recommended for new installs). | +| `https://mcp.absmartly.com/sse` | **SSE** | Legacy transport. Still works; required by some older clients. | + +The MCP spec deprecated SSE in March 2025, but `/sse` continues to be supported +for compatibility. + +You can also run the server locally over stdio with `npx @absmartly/mcp`, which +reuses your ABsmartly CLI config and macOS Keychain credentials. + +--- + +## Claude Desktop + + + + +Open **Settings → Connectors**, scroll to the bottom, and click +**Add custom connector**. + +![Claude Desktop Connectors settings](./images/claude-desktop-connectors.png) + +In the dialog, give the connector a name (e.g. `ABsmartly`) and paste the URL +into **Remote MCP server URL**: + +![Add custom connector dialog](./images/claude-desktop-add-custom-connector.png) + +**URL:** + +```text +https://mcp.absmartly.com/sse?absmartly-endpoint=https://your-instance.absmartly.com +``` + +Click **Add**, then **Connect**. Claude Desktop will open your browser for +ABsmartly login (SAML/credentials). After authentication, the MCP connection is +established automatically. + +:::tip +The `absmartly-endpoint` query parameter is optional. If omitted, the OAuth flow +will prompt you to enter the URL of your ABsmartly instance in the browser before +login: + +![Connect to ABsmartly endpoint prompt](./images/oauth-endpoint-prompt.png) +::: + + + + +Pass the API key directly in the URL — no `mcp-remote` bridge required: + +```text +https://mcp.absmartly.com/sse?api_key=YOUR_API_KEY&absmartly-endpoint=https://your-instance.absmartly.com +``` + + + + +To pass the key via headers instead of the URL, use the `mcp-remote` bridge — +Claude Desktop doesn't support custom headers natively for remote servers. + +**`~/Library/Application Support/Claude/claude_desktop_config.json`** (macOS): + +```json +{ + "mcpServers": { + "absmartly": { + "command": "npx", + "args": [ + "mcp-remote@latest", + "https://mcp.absmartly.com/sse", + "--header", "Authorization:YOUR_API_KEY", + "--header", "x-absmartly-endpoint:https://your-instance.absmartly.com" + ] + } + } +} +``` + + + + +Uses your ABsmartly CLI config and macOS Keychain credentials: + +```json +{ + "mcpServers": { + "absmartly": { + "command": "npx", + "args": ["@absmartly/mcp", "--profile=production"] + } + } +} +``` + + + + +--- + +## Claude Code + + + + +```bash +claude mcp add --transport sse --scope user absmartly \ + https://mcp.absmartly.com/sse \ + -H "Authorization:YOUR_API_KEY" \ + -H "x-absmartly-endpoint:https://your-instance.absmartly.com" +``` + +**Subdomain shorthand** — auto-constructs `https://.absmartly.com/v1`: + +```bash +claude mcp add --transport sse --scope user absmartly \ + https://mcp.absmartly.com/sse \ + -H "Authorization:my-subdomain YOUR_API_KEY" +``` + + + + +Recommended for new installations: + +```bash +claude mcp add --transport http --scope user absmartly \ + https://mcp.absmartly.com/mcp \ + -H "Authorization:YOUR_API_KEY" \ + -H "x-absmartly-endpoint:https://your-instance.absmartly.com" +``` + +:::info +Streamable HTTP is the modern MCP transport. The `/sse` URL still works for +older clients. +::: + + + + +```bash +claude mcp add --transport sse --scope user absmartly \ + "https://mcp.absmartly.com/sse?absmartly-endpoint=https://your-instance.absmartly.com" +``` + +Then run `/mcp` in Claude Code to authenticate when prompted. The +`absmartly-endpoint` query parameter is optional — omit it and the OAuth flow +will prompt you for the instance URL in the browser. + + + + +```bash +claude mcp add --scope user absmartly -- npx @absmartly/mcp --profile=production +``` + + + + +```json +{ + "mcpServers": { + "absmartly": { + "type": "sse", + "url": "https://mcp.absmartly.com/sse", + "headers": { + "Authorization": "YOUR_API_KEY", + "x-absmartly-endpoint": "https://your-instance.absmartly.com" + } + } + } +} +``` + +Use `--scope project` instead of `--scope user` to write to `.mcp.json` (shared +via version control). + + + + +--- + +## Cursor + +One-click install (OAuth): . Cursor will prompt you for the ABsmartly instance URL in the browser, then the flow completes automatically. + +Or configure manually in **`.cursor/mcp.json`** (project) or +**`~/.cursor/mcp.json`** (global): + + + + +```json +{ + "mcpServers": { + "absmartly": { + "url": "https://mcp.absmartly.com/sse", + "headers": { + "Authorization": "YOUR_API_KEY", + "x-absmartly-endpoint": "https://your-instance.absmartly.com" + } + } + } +} +``` + + + + +Modern transport, recommended: + +```json +{ + "mcpServers": { + "absmartly": { + "type": "http", + "url": "https://mcp.absmartly.com/mcp", + "headers": { + "Authorization": "YOUR_API_KEY", + "x-absmartly-endpoint": "https://your-instance.absmartly.com" + } + } + } +} +``` + + + + +```json +{ + "mcpServers": { + "absmartly": { + "url": "https://mcp.absmartly.com/sse?absmartly-endpoint=https://your-instance.absmartly.com" + } + } +} +``` + +Cursor will detect the OAuth requirement and open your browser for login. The +`absmartly-endpoint` query parameter is optional. + + + + +```json +{ + "mcpServers": { + "absmartly": { + "command": "npx", + "args": ["@absmartly/mcp", "--profile=production"] + } + } +} +``` + + + + +--- + +## Windsurf + +Configure **`~/.codeium/windsurf/mcp_config.json`**: + +:::note +Windsurf uses `"serverUrl"` (not `"url"`). +::: + + + + +```json +{ + "mcpServers": { + "absmartly": { + "serverUrl": "https://mcp.absmartly.com/sse", + "headers": { + "Authorization": "YOUR_API_KEY", + "x-absmartly-endpoint": "https://your-instance.absmartly.com" + } + } + } +} +``` + + + + +```json +{ + "mcpServers": { + "absmartly": { + "serverUrl": "https://mcp.absmartly.com/mcp", + "headers": { + "Authorization": "YOUR_API_KEY", + "x-absmartly-endpoint": "https://your-instance.absmartly.com" + } + } + } +} +``` + + + + +```json +{ + "mcpServers": { + "absmartly": { + "serverUrl": "https://mcp.absmartly.com/sse?absmartly-endpoint=https://your-instance.absmartly.com" + } + } +} +``` + + + + +```json +{ + "mcpServers": { + "absmartly": { + "command": "npx", + "args": ["@absmartly/mcp", "--profile=production"] + } + } +} +``` + + + + +--- + +## VS Code (GitHub Copilot) + +One-click install (OAuth): . You'll be prompted for your ABsmartly instance URL in the browser. + +Or configure manually in **`.vscode/mcp.json`** (workspace) or via Command +Palette → **MCP: Open User Configuration**. + +:::note +VS Code uses `"servers"` as the root key (not `"mcpServers"`). +::: + + + + +```json +{ + "servers": { + "absmartly": { + "type": "sse", + "url": "https://mcp.absmartly.com/sse", + "headers": { + "Authorization": "${input:absmartly-api-key}", + "x-absmartly-endpoint": "https://your-instance.absmartly.com" + } + } + }, + "inputs": [ + { + "type": "promptString", + "id": "absmartly-api-key", + "description": "ABsmartly API Key", + "password": true + } + ] +} +``` + + + + +Recommended: + +```json +{ + "servers": { + "absmartly": { + "type": "http", + "url": "https://mcp.absmartly.com/mcp", + "headers": { + "Authorization": "${input:absmartly-api-key}", + "x-absmartly-endpoint": "https://your-instance.absmartly.com" + } + } + }, + "inputs": [ + { + "type": "promptString", + "id": "absmartly-api-key", + "description": "ABsmartly API Key", + "password": true + } + ] +} +``` + + + + +```json +{ + "servers": { + "absmartly": { + "type": "sse", + "url": "https://mcp.absmartly.com/sse?absmartly-endpoint=https://your-instance.absmartly.com" + } + } +} +``` + +VS Code will detect the OAuth requirement and show an **Auth** CodeLens to +trigger the flow. The `absmartly-endpoint` query parameter is optional. + + + + +```json +{ + "servers": { + "absmartly": { + "command": "npx", + "args": ["@absmartly/mcp", "--profile=production"] + } + } +} +``` + + + + +:::caution +MCP tools only appear in **Agent mode** in VS Code, not in Ask or Edit mode. +::: + +--- + +## Gemini (CLI + Code Assist) + +Works for **Gemini CLI** (terminal) and **Gemini Code Assist** (VS Code, +JetBrains, Android Studio agent mode). Both read `~/.gemini/settings.json` +(user-wide) or `.gemini/settings.json` (per-project). JetBrains IDEs use +`mcp.json` instead. + + + + +```json +{ + "mcpServers": { + "absmartly": { + "url": "https://mcp.absmartly.com/sse?absmartly-endpoint=https://your-instance.absmartly.com" + } + } +} +``` + +Gemini auto-discovers the OAuth requirement. In Gemini CLI, run +`/mcp auth absmartly` when prompted; in Code Assist agent mode the browser opens +automatically. + + + + +```json +{ + "mcpServers": { + "absmartly": { + "url": "https://mcp.absmartly.com/sse", + "headers": { + "Authorization": "YOUR_API_KEY", + "x-absmartly-endpoint": "https://your-instance.absmartly.com" + } + } + } +} +``` + + + + +```bash +gemini mcp add --transport sse --scope user absmartly \ + https://mcp.absmartly.com/sse \ + -H "Authorization: YOUR_API_KEY" \ + -H "x-absmartly-endpoint: https://your-instance.absmartly.com" +``` + + + + +### Gemini Enterprise (Google Cloud Console) + +Gemini Enterprise's **Custom MCP Server** connector (Preview) requires +Streamable HTTP transport — use the `/mcp` endpoint: + +1. Google Cloud Console → **Gemini Enterprise** → **Data stores** → + **Create data store**. +2. Search "Custom MCP Server" → **Add MCP server**. +3. **Server URL:** `https://mcp.absmartly.com/mcp` +4. **Authentication:** OAuth — register Gemini Enterprise as an OAuth client + against your identity provider, then provide the `client_id` / + `client_secret`. Grant the `mcp:access` scope. +5. Save and wait for the connector status to become **Active**. + +:::caution +SSE transport (`/sse`) is **not** supported by this connector. The legacy Gemini +CLI / Code Assist sections above continue to use `/sse` until those clients add +Streamable HTTP support. + +Reload the IDE window after editing settings (VS Code: Command Palette → +**Developer: Reload Window**). MCP support in Code Assist requires +**agent preview mode** — set `"geminicodeassist.updateChannel": "Insiders"` in +VS Code settings if not already enabled. +::: + +--- + +## ChatGPT (Developer Mode) + +ChatGPT does not support one-click install deeplinks — you connect remote MCP +servers via the **Custom Connectors** UI. Requires a Pro, Team, Enterprise or +Edu plan, and Developer Mode enabled (Settings → **Connectors** → +**Advanced** → **Developer mode**). + +1. Go to **Settings → Connectors → Create**. +2. Fill in: + - **Name:** `ABsmartly` + - **MCP Server URL:** + `https://mcp.absmartly.com/sse?absmartly-endpoint=https://your-instance.absmartly.com` + - **Authentication:** OAuth +3. Save. In a new chat, click **+** → **Developer mode** → select the + **ABsmartly** connector to add it as a tool source. + +:::note +Workspace admins must first enable **Custom MCP connectors** in +**Workspace Settings → Permissions & Roles → Connected Data**. +::: + +--- + +## DXT Extension + +1. Download from + [mcp.absmartly.com/absmartly-mcp.dxt](https://mcp.absmartly.com/absmartly-mcp.dxt). +2. Double-click to install in Claude Desktop. +3. Enter your ABsmartly endpoint when prompted. diff --git a/docs/APIs-and-SDKs/MCP-Server/tools-reference.mdx b/docs/APIs-and-SDKs/MCP-Server/tools-reference.mdx new file mode 100644 index 00000000..58df7f54 --- /dev/null +++ b/docs/APIs-and-SDKs/MCP-Server/tools-reference.mdx @@ -0,0 +1,175 @@ +--- +title: Tools Reference +sidebar_position: 7 +--- + +# Tools Reference + +The MCP server exposes **4 tools**, plus a rich set of **resources** and +**prompts**. Together they cover 230+ commands across 33 ABsmartly command +groups. + +## Tools + +### `discover_commands` + +Browse available command groups or search by keyword. Call without parameters +to see all groups. + +```json +{ "group": "experiments" } +``` + +```json +{ "search": "archive" } +``` + +### `get_command_docs` + +Get detailed documentation for a specific command, including parameters, types +and examples. + +```json +{ + "group": "experiments", + "command": "createExperimentFromTemplate" +} +``` + +### `execute_command` + +Execute any ABsmartly command. Common commands are listed in the tool +description; use `discover_commands` for the full list. + +```json +{ + "group": "experiments", + "command": "listExperiments", + "params": { "state": "running", "items": 10 } +} +``` + +**Parameters:** + +| Name | Required | Description | +| --- | --- | --- | +| `group` | yes | Command group (e.g. `experiments`, `metrics`, `goals`) | +| `command` | yes | Command name (e.g. `listExperiments`, `createExperimentFromTemplate`) | +| `params` | no | Command parameters as a JSON object | +| `confirmed` | no | Set to `true` to confirm destructive actions | +| `raw` | no | Return the full `CommandResult` with rows / detail / warnings / pagination | +| `limit` | no | Max items for list operations (default: `20`) | + +:::caution Destructive actions require confirmation +`start`, `stop`, `archive` and `delete` operations all return a confirmation +prompt on the first call. Re-invoke the command with `confirmed: true` to +actually perform the action. +::: + +:::info Experiment creation previews +`createExperimentFromTemplate` also requires confirmation. The first call +returns a **preview** — the resolved API payload (with names mapped to IDs) +plus any warnings — without creating the experiment. Review the preview, then +call again with `confirmed: true` to actually create it. +::: + +#### Creating experiments with templates + +The recommended way to create experiments is with markdown templates: + +```json +{ + "group": "experiments", + "command": "createExperimentFromTemplate", + "params": { + "templateContent": "---\nname: my_experiment\ntype: test\npercentages: \"50/50\"\nunit_type: user_id\napplication: www\n---\n\n## Variants\n\n### variant_0\n\nname: control\nconfig: {}\n\n---\n\n### variant_1\n\nname: treatment\nconfig: {}\n" + } +} +``` + +Templates use YAML frontmatter for configuration and markdown for variants, +audience and description. Names for applications, unit types, metrics, teams, +tags and owners are **automatically resolved to IDs**. + +Read the `absmartly://docs/templates` resource for complete examples, +including: + +- Basic A/B test +- Feature flags +- Group Sequential Tests (GST) +- Screenshots +- Custom fields +- Multi-variant (A/B/C) tests + +### `get_auth_status` + +Check the current authentication state and connection info. Returns the +authenticated email, name, ABsmartly endpoint and authentication type. + +```text +Are we authenticated? +``` + +## Command groups + +| Group | Description | Key commands | +| --- | --- | --- | +| `experiments` | Experiment lifecycle | `list`, `get`, `create`, `start`, `stop`, `archive`, `clone`, `diff`, `export`, `bulk` | +| `metrics` | Metric definitions and review | `list`, `get`, `create`, `review`, `access` | +| `goals` | Goal definitions | `list`, `get`, `create`, `access` | +| `segments` | Audience segments | `list`, `get`, `create`, `delete` | +| `teams` | Team management | `list`, `get`, `create`, `members` | +| `users` | User management | `list`, `get`, `create`, `api-keys` | +| `apps` | Applications | `list`, `get`, `create`, `archive` | +| `envs` | Environments | `list`, `get`, `create` | +| `units` | Unit types | `list`, `get`, `create` | +| `tags` | Experiment tags | `list`, `get`, `create`, `delete` | +| `auth` | Authentication | `whoami`, `api-keys` | +| `insights` | Analytics | `velocity`, `decisions` | +| `webhooks` | Webhook management | `list`, `get`, `create`, `delete` | +| `roles` | Role management | `list`, `get`, `create` | +| `datasources` | Data source config | `list`, `test`, `introspect` | +| `exportconfigs` | Export configuration | `list`, `create`, `pause` | + +Additional groups: `goaltags`, `metrictags`, `metriccategories`, `apikeys`, +`permissions`, `assetroles`, `notifications`, `favorites`, `cors`, +`updateschedules`, `customsections`, `platformconfig`, `activity`, +`statistics`, `events`, `storageconfigs`, `actiondialogfields`. + +## Resources + +The MCP server exposes resources that AI assistants can read directly, +without having to call `execute_command`. They're handy for fast lookups +(entity lists) and embedded documentation. + +| URI | Description | +| --- | --- | +| `absmartly://entities/applications` | Cached applications list | +| `absmartly://entities/unit-types` | Available unit types | +| `absmartly://entities/teams` | Teams list | +| `absmartly://entities/users` | Users list | +| `absmartly://entities/metrics` | Metrics list | +| `absmartly://entities/goals` | Goals list | +| `absmartly://entities/tags` | Experiment tags | +| `absmartly://entities/custom-fields` | Custom field definitions | +| `absmartly://docs/templates` | Markdown template examples and reference | +| `absmartly://docs/api` | API documentation | +| `absmartly://docs/experiments` | Experiment management docs | +| `absmartly://docs/metrics` | Metrics docs | +| `absmartly://docs/goals` | Goals docs | +| `absmartly://docs/segments` | Segments docs | +| `absmartly://docs/analytics` | Analytics docs | +| `absmartly://examples/api-requests` | Common request patterns | + +## Prompts + +The server also ships built-in prompts that clients can surface as one-click +actions: + +| Prompt | Description | +| --- | --- | +| `experiment-status` | Quick overview of all running experiments | +| `create-experiment` | Guided experiment creation with entity context | +| `create-feature-flag` | Simplified feature flag creation | +| `analyze-experiment` | Deep analysis of a specific experiment | +| `experiment-review` | Review running experiments for issues | diff --git a/docs/APIs-and-SDKs/MCP-Server/troubleshooting.mdx b/docs/APIs-and-SDKs/MCP-Server/troubleshooting.mdx new file mode 100644 index 00000000..40e4a92c --- /dev/null +++ b/docs/APIs-and-SDKs/MCP-Server/troubleshooting.mdx @@ -0,0 +1,135 @@ +--- +title: Troubleshooting +sidebar_position: 6 +--- + +# Troubleshooting + +Common issues and how to fix them. + +## Authentication + +### "Failed to authenticate" or 401 errors with an API key + +Check the `Authorization` header format. The server accepts several variants: + +| Format | Example | +| --- | --- | +| Simple API key | `Authorization: BxYKd1U2...` | +| Explicit `Api-Key` | `Authorization: Api-Key BxYKd1U2...` | +| Subdomain shorthand | `Authorization: demo-1 BxYKd1U2...` | +| Full domain | `Authorization: demo-1.absmartly.com BxYKd1U2...` | + +If you use the **subdomain shorthand** or **full domain** form, you do **not** +need to send `x-absmartly-endpoint` — the server constructs the endpoint +automatically. + +If you use the **simple** or **`Api-Key`** form, you **must** send +`x-absmartly-endpoint: https://your-instance.absmartly.com` (or include +`?absmartly-endpoint=...` in the URL). + +### OAuth login loops or doesn't complete + +- **Wrong instance URL.** If `absmartly-endpoint` is missing or points at the + wrong subdomain, the OAuth callback can't reach your instance. Re-add the + connector with an explicit `?absmartly-endpoint=https://your-instance.absmartly.com`. +- **Browser blocking pop-ups or third-party cookies.** Allow pop-ups for + `mcp.absmartly.com` and your ABsmartly subdomain, and ensure third-party + cookies aren't fully blocked for the SAML/SSO callback. +- **Stale session.** Sign out of ABsmartly in the browser tab, then restart + the OAuth flow from the MCP client. + +### "API key sessions can't use OAuth" + +The server deliberately blocks mixing the two — you can't connect with an API +key and then trigger OAuth discovery on the same session. Pick one method per +connector. If you want to switch, remove the connector and add it again with +the new auth method. + +## Connection + +### MCP tools don't appear in the assistant + +- **VS Code (Copilot):** MCP tools only show up in **Agent mode**, not Ask or + Edit mode. Switch modes and reload the window if needed + (Command Palette → **Developer: Reload Window**). +- **Gemini Code Assist:** MCP support requires the agent preview channel. Set + `"geminicodeassist.updateChannel": "Insiders"` in VS Code settings, then + reload. +- **Claude Desktop / Cursor / Windsurf:** restart the app fully (quit, don't + just close the window) after editing the config file. + +### "Connection refused" or the server doesn't respond + +- For **remote**: check that `https://mcp.absmartly.com/sse` (or `/mcp` for + Streamable HTTP) is reachable from your machine — corporate proxies and + VPNs occasionally block it. +- For **local stdio (`npx @absmartly/mcp`)**: make sure Node.js 22+ is on + your PATH and that the ABsmartly CLI profile you reference exists (run + `absmartly profile list` to check). + +### SSE transport works but Streamable HTTP doesn't (or vice versa) + +The two endpoints expose the same tools. Older MCP clients sometimes only +support `/sse`; newer ones prefer `/mcp` (Streamable HTTP). If one transport +fails, switch to the other. + +The MCP spec deprecated SSE in March 2025, but `/sse` continues to be supported +for compatibility. + +## Commands + +### "Command requires confirmation" + +This is expected behavior for destructive operations — `start`, `stop`, +`archive` and `delete` all pause for confirmation. Re-call `execute_command` +with the same parameters plus `"confirmed": true` to actually perform the +action. + +### `createExperimentFromTemplate` returned a preview, not a created experiment + +Also expected. The first call returns the resolved API payload (with names +mapped to IDs) plus any warnings. Review the preview, then call again with +`"confirmed": true` to create the experiment. + +### "Method not found" or similar from `execute_command` + +The command catalog evolves over time. Use `discover_commands` (no parameters) +to list current groups, then `discover_commands(group: "...")` to list current +commands in a group. If a command from older docs is gone, the catalog will +suggest a replacement. + +### Validation errors when creating experiments or metrics + +The server surfaces the full validation error from the ABsmartly API. Common +causes: + +- A referenced **application**, **unit type** or **team** doesn't exist (names + are resolved to IDs automatically — make sure they match exactly) +- A referenced **metric** or **goal** is archived +- The template YAML frontmatter is missing required fields (`name`, `type`, + `unit_type`, `application`) + +Use the `absmartly://entities/...` resources to look up valid entity names +before retrying. + +## Performance + +### List operations are truncated + +`execute_command` defaults `limit` to **20** for list operations. To get more +results, pass `limit: 100` (or higher) in `params`, or use `raw: true` to get +the full pagination object back so the assistant can fetch additional pages. + +### Responses feel slow + +The first request after a cold start fetches all entities (applications, unit +types, users, teams, metrics, goals, tags, custom fields) in parallel and +caches them. Subsequent requests reuse the cache. If response time is +consistently bad, check your network path to `mcp.absmartly.com`. + +## Still stuck? + +- Check the server source and issues: + [github.com/absmartly/mcp/issues](https://github.com/absmartly/mcp/issues) +- For ABsmartly platform questions, reach out to the ABsmartly support team. diff --git a/docs/APIs-and-SDKs/MCP-Server/usage-examples.mdx b/docs/APIs-and-SDKs/MCP-Server/usage-examples.mdx new file mode 100644 index 00000000..dea2191f --- /dev/null +++ b/docs/APIs-and-SDKs/MCP-Server/usage-examples.mdx @@ -0,0 +1,71 @@ +--- +title: Usage Examples +sidebar_position: 5 +--- + +# Usage Examples + +Once the MCP server is connected in your AI assistant (see [Setup](./setup)), +you can drive ABsmartly with plain language. Here are prompts that map cleanly +to the underlying [tools](./tools-reference). + +## Discover what's available + +```text +What ABsmartly operations can you help me with? +``` + +The assistant will call `discover_commands` and summarize the available +groups. + +## List running experiments + +```text +Show me all running experiments +``` + +## Create an experiment from a template + +```text +Create a new A/B test called "checkout_cta_test" with Control and Blue Button +variants +``` + +The assistant will draft a markdown template, show a preview, then create the +experiment after you confirm. + +## Full lifecycle + +```text +Create an experiment, move it to ready, start it, then show me its details +``` + +## Feature flags + +```text +Create a feature flag called "dark_mode" and start it +``` + +## Clone and modify + +```text +Clone experiment 42 as "checkout_v2" and update the traffic split to 70/30 +``` + +## Compare experiments + +```text +Show me the differences between experiments 42 and 43 +``` + +--- + +## Tips + +- Destructive actions (`start`, `stop`, `archive`, `delete`) always pause for + a confirmation step. You'll see the planned operation before it runs. +- Experiment creation from templates also pauses with a preview of the + resolved payload — review the warnings before confirming. +- If you're unsure which command an assistant should call, ask it to + "browse the ABsmartly commands first" — that triggers `discover_commands` + and gives the model the right context before acting. diff --git a/docs/APIs-and-SDKs/_category_.json b/docs/APIs-and-SDKs/_category_.json index 5f1e1a37..881d19db 100644 --- a/docs/APIs-and-SDKs/_category_.json +++ b/docs/APIs-and-SDKs/_category_.json @@ -2,5 +2,5 @@ "position": 2, "collapsible": true, "collapsed": true, - "label": "APIs & SDKs" + "label": "Developer Tools" } diff --git a/docs/APIs-and-SDKs/overview.mdx b/docs/APIs-and-SDKs/overview.mdx index dd0602ff..9e31b4a9 100644 --- a/docs/APIs-and-SDKs/overview.mdx +++ b/docs/APIs-and-SDKs/overview.mdx @@ -4,15 +4,117 @@ sidebar_position: 0 # Overview -The APIs & SDKs section provides everything you need to integrate ABsmartly into your product. -This is the technical foundation that powers experiment assignment, event tracking, feature flags and consistent user experiences across all platforms. +The **Developer Tools** section is the technical home of ABsmartly. It covers +every way you can integrate, automate, extend or operate the platform from +code or AI tools — from running your first experiment in a web app to driving +the entire system from a Claude or Cursor chat. -Here you will find: -- SDK documentation for client side, server side and mobile environments -- API references for the ABsmartly platform and the SDKs -- implementation guides, examples and best practices -- details on exposure handling, goal tracking and variant assignment +If you are an engineer integrating ABsmartly, a platform team maintaining +experiment infrastructure, or a PM/analyst automating workflows, this section +is for you. -If you are an engineer integrating ABsmartly or maintaining experiment infrastructure, this section is for you. -It explains how the SDKs work, how to use the API programmatically and how to build a reliable experimentation setup that fits your architecture. +--- + +## What's in this section + +### SDK Guide + +The [SDK Guide](/docs/APIs-and-SDKs/SDK-Documentation/getting-started) is the +starting point for any product integration. It explains how to install the +client SDK, create a context, expose users to experiments and track goals — in +all officially supported languages and frameworks. + +Topics covered: + +- [Getting started](/docs/APIs-and-SDKs/SDK-Documentation/getting-started) — install the SDK and create your first context +- [Running your first experiment](/docs/APIs-and-SDKs/SDK-Documentation/running-your-first-experiment) — assign variants and use variant variables +- [Targeting and audiences](/docs/APIs-and-SDKs/SDK-Documentation/targeting-and-audiences) — restrict experiments by attribute +- [Managing the context lifecycle](/docs/APIs-and-SDKs/SDK-Documentation/managing-the-context-lifecycle) — initialise, publish, refresh and finalise +- [Pre-fetching context data](/docs/APIs-and-SDKs/SDK-Documentation/pre-fetching-context-data) — speed up first paint with server-side prefetch +- [Checking variants without exposures](/docs/APIs-and-SDKs/SDK-Documentation/checking-variants-without-exposures) — inspect treatments without logging an exposure +- [Using variables as code](/docs/APIs-and-SDKs/SDK-Documentation/using-variables-as-code) — push HTML, CSS or JS from the Web Console +- [Using custom fields](/docs/APIs-and-SDKs/SDK-Documentation/using-custom-fields) — read custom experiment metadata from your code + +Full end-to-end examples — covering JavaScript, React, Vue, Vue 3, +Python, Java, Go, PHP, Ruby, Swift, .NET and Flutter — are embedded as +language tabs in +[Running your first experiment](/docs/APIs-and-SDKs/SDK-Documentation/running-your-first-experiment). + +### SDK API + +The [SDK API](/docs/APIs-and-SDKs/SDK-API/absmartly-collector-api) reference +documents the **ABsmartly Collector API** — the HTTP endpoints the SDKs talk +to. Use this when you are building a custom integration in a language that +doesn't have an official SDK, or when you need to inspect the raw protocol the +SDKs use to fetch contexts and publish exposures. + +It includes endpoints for context creation, exposure publishing (single and +batch), experiment lookup and collector health checks. + +### Platform API + +The [Platform API](/docs/APIs-and-SDKs/Web-Console-API/backend) is the full +HTTP API behind the ABsmartly Web Console. Anything you can do in the UI you +can do through the API: list and create experiments, manage metrics and goals, +control teams and permissions, manage applications and unit types, post +annotations and comments, fetch experiment results, and configure webhooks. + +This is the right surface for: + +- CI/CD automation (e.g. create experiments from a deploy pipeline) +- Custom dashboards and internal tooling +- Bulk imports/exports +- Programmatic governance — auditing experiments, metrics or permissions + +See the [API examples](/docs/APIs-and-SDKs/Web-Console-API/Examples/Slack-Integration) +for end-to-end integrations such as a custom Slack notifier. + +### MCP Server + +The [MCP Server](/docs/APIs-and-SDKs/MCP-Server/overview) gives AI +assistants — Claude, Cursor, VS Code Copilot, Windsurf, Gemini, ChatGPT — full +access to the ABsmartly platform through a single connector. Drive +experiments, metrics, goals and teams conversationally instead of by writing +API calls. + +Common starting points: + +- [Quick Start](/docs/APIs-and-SDKs/MCP-Server/quick-start) — connect Claude Desktop with OAuth in under five minutes +- [Setup](/docs/APIs-and-SDKs/MCP-Server/setup) — every supported MCP client +- [Usage Examples](/docs/APIs-and-SDKs/MCP-Server/usage-examples) — natural-language prompts that work today +- [Tools Reference](/docs/APIs-and-SDKs/MCP-Server/tools-reference) — the full surface: 230+ commands across 33 groups + +--- + +## Which one should I use? + +| If you want to… | Use | +| --- | --- | +| Assign variants and track goals in your application code | [SDK Guide](/docs/APIs-and-SDKs/SDK-Documentation/getting-started) | +| Integrate ABsmartly in a language with no official SDK | [SDK API](/docs/APIs-and-SDKs/SDK-API/absmartly-collector-api) | +| Create, manage or query experiments programmatically | [Platform API](/docs/APIs-and-SDKs/Web-Console-API/backend) | +| Build automation, internal tools, or CI/CD integrations | [Platform API](/docs/APIs-and-SDKs/Web-Console-API/backend) | +| Drive ABsmartly from Claude, Cursor or another AI assistant | [MCP Server](/docs/APIs-and-SDKs/MCP-Server/overview) | +| Visually edit pages for an A/B test without writing code | [LaunchPad browser extension](/docs/web-console-docs/launchpad-browser-extension/getting-started) | +| Run experiments and configure the workspace from the UI | [Product Documentation](/docs/web-console-docs/overview) | + +The SDK and the APIs solve different problems and are commonly used together +in the same product: the **SDK** does the user-facing work (assignment, +exposure, goal tracking) while the **Platform API** is used by your internal +tools and automation to configure and inspect experiments. The **MCP Server** +sits on top of the Platform API — it's a conversational interface to the same +operations. + +--- + +## New to ABsmartly? + +If you are integrating the platform for the first time, the recommended path is: +1. Read [Get Started](/docs/get-started) to understand the core concepts +2. Follow [Running your first experiment](/docs/APIs-and-SDKs/SDK-Documentation/running-your-first-experiment) + in the language of your choice +3. Skim the [Platform API introduction](/docs/APIs-and-SDKs/Web-Console-API/backend) + so you know what's available when you need it +4. Try the [MCP Server Quick Start](/docs/APIs-and-SDKs/MCP-Server/quick-start) + if you'd like to drive ABsmartly from your AI assistant diff --git a/docs/get-started.mdx b/docs/get-started.mdx index ff5737bc..85650152 100644 --- a/docs/get-started.mdx +++ b/docs/get-started.mdx @@ -33,14 +33,15 @@ If you use ABsmartly day to day to run experiments or feature flags, this is whe --- -### APIs & SDKs +### Developer Tools -This [section](/docs/APIs-and-SDKs/overview) contains all technical resources for implementing ABsmartly in your product. +This [section](/docs/APIs-and-SDKs/overview) contains all technical resources for implementing, automating and extending ABsmartly. You will find: -- [SDK documentation](/docs/APIs-and-SDKs/sdk-documentation/) for client side, server side and mobile -- [SDK API](/docs/APIs-and-SDKs/SDK-API/absmartly-collector-api) references and examples -- The full platform [API reference](/docs/APIs-and-SDKs/SDK-API/absmartly-collector-api) for programmatic access +- [SDK documentation](/docs/APIs-and-SDKs/SDK-Documentation/getting-started) for client side, server side and mobile +- [SDK API](/docs/APIs-and-SDKs/SDK-API/absmartly-collector-api) reference for the ABsmartly Collector API used by the SDKs +- The full [Platform API](/docs/APIs-and-SDKs/Web-Console-API/backend) reference for programmatic access to everything in the Web Console +- The [MCP Server](/docs/APIs-and-SDKs/MCP-Server/overview) for driving ABsmartly from AI assistants like Claude, Cursor and Copilot Developers and engineers will spend most of their time here. @@ -64,7 +65,7 @@ Each guide explains how to connect ABsmartly with your existing systems. If you are new to ABsmartly, we recommend starting with: 1. **Product Documentation → Getting Started** -2. **APIs & SDKs** (choose the SDK you need to integrate first) +2. **Developer Tools** (choose the SDK you need to integrate first) 3. **Experiments** to learn how to create, run and analyse your first test You can explore the rest of the documentation as you become more familiar with the platform. diff --git a/docusaurus.config.js b/docusaurus.config.js index da4037c1..18816c27 100644 --- a/docusaurus.config.js +++ b/docusaurus.config.js @@ -149,7 +149,7 @@ const config = { { to: "docs/APIs-and-SDKs/overview", position: "left", - label: "APIs & SDKs", + label: "Developer Tools", }, { to: "https://absmartly.com/blog", @@ -181,6 +181,10 @@ const config = { label: "API Docs", to: "/docs/APIs-and-SDKs/SDK-API/absmartly-collector-api", }, + { + label: "MCP Server", + to: "/docs/APIs-and-SDKs/MCP-Server/overview", + }, ], }, {