server: Anthropic-compatible Messages API (/v1/messages) (#325) - #326
server: Anthropic-compatible Messages API (/v1/messages) (#325)#326jamesburton wants to merge 2 commits into
Conversation
Add an Anthropic Messages API layer to DotLLM.Server alongside the existing
OpenAI surface, so clients written for the `anthropic` SDKs can talk to dotLLM
unchanged. Purely additive — the engine, chat template, sampler and
tool-calling pipeline are reused verbatim; only the wire format differs.
Endpoints:
- POST /v1/messages — non-streaming (JSON) and streaming (named SSE events:
message_start, content_block_start/_delta/_stop, message_delta, message_stop).
- POST /v1/messages/count_tokens — { "input_tokens": N }.
Translation (AnthropicConverter):
- Top-level system (string or text-block array) -> leading system message.
- String-or-block message content; text/tool_use/tool_result blocks mapped to
ChatMessage/ToolCall and tool-role messages keyed by tool_use_id.
- tools/input_schema -> ToolDefinition; tool_choice auto/any/none/tool.
- FinishReason -> stop_reason (end_turn/max_tokens/stop_sequence/tool_use).
- Anthropic error envelope; AOT-clean source-gen DTOs in ServerJsonContext.
Tests: 26 unit tests (message flattening, tool_choice, stop_reason, tool_use
blocks, validation, response/error serialization shape).
Docs: docs/ANTHROPIC_API.md + SERVER.md/ROADMAP.md/README.md/CLAUDE.md sync.
Closes #325
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Adds an Anthropic-compatible Messages API surface to the server (request/response DTOs, converter, and endpoints), plus documentation and unit tests to validate the translation layer and JSON shapes.
Changes:
- Introduces
/v1/messagesand/v1/messages/count_tokensendpoints with non-streaming JSON and streaming SSE event support. - Adds Anthropic Messages API DTOs and a converter to map Anthropic wire format to dotLLM engine types.
- Updates JSON source-generation context and documentation; adds unit tests for converter/serialization/validation.
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/DotLLM.Tests.Unit/Server/AnthropicConverterTests.cs | Adds unit tests for Anthropic conversion, validation, and serialization shapes. |
| src/DotLLM.Server/ServerJsonContext.cs | Registers Anthropic DTOs/events for System.Text.Json source generation. |
| src/DotLLM.Server/Models/AnthropicMessagesModels.cs | Introduces DTOs for Anthropic requests/responses/errors/streaming events. |
| src/DotLLM.Server/Endpoints/MessagesEndpoint.cs | Implements Anthropic-compatible endpoints, streaming SSE, and request validation. |
| src/DotLLM.Server/EndpointExtensions.cs | Wires the new endpoint mapping into the server. |
| src/DotLLM.Server/AnthropicConverter.cs | Adds conversion logic between Anthropic DTOs and engine types. |
| docs/SERVER.md | Documents the new endpoints at a high level and links to full Anthropic API docs. |
| docs/ROADMAP.md | Marks Anthropic Messages API support as done and references docs. |
| docs/ANTHROPIC_API.md | Adds detailed API documentation for request/response and streaming event sequence. |
| README.md | Adds feature bullet + news entry about the new Anthropic-compatible API. |
| CLAUDE.md | Adds a documentation index entry for the new Anthropic API doc. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| public static JsonElement ParseInput(string? arguments) | ||
| { | ||
| if (string.IsNullOrWhiteSpace(arguments)) | ||
| return EmptyObject(); | ||
| try | ||
| { | ||
| using var doc = JsonDocument.Parse(arguments); | ||
| return doc.RootElement.Clone(); | ||
| } |
| internal static string? ValidateRequest(AnthropicMessagesRequest request, bool requireMaxTokens) | ||
| { | ||
| if (request.Messages is null || request.Messages.Length == 0) | ||
| return "messages: at least one message is required"; | ||
|
|
||
| if (request.Messages.Length > RequestValidator.MaxMessages) | ||
| return $"messages: array exceeds maximum of {RequestValidator.MaxMessages}"; | ||
|
|
||
| if (requireMaxTokens) | ||
| { | ||
| if (!request.MaxTokens.HasValue) | ||
| return "max_tokens: field required"; | ||
| if (request.MaxTokens.Value <= 0) | ||
| return "max_tokens: must be a positive integer"; | ||
| } | ||
|
|
||
| return null; | ||
| } |
| httpContext.Response.ContentType = "text/event-stream"; | ||
| httpContext.Response.Headers.CacheControl = "no-cache"; | ||
| httpContext.Response.Headers.Connection = "keep-alive"; |
| "tool_choice": {"type": "auto"}, | ||
| "stream": false, | ||
| "lora_adapter": "customer-support" | ||
| } |
| private static JsonElement EmptyObject() | ||
| { | ||
| using var doc = JsonDocument.Parse("{}"); | ||
| return doc.RootElement.Clone(); | ||
| } |
| private static async Task HandleStreamingAsync( | ||
| AnthropicMessagesRequest request, | ||
| TextGenerator generator, | ||
| ServerState state, | ||
| HttpContext httpContext, | ||
| string prompt, | ||
| DotLLM.Core.Configuration.InferenceOptions options, | ||
| string messageId, string modelId, | ||
| ToolDefinition[]? tools, | ||
| int promptTokenCount, | ||
| CancellationToken ct) |
Review follow-ups on the /v1/messages surface:
- ParseInput: Anthropic requires tool_use.input to be an object, so a non-object
JSON root (bare string/array/scalar from a model that ignored the schema) now
collapses to {} instead of being emitted as an invalid wire shape.
- ParseInput/EmptyObject: share one cloned empty-object JsonElement instead of
reparsing "{}" on every empty or invalid tool input.
- ValidateRequest: reject messages[].role other than user/assistant, and
messages[].content that is neither a string nor an array of blocks. The
converter passes role straight to the chat template, so an unchecked role let a
caller inject a system turn mid-conversation; an unchecked content kind
silently flattened to an empty message.
- Streaming: drop the `Connection: keep-alive` response header — it is a
connection-specific header that is illegal over HTTP/2 and HTTP/3, and SSE only
needs content-type + no-cache.
- Extract the SSE emission into internal WriteMessageStreamAsync with the token
source and the model-serialisation gate injected, so the event sequence is
testable without loading a model. Behaviour is unchanged: the gate still wraps
only the generation loop, so message_start is emitted before queueing.
- docs/ANTHROPIC_API.md: drop the lora_adapter example — per-request adapter
selection is not implemented on the server (the OpenAI surface does not honour
it either); document the role restriction instead.
Tests: 49 (was 26) — endpoint-level SSE tests asserting event names, ordering and
payload shapes for text-only, max_tokens, stop_sequence and tool_use streams,
plus ParseInput non-object and role/content validation regressions.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Thanks for the review — all six comments were assessed individually; five are addressed in 1.
|
Summary
Adds an Anthropic-compatible Messages API layer to
DotLLM.Server, served alongside the existing OpenAI-compatible endpoints (Step 34). Clients and SDKs written for the Anthropic Messages API (theanthropicPython/TS SDKs, anything targetingPOST /v1/messages) can now point at a dotLLM server unchanged.This is purely additive — the engine, tokenizer, chat-template, sampler and tool-calling pipeline are reused verbatim. Only a new HTTP DTO + translation surface is introduced; no existing endpoint, engine, or sampling behaviour changes.
Closes #325.
Endpoints
POST /v1/messages— non-streaming (JSON) and streaming (event-based SSE:message_start,content_block_start/_delta/_stop,message_delta,message_stop,ping).POST /v1/messages/count_tokens— returns{ "input_tokens": N }.Translation (
AnthropicConverter)system(string or text-block array) → leadingsystemmessage.contentas a string or a content-block array;text/tool_use/tool_resultblocks mapped to engineChatMessage/ToolCallandtool-role messages keyed bytool_use_id.tools[].input_schema→ToolDefinition;tool_choiceauto/any/none/tool→ engineToolChoice.FinishReason→stop_reason(end_turn/max_tokens/stop_sequence/tool_use).{"type":"error","error":{...}}); all DTOs registered in the source-genServerJsonContext(AOT-clean, no reflection).Tests & Docs
AnthropicConverterTests): message flattening (string/blocks/system),tool_choice,stop_reasonmapping,tool_useblock emission, request validation, and response/error serialization shape. Full server unit suite green.docs/ANTHROPIC_API.md(full mapping + streaming sequence) anddocs/SERVER.md/ROADMAP.md(Step 34b) /README.md/CLAUDE.mdupdates.Notes / Limitations
tool_useblocks are emitted after the text block closes rather than incrementally.image/ multimodal content blocks are not yet supported (no multimodal pipeline).🤖 Generated with Claude Code