Skip to content

Commit 9e285f7

Browse files
Update blog 6.1 and API submodule for Microsoft.Extensions.AI
- Blog 6.1: reflect MEA IChatClient usage, GA status, OllamaSharp 5.x implementation - API submodule: point to feature/mea-update with MEA-based OllamaAiService
1 parent febbf0b commit 9e285f7

2 files changed

Lines changed: 59 additions & 45 deletions

File tree

blogs/series-6-ai-app-features/6.1-dotnet-ai-foundation.md

Lines changed: 58 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Run a Local LLM in Your .NET 10 API with Ollama
22

3-
## How OllamaSharp and a Custom Service Interface Make Your .NET 10 API AI-Ready
3+
## How Microsoft.Extensions.AI, OllamaSharp, and a Custom Service Interface Make Your .NET 10 API AI-Ready
44

55
Every developer wants AI in their app. The problem is getting started: API keys, cloud costs, rate limits, and the fear of betting your architecture on one vendor. What if you could add a working AI endpoint to your .NET 10 API in under an hour — for free, running entirely on your laptop?
66

@@ -16,11 +16,12 @@ This article is part of the **AngularNetTutorial** series. The full-stack tutori
1616

1717
## 🎓 What You'll Learn
1818

19-
* **OllamaSharp streaming** — How `IOllamaApiClient` streams tokens from Ollama using `IAsyncEnumerable<>` so responses appear progressively, not all at once
19+
* **Microsoft.Extensions.AI (MEA)** — The GA abstraction layer shipping with .NET 10 that gives you a single `IChatClient` interface across all AI providers
20+
* **OllamaSharp + MEA** — How OllamaSharp 5.x natively implements `IChatClient`, making Ollama a drop-in MEA provider with no extra package
2021
* **Ollama integration** — Pull a free local model and connect it to your .NET API in minutes
21-
* **Feature flag gating** — Why `[FeatureGate("AiEnabled")]` is the safest way to ship AI without breaking existing users
22+
* **Feature flag gating** — Why per-method `IsEnabledAsync` checks return `503` instead of the misleading `404` from `[FeatureGate]`
2223
* **Clean Architecture placement** — Where AI interfaces, implementations, and controllers belong in the layer structure
23-
* **Custom `IAiChatService` interface** — How defining your own service interface in the Application layer hides OllamaSharp from callers and makes the implementation swappable
24+
* **Custom `IAiChatService` interface** — How defining your own service interface in the Application layer hides MEA/OllamaSharp from callers and makes the implementation swappable
2425

2526
---
2627

@@ -54,18 +55,21 @@ Beyond getting started, there's an architectural risk: if your AI code reaches d
5455

5556
## 💡 The Solution
5657

57-
[OllamaSharp](https://github.com/awaescher/OllamaSharp) is a .NET client for Ollama that exposes `IOllamaApiClient` and native token streaming via `IAsyncEnumerable<>`. We use it directly in `Infrastructure.Shared` — one package, no additional provider abstraction library needed.
58+
**[Microsoft.Extensions.AI](https://learn.microsoft.com/en-us/dotnet/ai/microsoft-extensions-ai)** (MEA) is the standard AI abstraction layer that ships GA with .NET 10. It defines `IChatClient` — a single interface for chat completions that works across OpenAI, Azure OpenAI, Ollama, and any other provider. You code against `IChatClient`; switching providers is a one-line DI change.
59+
60+
**[OllamaSharp](https://github.com/awaescher/OllamaSharp) 5.x** natively implements `IChatClient` from MEA. The former `Microsoft.Extensions.AI.Ollama` provider package has been deprecated — OllamaSharp is now the recommended path. That means you need just two NuGet packages in `Infrastructure.Shared`: `Microsoft.Extensions.AI.Abstractions` for the interface and `OllamaSharp` for the implementation.
5861

5962
[Ollama](https://ollama.com) runs open-weight models like `llama3.2` locally. No API key. No cloud. Works offline. Perfect for tutorials and development.
6063

61-
Provider independence comes from our own `IAiChatService` interface defined in the Application layer. `OllamaAiService` implements it using OllamaSharp. To swap providers (e.g., Azure OpenAI in production), you write a new implementation of `IAiChatService` in Infrastructure.Shared and change the DI registration — the Application layer, handlers, and controller are untouched.
64+
Provider independence has two layers: MEA's `IChatClient` (standard .NET 10 interface) and our own `IAiChatService` (defined in the Application layer). `OllamaAiService` receives `IChatClient` from DI and wraps it in the Application-layer contract. To swap from Ollama to Azure OpenAI, you register a different `IChatClient` implementation `OllamaAiService` itself is untouched, and the Application layer, handlers, and controller never change.
6265

63-
We gate the entire `AiController` behind a `[FeatureGate("AiEnabled")]` attribute. When `"AiEnabled": false` (the default), the controller doesn't even respond to requests — no Ollama connection is attempted, the rest of the API is unaffected.
66+
We gate AI per-method with `IFeatureManagerSnapshot.IsEnabledAsync("AiEnabled")` rather than a class-level `[FeatureGate]` attribute. The attribute returns `404 Not Found` when disabled — confusing for a known endpoint. The per-method check returns `503 Service Unavailable` with a `detail` message that tells the developer exactly what to enable and where.
6467

6568
**Key benefits:**
6669

6770
***Zero cost** — Ollama is free; no API key, no credit card, no rate limits
68-
***Provider-independent Application layer**`IAiChatService` hides OllamaSharp from all callers; swap the implementation without touching handlers or controllers
71+
***Standard .NET 10 AI abstraction** — MEA's `IChatClient` is GA and built into the platform; no preview dependencies
72+
***Provider swap in one line** — Register a different `IChatClient` to move from Ollama to Azure OpenAI; no service code changes
6973
***Safe coexistence** — Feature flag default `false` means original tutorial (Series 0–5) works unchanged
7074
***Clean Architecture** — Interface in Application, implementation in Infrastructure.Shared, controller in WebApi
7175

@@ -92,15 +96,22 @@ curl http://localhost:11434/api/tags
9296

9397
### Step 2: Add NuGet Packages
9498

95-
Add OllamaSharp to the Infrastructure.Shared project — this is the only AI package needed:
99+
Add two packages to the Infrastructure.Shared project — the MEA abstraction and the OllamaSharp implementation:
96100

97101
**`TalentManagementAPI.Infrastructure.Shared.csproj`**:
98102

99103
```xml
104+
<PackageReference Include="Microsoft.Extensions.AI.Abstractions" Version="10.3.0" />
100105
<PackageReference Include="OllamaSharp" Version="5.3.4" />
101106
```
102107

103-
**Why OllamaSharp instead of `Microsoft.Extensions.AI`?** OllamaSharp provides native streaming support via `IAsyncEnumerable<>` (`await foreach`), so tokens stream out of Ollama as they are generated — important when local model responses take several seconds. `Microsoft.Extensions.AI` is the right abstraction when you need to swap between Azure OpenAI, OpenAI, and Ollama without touching service code. For this tutorial, OllamaSharp keeps the dependency footprint minimal: one package, only in Infrastructure.Shared, no provider registration boilerplate in Program.cs.
108+
**Why both packages?**
109+
110+
`Microsoft.Extensions.AI.Abstractions` ships GA with .NET 10. It defines `IChatClient` — the standard interface for chat completions across all AI providers. Coding against `IChatClient` means your service code never changes when you switch from Ollama to Azure OpenAI or any other provider.
111+
112+
`OllamaSharp` 5.x natively implements `IChatClient`. The former `Microsoft.Extensions.AI.Ollama` provider package has been **deprecated** — OllamaSharp is the recommended path for Ollama integration. This means you do not need a separate adapter package: `OllamaApiClient` from OllamaSharp implements both `IChatClient` (for chat) and `IOllamaApiClient` (for embeddings), and you register it once as a singleton.
113+
114+
**No provider boilerplate in Program.cs.** All AI registration stays inside `AddSharedInfrastructure` — the WebApi project adds zero AI-specific setup.
104115

105116
### Step 3: Add Feature Flag and Ollama Config
106117

@@ -151,60 +162,48 @@ namespace TalentManagementAPI.Application.Interfaces
151162
Create `TalentManagementAPI.Infrastructure.Shared/Services/OllamaAiService.cs`:
152163

153164
```csharp
165+
#nullable enable
166+
using Microsoft.Extensions.AI;
154167
using TalentManagementAPI.Application.Interfaces;
155168

156169
namespace TalentManagementAPI.Infrastructure.Shared.Services
157170
{
158171
public class OllamaAiService : IAiChatService
159172
{
160-
private readonly IOllamaApiClient _ollamaApiClient;
173+
private readonly IChatClient _chatClient;
161174

162-
public OllamaAiService(IOllamaApiClient ollamaApiClient)
175+
public OllamaAiService(IChatClient chatClient)
163176
{
164-
_ollamaApiClient = ollamaApiClient;
177+
_chatClient = chatClient;
165178
}
166179

167180
public async Task<string> ChatAsync(string message, string? systemPrompt = null,
168181
CancellationToken cancellationToken = default)
169182
{
170-
var messages = new List<Message>();
183+
var messages = new List<ChatMessage>();
171184

172185
if (!string.IsNullOrWhiteSpace(systemPrompt))
173-
messages.Add(new Message(new ChatRole("system"), systemPrompt));
174-
175-
messages.Add(new Message(new ChatRole("user"), message));
176-
177-
var request = new ChatRequest
178-
{
179-
Model = _ollamaApiClient.SelectedModel,
180-
Messages = messages,
181-
Stream = true
182-
};
183-
184-
var responseBuilder = new MessageBuilder();
186+
messages.Add(new ChatMessage(Microsoft.Extensions.AI.ChatRole.System, systemPrompt));
185187

186-
await foreach (var response in _ollamaApiClient.ChatAsync(request, cancellationToken)
187-
.WithCancellation(cancellationToken))
188-
{
189-
if (response?.Message is not null)
190-
responseBuilder.Append(response);
191-
}
188+
messages.Add(new ChatMessage(Microsoft.Extensions.AI.ChatRole.User, message));
192189

193-
return responseBuilder.HasValue
194-
? responseBuilder.ToMessage().Content ?? string.Empty
195-
: string.Empty;
190+
var response = await _chatClient.GetResponseAsync(messages, cancellationToken: cancellationToken);
191+
return response.Text ?? string.Empty;
196192
}
197193
}
198194
}
199195
```
200196

201-
**What this does:** `OllamaAiService` takes `IOllamaApiClient` from DI (registered in Step 6). OllamaSharp streams tokens back using `IAsyncEnumerable<>` — the `await foreach` loop accumulates each chunk into a `MessageBuilder`, then returns the fully assembled reply. An optional system prompt lets callers control the AI's persona or constraints without the service knowing anything about the caller's intent.
197+
**What this does:** `OllamaAiService` takes `IChatClient` (the MEA standard interface) from DI — registered in Step 6 as `OllamaApiClient`. It builds a message list, optionally prepending a system prompt that lets callers control the AI's persona or constraints. `GetResponseAsync` is the MEA 10.x API for a single-turn chat completion. `response.Text` returns the assistant's reply as a plain string.
198+
199+
**Why `Microsoft.Extensions.AI.ChatRole.System` instead of just `ChatRole.System`?** Both OllamaSharp and MEA define a `ChatRole` type. Fully qualifying the namespace resolves the ambiguity cleanly without needing a using alias.
202200

203201
### Step 6: Register Services
204202

205-
In `Infrastructure.Shared/ServiceRegistration.cs`, register `IOllamaApiClient` and wire `IAiChatService` to a caching decorator that wraps `OllamaAiService`:
203+
In `Infrastructure.Shared/ServiceRegistration.cs`, register `OllamaApiClient` as a concrete singleton and expose it as both `IChatClient` and `IOllamaApiClient` — so both interfaces resolve to the same instance:
206204

207205
```csharp
206+
using Microsoft.Extensions.AI;
208207
using TalentManagementAPI.Application.Interfaces;
209208
using TalentManagementAPI.Infrastructure.Shared.Services;
210209

@@ -215,13 +214,16 @@ public static void AddSharedInfrastructure(this IServiceCollection services, ICo
215214
services.AddTransient<IEmailService, EmailService>();
216215
services.AddTransient<IMockService, MockService>();
217216

218-
// Register the Ollama client as a singleton — one connection reused across requests
219-
services.AddSingleton<IOllamaApiClient>(_ =>
217+
// OllamaApiClient implements both IChatClient (Microsoft.Extensions.AI) and IOllamaApiClient.
218+
// Register as a singleton so both interfaces resolve to the same instance.
219+
services.AddSingleton<OllamaApiClient>(_ =>
220220
{
221221
var baseUrl = config["Ollama:BaseUrl"] ?? "http://localhost:11434";
222222
var model = config["Ollama:Model"] ?? "llama3.2";
223223
return new OllamaApiClient(new Uri(baseUrl), model);
224224
});
225+
services.AddSingleton<IChatClient>(sp => sp.GetRequiredService<OllamaApiClient>());
226+
services.AddSingleton<IOllamaApiClient>(sp => sp.GetRequiredService<OllamaApiClient>());
225227

226228
// Metadata scoped per-request so the controller can read cache hit/miss
227229
services.AddScoped<IAiResponseMetadata, AiResponseMetadata>();
@@ -243,9 +245,18 @@ In `WebApi/Program.cs`, the only AI-related line is the call to `AddSharedInfras
243245
```csharp
244246
builder.Services.AddApplicationLayer();
245247
builder.Services.AddPersistenceInfrastructure(builder.Configuration);
246-
builder.Services.AddSharedInfrastructure(builder.Configuration); // ← registers IOllamaApiClient + IAiChatService
248+
builder.Services.AddSharedInfrastructure(builder.Configuration); // ← registers IChatClient + IAiChatService
247249
```
248250

251+
**Why three registrations for one object?** `OllamaApiClient` implements two interfaces from different libraries:
252+
253+
* `IChatClient` (from `Microsoft.Extensions.AI`) — used by `OllamaAiService` for chat completions
254+
* `IOllamaApiClient` (from OllamaSharp) — used by `OllamaEmbeddingService` for embedding generation (introduced in Series 6.5)
255+
256+
Registering the concrete type first as a singleton, then aliasing both interfaces to it, ensures both resolve to the same underlying instance — one HTTP connection, one model selection, shared across the app.
257+
258+
**To swap Ollama for Azure OpenAI in production:** Replace the three `AddSingleton` calls with your Azure OpenAI `IChatClient` registration. `OllamaAiService`, the Application layer, MediatR handlers, and the controller require zero changes.
259+
249260
**What the caching decorator does:** `CachingAiChatService` wraps `OllamaAiService`. On the first call for a given `(message, systemPrompt)` pair, it calls Ollama and stores the reply. On subsequent identical calls within the TTL window, it returns the cached reply — skipping the 1–4 second Ollama inference. The `IAiResponseMetadata` flag tells the controller whether the response was a cache hit, which is surfaced as the `X-AI-Cache: HIT/MISS` response header.
250261

251262
### Step 7: Create the AI Controller
@@ -362,7 +373,7 @@ curl -X POST https://localhost:44378/api/v1/ai/chat \
362373
-d '{"message": "Explain JWT tokens in one paragraph."}'
363374
```
364375

365-
**To verify the feature flag** — set `"AiEnabled": false`, restart the API, and try the same curl. You'll get a `404` — the controller is invisible.
376+
**To verify the feature flag** — set `"AiEnabled": false`, restart the API, and try the same curl. You'll get a `503 Service Unavailable` with a `detail` message explaining exactly what to enable.
366377

367378
---
368379

@@ -384,18 +395,21 @@ curl -X POST https://localhost:44378/api/v1/ai/chat \
384395

385396
## 🌟 Why This Matters
386397

387-
OllamaSharp's native `IAsyncEnumerable<>` streaming means tokens appear progressively as Ollama generates them — critical when a local model takes several seconds per response. Buffering the entire reply before returning it would feel broken to users.
398+
**Microsoft.Extensions.AI ships GA with .NET 10** — it is not a preview library. `IChatClient` is the platform-standard interface for chat completions, with first-party support from Microsoft and implementations across OpenAI, Azure OpenAI, Ollama (via OllamaSharp), and more. Building against `IChatClient` means your service layer is future-proof: new providers ship as NuGet packages, and switching is a one-line DI change.
399+
400+
**OllamaSharp 5.x natively implements `IChatClient`.** The former `Microsoft.Extensions.AI.Ollama` adapter package has been deprecated in favor of OllamaSharp directly. That means one fewer dependency and no abstraction-over-an-abstraction: `OllamaApiClient` *is* the `IChatClient` — no adapter wrapper needed.
388401

389-
The custom `IAiChatService` interface pattern is the key architectural decision. It places the Ollama dependency entirely inside `Infrastructure.Shared`. Application-layer code (handlers, queries) and the controller depend only on the interface — they are completely unaware of OllamaSharp. When you are ready to move to a cloud provider (Azure OpenAI, Anthropic), you add a new infrastructure implementation and update one DI registration. Nothing else changes.
402+
The custom `IAiChatService` interface adds a second layer of provider independence specific to this application's contract (`ChatAsync` with an optional system prompt). Application-layer code (handlers, queries) and the controller depend only on `IAiChatService` — they are completely unaware of MEA or OllamaSharp. When you are ready to move to a cloud provider, you register a different `IChatClient` (one line in ServiceRegistration.cs) and the rest of the codebase is untouched.
390403

391404
For tutorial purposes, Ollama removes the biggest barrier to learning: access. Every developer on every OS can pull `llama3.2`, type `ollama serve`, and have a working LLM in their local environment. No billing, no configuration, no waiting for API access.
392405

393406
The feature flag pattern ensures this is safe to ship: the codebase always builds, always runs, and the original Series 0–5 experience is completely unchanged. AI features activate on demand.
394407

395408
**Transferable skills:**
396409

410+
* **Microsoft.Extensions.AI**`IChatClient` is the standard .NET 10 AI interface; the same pattern applies to OpenAI, Azure OpenAI, and any future MEA provider
397411
* **Custom service interface for AI** — The `IAiChatService` pattern applies to any AI provider; define the contract in Application, implement in Infrastructure
398-
* **Feature flag architecture**The `[FeatureGate]` pattern applies to any experimental or optional feature
412+
* **Per-method feature flag checks**`IsEnabledAsync` returning `503 Service Unavailable` is more developer-friendly than the `404` from `[FeatureGate]` on a class
399413
* **Clean Architecture for external services** — Interface in Application, implementation in Infrastructure, DI registration in WebApi
400414

401415
---

0 commit comments

Comments
 (0)