You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Update blog 6.1 and API submodule for Microsoft.Extensions.AI
- Blog 6.1: reflect MEA IChatClient usage, GA status, OllamaSharp 5.x implementation
- API submodule: point to feature/mea-update with MEA-based OllamaAiService
Copy file name to clipboardExpand all lines: blogs/series-6-ai-app-features/6.1-dotnet-ai-foundation.md
+58-44Lines changed: 58 additions & 44 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,6 +1,6 @@
1
1
# Run a Local LLM in Your .NET 10 API with Ollama
2
2
3
-
## How OllamaSharp and a Custom Service Interface Make Your .NET 10 API AI-Ready
3
+
## How Microsoft.Extensions.AI, OllamaSharp, and a Custom Service Interface Make Your .NET 10 API AI-Ready
4
4
5
5
Every developer wants AI in their app. The problem is getting started: API keys, cloud costs, rate limits, and the fear of betting your architecture on one vendor. What if you could add a working AI endpoint to your .NET 10 API in under an hour — for free, running entirely on your laptop?
6
6
@@ -16,11 +16,12 @@ This article is part of the **AngularNetTutorial** series. The full-stack tutori
16
16
17
17
## 🎓 What You'll Learn
18
18
19
-
***OllamaSharp streaming** — How `IOllamaApiClient` streams tokens from Ollama using `IAsyncEnumerable<>` so responses appear progressively, not all at once
19
+
***Microsoft.Extensions.AI (MEA)** — The GA abstraction layer shipping with .NET 10 that gives you a single `IChatClient` interface across all AI providers
20
+
***OllamaSharp + MEA** — How OllamaSharp 5.x natively implements `IChatClient`, making Ollama a drop-in MEA provider with no extra package
20
21
***Ollama integration** — Pull a free local model and connect it to your .NET API in minutes
21
-
***Feature flag gating** — Why `[FeatureGate("AiEnabled")]` is the safest way to ship AI without breaking existing users
22
+
***Feature flag gating** — Why per-method `IsEnabledAsync` checks return `503` instead of the misleading `404` from `[FeatureGate]`
22
23
***Clean Architecture placement** — Where AI interfaces, implementations, and controllers belong in the layer structure
23
-
***Custom `IAiChatService` interface** — How defining your own service interface in the Application layer hides OllamaSharp from callers and makes the implementation swappable
24
+
***Custom `IAiChatService` interface** — How defining your own service interface in the Application layer hides MEA/OllamaSharp from callers and makes the implementation swappable
24
25
25
26
---
26
27
@@ -54,18 +55,21 @@ Beyond getting started, there's an architectural risk: if your AI code reaches d
54
55
55
56
## 💡 The Solution
56
57
57
-
[OllamaSharp](https://github.com/awaescher/OllamaSharp) is a .NET client for Ollama that exposes `IOllamaApiClient` and native token streaming via `IAsyncEnumerable<>`. We use it directly in `Infrastructure.Shared` — one package, no additional provider abstraction library needed.
58
+
**[Microsoft.Extensions.AI](https://learn.microsoft.com/en-us/dotnet/ai/microsoft-extensions-ai)** (MEA) is the standard AI abstraction layer that ships GA with .NET 10. It defines `IChatClient` — a single interface for chat completions that works across OpenAI, Azure OpenAI, Ollama, and any other provider. You code against `IChatClient`; switching providers is a one-line DI change.
59
+
60
+
**[OllamaSharp](https://github.com/awaescher/OllamaSharp) 5.x** natively implements `IChatClient` from MEA. The former `Microsoft.Extensions.AI.Ollama` provider package has been deprecated — OllamaSharp is now the recommended path. That means you need just two NuGet packages in `Infrastructure.Shared`: `Microsoft.Extensions.AI.Abstractions` for the interface and `OllamaSharp` for the implementation.
58
61
59
62
[Ollama](https://ollama.com) runs open-weight models like `llama3.2` locally. No API key. No cloud. Works offline. Perfect for tutorials and development.
60
63
61
-
Provider independence comes from our own `IAiChatService`interface defined in the Application layer. `OllamaAiService`implements it using OllamaSharp. To swap providers (e.g., Azure OpenAI in production), you write a new implementation of `IAiChatService` in Infrastructure.Shared and change the DI registration — the Application layer, handlers, and controller are untouched.
64
+
Provider independence has two layers: MEA's `IChatClient` (standard .NET 10 interface) and our own `IAiChatService`(defined in the Application layer). `OllamaAiService`receives `IChatClient` from DI and wraps it in the Application-layer contract. To swap from Ollama to Azure OpenAI, you register a different `IChatClient`implementation — `OllamaAiService` itself is untouched, and the Application layer, handlers, and controller never change.
62
65
63
-
We gate the entire `AiController` behind a `[FeatureGate("AiEnabled")]` attribute. When `"AiEnabled": false` (the default), the controller doesn't even respond to requests — no Ollama connection is attempted, the rest of the API is unaffected.
66
+
We gate AI per-method with `IFeatureManagerSnapshot.IsEnabledAsync("AiEnabled")` rather than a class-level `[FeatureGate]` attribute. The attribute returns `404 Not Found` when disabled — confusing for a known endpoint. The per-method check returns `503 Service Unavailable` with a `detail` message that tells the developer exactly what to enable and where.
64
67
65
68
**Key benefits:**
66
69
67
70
* ✅ **Zero cost** — Ollama is free; no API key, no credit card, no rate limits
68
-
* ✅ **Provider-independent Application layer** — `IAiChatService` hides OllamaSharp from all callers; swap the implementation without touching handlers or controllers
71
+
* ✅ **Standard .NET 10 AI abstraction** — MEA's `IChatClient` is GA and built into the platform; no preview dependencies
72
+
* ✅ **Provider swap in one line** — Register a different `IChatClient` to move from Ollama to Azure OpenAI; no service code changes
69
73
* ✅ **Safe coexistence** — Feature flag default `false` means original tutorial (Series 0–5) works unchanged
70
74
* ✅ **Clean Architecture** — Interface in Application, implementation in Infrastructure.Shared, controller in WebApi
**Why OllamaSharp instead of `Microsoft.Extensions.AI`?** OllamaSharp provides native streaming support via `IAsyncEnumerable<>` (`await foreach`), so tokens stream out of Ollama as they are generated — important when local model responses take several seconds. `Microsoft.Extensions.AI` is the right abstraction when you need to swap between Azure OpenAI, OpenAI, and Ollama without touching service code. For this tutorial, OllamaSharp keeps the dependency footprint minimal: one package, only in Infrastructure.Shared, no provider registration boilerplate in Program.cs.
108
+
**Why both packages?**
109
+
110
+
`Microsoft.Extensions.AI.Abstractions` ships GA with .NET 10. It defines `IChatClient` — the standard interface for chat completions across all AI providers. Coding against `IChatClient` means your service code never changes when you switch from Ollama to Azure OpenAI or any other provider.
111
+
112
+
`OllamaSharp` 5.x natively implements `IChatClient`. The former `Microsoft.Extensions.AI.Ollama` provider package has been **deprecated** — OllamaSharp is the recommended path for Ollama integration. This means you do not need a separate adapter package: `OllamaApiClient` from OllamaSharp implements both `IChatClient` (for chat) and `IOllamaApiClient` (for embeddings), and you register it once as a singleton.
113
+
114
+
**No provider boilerplate in Program.cs.** All AI registration stays inside `AddSharedInfrastructure` — the WebApi project adds zero AI-specific setup.
**What this does:**`OllamaAiService` takes `IOllamaApiClient` from DI (registered in Step 6). OllamaSharp streams tokens back using `IAsyncEnumerable<>` — the `await foreach` loop accumulates each chunk into a `MessageBuilder`, then returns the fully assembled reply. An optional system prompt lets callers control the AI's persona or constraints without the service knowing anything about the caller's intent.
197
+
**What this does:**`OllamaAiService` takes `IChatClient` (the MEA standard interface) from DI — registered in Step 6 as `OllamaApiClient`. It builds a message list, optionally prepending a system prompt that lets callers control the AI's persona or constraints. `GetResponseAsync` is the MEA 10.x API for a single-turn chat completion. `response.Text` returns the assistant's reply as a plain string.
198
+
199
+
**Why `Microsoft.Extensions.AI.ChatRole.System` instead of just `ChatRole.System`?** Both OllamaSharp and MEA define a `ChatRole` type. Fully qualifying the namespace resolves the ambiguity cleanly without needing a using alias.
202
200
203
201
### Step 6: Register Services
204
202
205
-
In `Infrastructure.Shared/ServiceRegistration.cs`, register `IOllamaApiClient`and wire `IAiChatService` to a caching decorator that wraps `OllamaAiService`:
203
+
In `Infrastructure.Shared/ServiceRegistration.cs`, register `OllamaApiClient` as a concrete singleton and expose it as both `IChatClient` and `IOllamaApiClient` — so both interfaces resolve to the same instance:
**Why three registrations for one object?**`OllamaApiClient` implements two interfaces from different libraries:
252
+
253
+
*`IChatClient` (from `Microsoft.Extensions.AI`) — used by `OllamaAiService` for chat completions
254
+
*`IOllamaApiClient` (from OllamaSharp) — used by `OllamaEmbeddingService` for embedding generation (introduced in Series 6.5)
255
+
256
+
Registering the concrete type first as a singleton, then aliasing both interfaces to it, ensures both resolve to the same underlying instance — one HTTP connection, one model selection, shared across the app.
257
+
258
+
**To swap Ollama for Azure OpenAI in production:** Replace the three `AddSingleton` calls with your Azure OpenAI `IChatClient` registration. `OllamaAiService`, the Application layer, MediatR handlers, and the controller require zero changes.
259
+
249
260
**What the caching decorator does:**`CachingAiChatService` wraps `OllamaAiService`. On the first call for a given `(message, systemPrompt)` pair, it calls Ollama and stores the reply. On subsequent identical calls within the TTL window, it returns the cached reply — skipping the 1–4 second Ollama inference. The `IAiResponseMetadata` flag tells the controller whether the response was a cache hit, which is surfaced as the `X-AI-Cache: HIT/MISS` response header.
250
261
251
262
### Step 7: Create the AI Controller
@@ -362,7 +373,7 @@ curl -X POST https://localhost:44378/api/v1/ai/chat \
362
373
-d '{"message": "Explain JWT tokens in one paragraph."}'
363
374
```
364
375
365
-
**To verify the feature flag** — set `"AiEnabled": false`, restart the API, and try the same curl. You'll get a `404` — the controller is invisible.
376
+
**To verify the feature flag** — set `"AiEnabled": false`, restart the API, and try the same curl. You'll get a `503 Service Unavailable` with a `detail` message explaining exactly what to enable.
366
377
367
378
---
368
379
@@ -384,18 +395,21 @@ curl -X POST https://localhost:44378/api/v1/ai/chat \
384
395
385
396
## 🌟 Why This Matters
386
397
387
-
OllamaSharp's native `IAsyncEnumerable<>` streaming means tokens appear progressively as Ollama generates them — critical when a local model takes several seconds per response. Buffering the entire reply before returning it would feel broken to users.
398
+
**Microsoft.Extensions.AI ships GA with .NET 10** — it is not a preview library. `IChatClient` is the platform-standard interface for chat completions, with first-party support from Microsoft and implementations across OpenAI, Azure OpenAI, Ollama (via OllamaSharp), and more. Building against `IChatClient` means your service layer is future-proof: new providers ship as NuGet packages, and switching is a one-line DI change.
399
+
400
+
**OllamaSharp 5.x natively implements `IChatClient`.** The former `Microsoft.Extensions.AI.Ollama` adapter package has been deprecated in favor of OllamaSharp directly. That means one fewer dependency and no abstraction-over-an-abstraction: `OllamaApiClient`*is* the `IChatClient` — no adapter wrapper needed.
388
401
389
-
The custom `IAiChatService` interface pattern is the key architectural decision. It places the Ollama dependency entirely inside `Infrastructure.Shared`. Application-layer code (handlers, queries) and the controller depend only on the interface — they are completely unaware of OllamaSharp. When you are ready to move to a cloud provider (Azure OpenAI, Anthropic), you add a new infrastructure implementation and update one DI registration. Nothing else changes.
402
+
The custom `IAiChatService` interface adds a second layer of provider independence specific to this application's contract (`ChatAsync` with an optional system prompt). Application-layer code (handlers, queries) and the controller depend only on `IAiChatService`— they are completely unaware of MEA or OllamaSharp. When you are ready to move to a cloud provider, you register a different `IChatClient` (one line in ServiceRegistration.cs) and the rest of the codebase is untouched.
390
403
391
404
For tutorial purposes, Ollama removes the biggest barrier to learning: access. Every developer on every OS can pull `llama3.2`, type `ollama serve`, and have a working LLM in their local environment. No billing, no configuration, no waiting for API access.
392
405
393
406
The feature flag pattern ensures this is safe to ship: the codebase always builds, always runs, and the original Series 0–5 experience is completely unchanged. AI features activate on demand.
394
407
395
408
**Transferable skills:**
396
409
410
+
***Microsoft.Extensions.AI** — `IChatClient` is the standard .NET 10 AI interface; the same pattern applies to OpenAI, Azure OpenAI, and any future MEA provider
397
411
***Custom service interface for AI** — The `IAiChatService` pattern applies to any AI provider; define the contract in Application, implement in Infrastructure
398
-
***Feature flag architecture** — The `[FeatureGate]` pattern applies to any experimental or optional feature
412
+
***Per-method feature flag checks** — `IsEnabledAsync` returning `503 Service Unavailable` is more developer-friendly than the `404` from `[FeatureGate]` on a class
399
413
***Clean Architecture for external services** — Interface in Application, implementation in Infrastructure, DI registration in WebApi
0 commit comments