Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions docs/features/session-persistence.md
Original file line number Diff line number Diff line change
Expand Up @@ -257,7 +257,9 @@ When resuming a session, you can optionally reconfigure many settings. This is u

### Auto tier persistence

With `model: "auto"`, the optional `capi.autoTier` setting selects an Auto routing preference: `efficiency`, `balance`, or `intelligence`. In Python, use `capi={"auto_tier": "balance"}`. This requires Copilot CLI `1.0.82-1` or later with V2 Auto routing; V1 Auto requests are unchanged.
With `model: "auto"`, the optional `capi.autoTier` setting selects an Auto routing preference: `efficiency`, `balance`, `intelligence`, or `fast`. In Python, use `capi={"auto_tier": "balance"}`. This setting applies to V2 Auto routing; V1 Auto requests are unchanged.

`fast` is an integrator-only latency preset, not a first-party GitHub Copilot product preference. The SDK does not decide Fast eligibility, inspect client identity, choose it as a default, or fall back to another tier when a runtime does not support it—an older runtime returns its native error unchanged.

The runtime persists the selected tier, so applications do not need to resend it on every resume:

Expand All @@ -272,7 +274,7 @@ The `session.start` and `session.resume` events expose the selected tier in thei

### Changing the Auto tier during a session

Call `setAutoTier` to change the routing preference on a live session without changing the selected model. Pass `null` (Python `None`, Go `nil`) to return to the provider's default Auto routing. This requires Copilot CLI `1.0.83-4` or later, which is newer than the `1.0.82-1` needed to select a tier when creating or resuming a session.
Call `setAutoTier` to change the routing preference on a live session without changing the selected model. Pass `null` (Python `None`, Go `nil`) to return to the provider's default Auto routing.

```typescript
const result = await session.setAutoTier("intelligence");
Expand Down
6 changes: 5 additions & 1 deletion dotnet/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -317,9 +317,13 @@ await session2.DisposeAsync();

## Auto routing tiers

The canonical values are `AutoTier.Efficiency`, `AutoTier.Balance`, `AutoTier.Intelligence`, and `AutoTier.Fast`, which send `efficiency`, `balance`, `intelligence`, and `fast` on the wire. Fast is an integrator-only latency preset, not a fourth first-party GitHub Copilot preference. The SDK forwards the requested value without deciding eligibility or inspecting client identity. An externally supplied older runtime returns its native runtime or JSON-RPC error; the SDK does not downgrade or silently ignore the request.

Omitting the tier on create uses the runtime default rather than Balance. A cold resume restores the persisted tier unless the resume request supplies an explicit override.

Change the Auto routing preference without changing the selected model. The runtime does not apply the preference immediately: it records the request and commits it only when a later user turn using the `auto` model successfully obtains a usable model from the provider, so a `pending` status confirms acceptance rather than effect. Only the most recent request survives.

Watch for the outcome through the `session.model_change` event on success or the ephemeral `session.auto_tier_switch_failed` event on failure. Read the authoritative committed, pending, and activating preferences at any time through the session's `model.getCurrent` RPC method.
Watch for the outcome through the `session.model_change` event on success or the ephemeral `session.auto_tier_switch_failed` event on failure. A failed activation leaves the incumbent effective tier unchanged. Read the authoritative committed, pending, and activating preferences at any time through the session's `model.getCurrent` RPC method.

```csharp
var result = await session.SetAutoTierAsync(AutoTier.Intelligence);
Expand Down
229 changes: 225 additions & 4 deletions dotnet/test/E2E/AutoTierE2ETests.cs
Original file line number Diff line number Diff line change
Expand Up @@ -45,10 +45,16 @@ public async Task Should_Stage_And_Reset_Auto_Tier_Preference()
await AssertPendingAutoTierAsync(session, AutoTier.Efficiency);

// A second request replaces the first and reports the one it displaced.
var superseded = await session.SetAutoTierAsync(AutoTier.Intelligence);
var superseded = await session.SetAutoTierAsync(AutoTier.Fast);
Assert.Equal(ModelSwitchAutoTierStatus.Pending, superseded.Status);
Assert.Equal(AutoTier.Intelligence, superseded.PendingAutoTier);
Assert.Equal(AutoTier.Fast, superseded.PendingAutoTier);
Assert.Equal(AutoTier.Efficiency, superseded.SupersededAutoTier);
await AssertPendingAutoTierAsync(session, AutoTier.Fast);

var replacedFast = await session.SetAutoTierAsync(AutoTier.Intelligence);
Assert.Equal(ModelSwitchAutoTierStatus.Pending, replacedFast.Status);
Assert.Equal(AutoTier.Intelligence, replacedFast.PendingAutoTier);
Assert.Equal(AutoTier.Fast, replacedFast.SupersededAutoTier);
await AssertPendingAutoTierAsync(session, AutoTier.Intelligence);

// A null tier returns the session to provider-default routing. The status is
Expand Down Expand Up @@ -77,12 +83,227 @@ public async Task Should_Preserve_Auto_Tier_When_Set_Model_Omits_It()
await AssertPendingAutoTierAsync(session, AutoTier.Balance);

// Supplying a tier replaces it.
await session.SetModelAsync("auto", new SetModelOptions { AutoTier = AutoTier.Intelligence });
await AssertPendingAutoTierAsync(session, AutoTier.Intelligence);
await session.SetModelAsync("auto", new SetModelOptions { AutoTier = AutoTier.Fast });
await AssertPendingAutoTierAsync(session, AutoTier.Fast);

// ResetAutoTier clears it. Omission, a value, and a reset are three distinct
// outcomes, which is why a single nullable property cannot express the request.
await session.SetModelAsync("auto", new SetModelOptions { ResetAutoTier = true });
await AssertPendingAutoTierAsync(session, null);
}

[Fact]
public async Task Should_Restore_And_Override_Fast_Auto_Tier_On_Cold_Resume()
{
var initialClient = Ctx.CreateClient();
string fastSessionId;
string tierlessSessionId;

await using (initialClient)
{
await using (var fastSession = await Ctx.CreateSessionAsync(initialClient, new SessionConfig
{
Model = "auto",
OnPermissionRequest = PermissionHandler.ApproveAll,
Capi = new CapiSessionOptions
{
AutoTier = AutoTier.Fast,
EnableWebSocketResponses = false,
},
}))
await using (var tierlessSession = await Ctx.CreateSessionAsync(initialClient, new SessionConfig
{
Model = "auto",
OnPermissionRequest = PermissionHandler.ApproveAll,
Capi = new CapiSessionOptions { EnableWebSocketResponses = false },
}))
{
fastSessionId = fastSession.SessionId;
tierlessSessionId = tierlessSession.SessionId;

await fastSession.SendAndWaitAsync(new MessageOptions
{
Prompt = "Reply with exactly AUTO_TIER_COLD_RESUME_READY.",
});
await tierlessSession.SendAndWaitAsync(new MessageOptions
{
Prompt = "Reply with exactly AUTO_TIER_TIERLESS_READY.",
});

var fastCurrent = await fastSession.Rpc.Model.GetCurrentAsync();
Assert.Equal(AutoTier.Fast, fastCurrent.AutoTier);
var tierlessCurrent = await tierlessSession.Rpc.Model.GetCurrentAsync();
Assert.Null(tierlessCurrent.AutoTier);
}

await initialClient.StopAsync();
}

var restoredClient = Ctx.CreateClient();
await using (restoredClient)
{
await using (var restoredFast = await Ctx.ResumeSessionAsync(
restoredClient,
fastSessionId,
new ResumeSessionConfig { OnPermissionRequest = PermissionHandler.ApproveAll }))
await using (var restoredTierless = await Ctx.ResumeSessionAsync(
restoredClient,
tierlessSessionId,
new ResumeSessionConfig { OnPermissionRequest = PermissionHandler.ApproveAll }))
{
var restoredFastCurrent = await restoredFast.Rpc.Model.GetCurrentAsync();
Assert.Equal(AutoTier.Fast, restoredFastCurrent.AutoTier);
var restoredTierlessCurrent = await restoredTierless.Rpc.Model.GetCurrentAsync();
Assert.Null(restoredTierlessCurrent.AutoTier);
}

await restoredClient.StopAsync();
}

var overrideClient = Ctx.CreateClient();
await using (overrideClient)
{
await using var overridden = await Ctx.ResumeSessionAsync(
overrideClient,
fastSessionId,
new ResumeSessionConfig
{
Model = "auto",
OnPermissionRequest = PermissionHandler.ApproveAll,
Capi = new CapiSessionOptions
{
AutoTier = AutoTier.Balance,
EnableWebSocketResponses = false,
},
});

var overriddenCurrent = await overridden.Rpc.Model.GetCurrentAsync();
Assert.Equal(AutoTier.Balance, overriddenCurrent.AutoTier);
}
}

[Fact]
[Trait(E2ETestTraits.Backend, E2ETestTraits.CapiOnly)]
public async Task Should_Commit_Fast_Auto_Tier_After_Successful_Turn()
{
await using var client = Ctx.CreateClient();
await using var session = await Ctx.CreateSessionAsync(client, new SessionConfig
{
Model = "auto",
OnPermissionRequest = PermissionHandler.ApproveAll,
Capi = new CapiSessionOptions
{
AutoTier = AutoTier.Efficiency,
EnableWebSocketResponses = false,
},
});

var modelChangedTask = TestHelper.GetNextEventOfTypeAsync<SessionModelChangeEvent>(
session,
evt => evt.Data.AutoTier == AutoTier.Fast);

var staged = await session.SetAutoTierAsync(AutoTier.Fast);
Assert.Equal(ModelSwitchAutoTierStatus.Pending, staged.Status);
Assert.Equal(AutoTier.Efficiency, staged.EffectiveAutoTier);
Assert.Equal(AutoTier.Fast, staged.PendingAutoTier);

var beforeTurn = await session.Rpc.Model.GetCurrentAsync();
Assert.Equal(AutoTier.Efficiency, beforeTurn.AutoTier);
Assert.Equal(AutoTier.Fast, beforeTurn.PendingAutoTier);

await session.SendAndWaitAsync(new MessageOptions
{
Prompt = "Reply with exactly AUTO_TIER_FAST_COMMITTED.",
});

var modelChanged = await modelChangedTask;
Assert.Equal("auto", modelChanged.Data.PreviousModel);
Assert.Equal("auto", modelChanged.Data.NewModel);
Assert.Equal(AutoTier.Efficiency, modelChanged.Data.PreviousAutoTier);
Assert.Equal(AutoTier.Fast, modelChanged.Data.AutoTier);

var committed = await session.Rpc.Model.GetCurrentAsync();
Assert.Equal(AutoTier.Fast, committed.AutoTier);
Assert.Null(committed.PendingAutoTier);
Assert.Null(committed.ActivatingAutoTier);
}

[Fact]
[Trait(E2ETestTraits.Backend, E2ETestTraits.CapiOnly)]
public async Task Should_Preserve_Effective_Tier_When_Fast_Activation_Fails()
{
string sessionId;
var initialClient = Ctx.CreateClient();

await using (initialClient)
{
await using (var session = await Ctx.CreateSessionAsync(initialClient, new SessionConfig
{
Model = "auto",
OnPermissionRequest = PermissionHandler.ApproveAll,
Capi = new CapiSessionOptions
{
AutoTier = AutoTier.Efficiency,
EnableWebSocketResponses = false,
},
}))
{
sessionId = session.SessionId;
await session.SendAndWaitAsync(new MessageOptions
{
Prompt = "Reply with exactly AUTO_TIER_INITIAL_READY.",
});

var failureTask = TestHelper.GetNextEventOfTypeAsync<SessionAutoTierSwitchFailedEvent>(session);
var fastCommit = new TaskCompletionSource<SessionModelChangeEvent>(
TaskCreationOptions.RunContinuationsAsynchronously);
using var modelChangeSubscription = session.On<SessionModelChangeEvent>(evt =>
{
if (evt.Data.AutoTier == AutoTier.Fast)
{
fastCommit.TrySetResult(evt);
}
});

var staged = await session.SetAutoTierAsync(AutoTier.Fast);
Assert.Equal(ModelSwitchAutoTierStatus.Pending, staged.Status);
Assert.Equal(AutoTier.Efficiency, staged.EffectiveAutoTier);
Assert.Equal(AutoTier.Fast, staged.PendingAutoTier);

await session.SendAndWaitAsync(new MessageOptions
{
Prompt = "Reply with exactly AUTO_TIER_FAILURE_RECOVERED.",
});

var failure = await failureTask;
Assert.True(failure.Ephemeral);
Assert.Equal(AutoTier.Efficiency, failure.Data.EffectiveAutoTier);
Assert.Equal(AutoTier.Fast, failure.Data.RequestedAutoTier);
Assert.Equal(AutoTierSwitchFailureReason.RequestFailed, failure.Data.Reason);
var noFastCommit = await Task.WhenAny(fastCommit.Task, Task.Delay(TimeSpan.FromMilliseconds(100)));
Assert.NotSame(fastCommit.Task, noFastCommit);

var current = await session.Rpc.Model.GetCurrentAsync();
Assert.Equal(AutoTier.Efficiency, current.AutoTier);
Assert.Null(current.PendingAutoTier);
Assert.Null(current.ActivatingAutoTier);
}

await initialClient.StopAsync();
}

await using var resumedClient = Ctx.CreateClient();
await using var resumed = await Ctx.ResumeSessionAsync(
resumedClient,
sessionId,
new ResumeSessionConfig { OnPermissionRequest = PermissionHandler.ApproveAll });

var resumedCurrent = await resumed.Rpc.Model.GetCurrentAsync();
Assert.Equal(AutoTier.Efficiency, resumedCurrent.AutoTier);
Assert.Null(resumedCurrent.PendingAutoTier);
Assert.Null(resumedCurrent.ActivatingAutoTier);

var persisted = await resumed.Rpc.EventLog.ReadAsync(max: 100, waitMs: TimeSpan.Zero);
Assert.DoesNotContain(persisted.Events, evt => evt is SessionAutoTierSwitchFailedEvent);
}
}
7 changes: 5 additions & 2 deletions dotnet/test/Unit/ClientSessionLifetimeTests.cs
Original file line number Diff line number Diff line change
Expand Up @@ -471,9 +471,11 @@ public async Task CreateSessionAsync_Omits_CustomAgent_ReasoningEffort_When_Unse
{ AutoTier.Efficiency, "efficiency", null },
{ AutoTier.Balance, "balance", null },
{ AutoTier.Intelligence, "intelligence", null },
{ AutoTier.Fast, "fast", null },
{ AutoTier.Efficiency, "efficiency", false },
{ AutoTier.Balance, "balance", false },
{ AutoTier.Intelligence, "intelligence", false },
{ AutoTier.Fast, "fast", false },
};

[Theory]
Expand Down Expand Up @@ -517,6 +519,7 @@ public async Task SessionRequests_Serialize_CapiAutoTier(AutoTier tier, string e
[InlineData("efficiency")]
[InlineData("balance")]
[InlineData("intelligence")]
[InlineData("fast")]
public async Task SetModelAsync_Serializes_AutoTier(string expectedTier)
{
await using var server = await FakeCopilotServer.StartAsync();
Expand Down Expand Up @@ -593,10 +596,10 @@ public async Task SetAutoTierAsync_Serializes_Tier_And_Returns_Snapshot()
OnPermissionRequest = PermissionHandler.ApproveAll
});

var result = await session.SetAutoTierAsync(AutoTier.Intelligence);
var result = await session.SetAutoTierAsync(AutoTier.Fast);

var request = Assert.Single(server.Requests, request => request.Method == "session.model.switchAutoTier");
Assert.Equal("intelligence", request.Params.GetProperty("autoTier").GetString());
Assert.Equal("fast", request.Params.GetProperty("autoTier").GetString());
Assert.Equal(ModelSwitchAutoTierStatus.Pending, result.Status);
Assert.Equal(AutoTier.Balance, result.EffectiveAutoTier);
}
Expand Down
5 changes: 3 additions & 2 deletions dotnet/test/Unit/SessionEventSerializationTests.cs
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ public class SessionEventSerializationTests
{ AutoTier.Efficiency, "efficiency" },
{ AutoTier.Balance, "balance" },
{ AutoTier.Intelligence, "intelligence" },
{ AutoTier.Fast, "fast" },
{ null, null },
};

Expand Down Expand Up @@ -102,7 +103,7 @@ public void SessionEvent_Deserializes_AutoTierSwitchFailed(string wireReason)
"type": "session.auto_tier_switch_failed",
"data": {
"effectiveAutoTier": "balance",
"requestedAutoTier": "intelligence",
"requestedAutoTier": "fast",
"reason": "{{wireReason}}"
}
}
Expand All @@ -113,7 +114,7 @@ public void SessionEvent_Deserializes_AutoTierSwitchFailed(string wireReason)
var data = Assert.IsType<SessionAutoTierSwitchFailedEvent>(sessionEvent).Data;
Assert.Equal(new AutoTierSwitchFailureReason(wireReason), data.Reason);
Assert.Equal(AutoTier.Balance, data.EffectiveAutoTier);
Assert.Equal(AutoTier.Intelligence, data.RequestedAutoTier);
Assert.Equal(AutoTier.Fast, data.RequestedAutoTier);
}

[Fact]
Expand Down
6 changes: 5 additions & 1 deletion go/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -361,9 +361,13 @@ Unknown section IDs are handled gracefully: content from `replace`/`append`/`pre

## Auto routing tiers

The canonical values are `AutoTierEfficiency`, `AutoTierBalance`, `AutoTierIntelligence`, and `AutoTierFast`, which send `efficiency`, `balance`, `intelligence`, and `fast` on the wire. Fast is an integrator-only latency preset, not a fourth first-party GitHub Copilot preference. The SDK forwards the requested value without deciding eligibility or inspecting client identity. An externally supplied older runtime returns its native runtime or JSON-RPC error; the SDK does not downgrade or silently ignore the request.

Omitting the tier on create uses the runtime default rather than Balance. A cold resume restores the persisted tier unless the resume request supplies an explicit override.

Change the Auto routing preference without changing the selected model. The runtime does not apply the preference immediately: it records the request and commits it only when a later user turn using the `auto` model successfully obtains a usable model from the provider, so a `pending` status confirms acceptance rather than effect. Only the most recent request survives.

Watch for the outcome through the `session.model_change` event on success or the ephemeral `session.auto_tier_switch_failed` event on failure. Read the authoritative committed, pending, and activating preferences at any time through the session's `model.getCurrent` RPC method.
Watch for the outcome through the `session.model_change` event on success or the ephemeral `session.auto_tier_switch_failed` event on failure. A failed activation leaves the incumbent effective tier unchanged. Read the authoritative committed, pending, and activating preferences at any time through the session's `model.getCurrent` RPC method.

```go
tier := copilot.AutoTierIntelligence
Expand Down
Loading
Loading