From 4fb572d0e32d6a7263c16b613761417afbe3109e Mon Sep 17 00:00:00 2001 From: Andy Feller Date: Tue, 15 Sep 2026 08:20:21 -0400 Subject: [PATCH 1/4] Complete Fast Auto-tier SDK support Add Fast Auto-tier exposure, contract tests, and lifecycle E2E coverage across the SDKs while preserving runtime-owned eligibility and omitted-tier semantics. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- CHANGELOG.md | 6 + docs/features/session-persistence.md | 4 +- dotnet/README.md | 6 +- dotnet/test/E2E/AutoTierE2ETests.cs | 227 ++++++++++- .../test/Unit/ClientSessionLifetimeTests.cs | 7 +- .../Unit/SessionEventSerializationTests.cs | 5 +- go/README.md | 6 +- go/client_test.go | 2 + go/internal/e2e/auto_tier_e2e_test.go | 352 +++++++++++++++- go/session_event_serialization_test.go | 8 +- go/session_test.go | 12 +- java/README.md | 12 +- .../java/com/github/copilot/rpc/AutoTier.java | 8 +- .../java/com/github/copilot/AutoTierIT.java | 196 ++++++++- .../copilot/CapiSessionOptionsTest.java | 2 +- .../copilot/SessionAutoTierEventTest.java | 9 +- .../copilot/SessionAutoTierSwitchTest.java | 14 +- nodejs/README.md | 4 +- nodejs/test/client.test.ts | 13 +- nodejs/test/e2e/auto_tier.e2e.test.ts | 163 +++++++- nodejs/test/session-event-types.test.ts | 6 +- python/README.md | 4 +- python/copilot/client.py | 6 +- python/copilot/session.py | 2 +- python/e2e/test_auto_tier_e2e.py | 189 ++++++++- python/test_client.py | 18 +- python/test_event_forward_compatibility.py | 6 +- rust/README.md | 12 +- rust/src/types.rs | 7 +- rust/tests/api_types_test.rs | 9 +- rust/tests/e2e/auto_tier.rs | 378 +++++++++++++++++- test/harness/replayingCapiProxy.test.ts | 50 +++ test/harness/replayingCapiProxy.ts | 58 +++ ..._fast_auto_tier_after_successful_turn.yaml | 21 + ...ctive_tier_when_fast_activation_fails.yaml | 30 ++ ...verride_fast_auto_tier_on_cold_resume.yaml | 38 ++ 36 files changed, 1785 insertions(+), 105 deletions(-) create mode 100644 test/snapshots/auto_tier/should_commit_fast_auto_tier_after_successful_turn.yaml create mode 100644 test/snapshots/auto_tier/should_preserve_effective_tier_when_fast_activation_fails.yaml create mode 100644 test/snapshots/auto_tier/should_restore_and_override_fast_auto_tier_on_cold_resume.yaml diff --git a/CHANGELOG.md b/CHANGELOG.md index d71e9b8248..f4cf3a90a4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,12 @@ See [GitHub Releases](https://github.com/github/copilot-sdk/releases) for the fu ## [Unreleased] +### Feature: complete Fast Auto tier support + +All six SDKs now expose the canonical `fast` Auto routing tier across session creation, cold resume, resident switching, authoritative state reads, and terminal lifecycle events. Fast is an integrator-only latency preset and requires Copilot CLI `1.0.84-0` or later. + +The SDKs preserve the runtime's behavior: omitted tiers use the runtime default or persisted value, explicit resume tiers override persisted state, resident switches remain pending until a successful Auto turn commits them, and failed activation leaves the incumbent effective tier unchanged. SDKs do not decide Fast eligibility, downgrade unsupported requests, or hide errors from externally supplied older runtimes. + ## [v1.0.13](https://github.com/github/copilot-sdk/releases/tag/v1.0.13) (2026-09-04) ### Feature: cancellation for host-owned external tools diff --git a/docs/features/session-persistence.md b/docs/features/session-persistence.md index f2e1c7d5d9..c99f71c53f 100644 --- a/docs/features/session-persistence.md +++ b/docs/features/session-persistence.md @@ -257,7 +257,9 @@ When resuming a session, you can optionally reconfigure many settings. This is u ### Auto tier persistence -With `model: "auto"`, the optional `capi.autoTier` setting selects an Auto routing preference: `efficiency`, `balance`, or `intelligence`. In Python, use `capi={"auto_tier": "balance"}`. This requires Copilot CLI `1.0.82-1` or later with V2 Auto routing; V1 Auto requests are unchanged. +With `model: "auto"`, the optional `capi.autoTier` setting selects an Auto routing preference: `efficiency`, `balance`, `intelligence`, or `fast`. In Python, use `capi={"auto_tier": "balance"}`. This requires Copilot CLI `1.0.82-1` or later with V2 Auto routing; V1 Auto requests are unchanged. `fast` additionally requires Copilot CLI `1.0.84-0` or later. + +`fast` is an integrator-only latency preset, not a first-party GitHub Copilot product preference. The SDK does not decide Fast eligibility, inspect client identity, choose it as a default, or fall back to another tier when a runtime does not support it — an older runtime returns its native error unchanged. The runtime persists the selected tier, so applications do not need to resend it on every resume: diff --git a/dotnet/README.md b/dotnet/README.md index c518a40326..a952b2aae2 100644 --- a/dotnet/README.md +++ b/dotnet/README.md @@ -317,9 +317,13 @@ await session2.DisposeAsync(); ## Auto routing tiers +The canonical values are `AutoTier.Efficiency`, `AutoTier.Balance`, `AutoTier.Intelligence`, and `AutoTier.Fast`, which send `efficiency`, `balance`, `intelligence`, and `fast` on the wire. Fast is an integrator-only latency preset, not a fourth first-party GitHub Copilot preference, and requires Copilot CLI `1.0.84-0` or later. The SDK forwards the requested value without deciding eligibility or inspecting client identity. An externally supplied older runtime returns its native runtime or JSON-RPC error; the SDK does not downgrade or silently ignore the request. + +Omitting the tier on create uses the runtime default rather than Balance. A cold resume restores the persisted tier unless the resume request supplies an explicit override. + Change the Auto routing preference without changing the selected model. The runtime does not apply the preference immediately: it records the request and commits it only when a later user turn using the `auto` model successfully obtains a usable model from the provider, so a `pending` status confirms acceptance rather than effect. Only the most recent request survives. -Watch for the outcome through the `session.model_change` event on success or the ephemeral `session.auto_tier_switch_failed` event on failure. Read the authoritative committed, pending, and activating preferences at any time through the session's `model.getCurrent` RPC method. +Watch for the outcome through the `session.model_change` event on success or the ephemeral `session.auto_tier_switch_failed` event on failure. A failed activation leaves the incumbent effective tier unchanged. Read the authoritative committed, pending, and activating preferences at any time through the session's `model.getCurrent` RPC method. ```csharp var result = await session.SetAutoTierAsync(AutoTier.Intelligence); diff --git a/dotnet/test/E2E/AutoTierE2ETests.cs b/dotnet/test/E2E/AutoTierE2ETests.cs index 3758eee0f5..c04a765038 100644 --- a/dotnet/test/E2E/AutoTierE2ETests.cs +++ b/dotnet/test/E2E/AutoTierE2ETests.cs @@ -45,10 +45,16 @@ public async Task Should_Stage_And_Reset_Auto_Tier_Preference() await AssertPendingAutoTierAsync(session, AutoTier.Efficiency); // A second request replaces the first and reports the one it displaced. - var superseded = await session.SetAutoTierAsync(AutoTier.Intelligence); + var superseded = await session.SetAutoTierAsync(AutoTier.Fast); Assert.Equal(ModelSwitchAutoTierStatus.Pending, superseded.Status); - Assert.Equal(AutoTier.Intelligence, superseded.PendingAutoTier); + Assert.Equal(AutoTier.Fast, superseded.PendingAutoTier); Assert.Equal(AutoTier.Efficiency, superseded.SupersededAutoTier); + await AssertPendingAutoTierAsync(session, AutoTier.Fast); + + var replacedFast = await session.SetAutoTierAsync(AutoTier.Intelligence); + Assert.Equal(ModelSwitchAutoTierStatus.Pending, replacedFast.Status); + Assert.Equal(AutoTier.Intelligence, replacedFast.PendingAutoTier); + Assert.Equal(AutoTier.Fast, replacedFast.SupersededAutoTier); await AssertPendingAutoTierAsync(session, AutoTier.Intelligence); // A null tier returns the session to provider-default routing. The status is @@ -77,12 +83,225 @@ public async Task Should_Preserve_Auto_Tier_When_Set_Model_Omits_It() await AssertPendingAutoTierAsync(session, AutoTier.Balance); // Supplying a tier replaces it. - await session.SetModelAsync("auto", new SetModelOptions { AutoTier = AutoTier.Intelligence }); - await AssertPendingAutoTierAsync(session, AutoTier.Intelligence); + await session.SetModelAsync("auto", new SetModelOptions { AutoTier = AutoTier.Fast }); + await AssertPendingAutoTierAsync(session, AutoTier.Fast); // ResetAutoTier clears it. Omission, a value, and a reset are three distinct // outcomes, which is why a single nullable property cannot express the request. await session.SetModelAsync("auto", new SetModelOptions { ResetAutoTier = true }); await AssertPendingAutoTierAsync(session, null); } + + [Fact] + public async Task Should_Restore_And_Override_Fast_Auto_Tier_On_Cold_Resume() + { + var initialClient = Ctx.CreateClient(); + string fastSessionId; + string tierlessSessionId; + + await using (initialClient) + { + await using (var fastSession = await Ctx.CreateSessionAsync(initialClient, new SessionConfig + { + Model = "auto", + OnPermissionRequest = PermissionHandler.ApproveAll, + Capi = new CapiSessionOptions + { + AutoTier = AutoTier.Fast, + EnableWebSocketResponses = false, + }, + })) + await using (var tierlessSession = await Ctx.CreateSessionAsync(initialClient, new SessionConfig + { + Model = "auto", + OnPermissionRequest = PermissionHandler.ApproveAll, + Capi = new CapiSessionOptions { EnableWebSocketResponses = false }, + })) + { + fastSessionId = fastSession.SessionId; + tierlessSessionId = tierlessSession.SessionId; + + await fastSession.SendAndWaitAsync(new MessageOptions + { + Prompt = "Reply with exactly AUTO_TIER_COLD_RESUME_READY.", + }); + await tierlessSession.SendAndWaitAsync(new MessageOptions + { + Prompt = "Reply with exactly AUTO_TIER_TIERLESS_READY.", + }); + + var fastCurrent = await fastSession.Rpc.Model.GetCurrentAsync(); + Assert.Equal(AutoTier.Fast, fastCurrent.AutoTier); + var tierlessCurrent = await tierlessSession.Rpc.Model.GetCurrentAsync(); + Assert.Null(tierlessCurrent.AutoTier); + } + + await initialClient.StopAsync(); + } + + var restoredClient = Ctx.CreateClient(); + await using (restoredClient) + { + await using (var restoredFast = await Ctx.ResumeSessionAsync( + restoredClient, + fastSessionId, + new ResumeSessionConfig { OnPermissionRequest = PermissionHandler.ApproveAll })) + await using (var restoredTierless = await Ctx.ResumeSessionAsync( + restoredClient, + tierlessSessionId, + new ResumeSessionConfig { OnPermissionRequest = PermissionHandler.ApproveAll })) + { + var restoredFastCurrent = await restoredFast.Rpc.Model.GetCurrentAsync(); + Assert.Equal(AutoTier.Fast, restoredFastCurrent.AutoTier); + var restoredTierlessCurrent = await restoredTierless.Rpc.Model.GetCurrentAsync(); + Assert.Null(restoredTierlessCurrent.AutoTier); + } + + await restoredClient.StopAsync(); + } + + var overrideClient = Ctx.CreateClient(); + await using (overrideClient) + { + await using var overridden = await Ctx.ResumeSessionAsync( + overrideClient, + fastSessionId, + new ResumeSessionConfig + { + Model = "auto", + OnPermissionRequest = PermissionHandler.ApproveAll, + Capi = new CapiSessionOptions + { + AutoTier = AutoTier.Balance, + EnableWebSocketResponses = false, + }, + }); + + var overriddenCurrent = await overridden.Rpc.Model.GetCurrentAsync(); + Assert.Equal(AutoTier.Balance, overriddenCurrent.AutoTier); + } + } + + [Fact] + public async Task Should_Commit_Fast_Auto_Tier_After_Successful_Turn() + { + await using var client = Ctx.CreateClient(); + await using var session = await Ctx.CreateSessionAsync(client, new SessionConfig + { + Model = "auto", + OnPermissionRequest = PermissionHandler.ApproveAll, + Capi = new CapiSessionOptions + { + AutoTier = AutoTier.Efficiency, + EnableWebSocketResponses = false, + }, + }); + + var modelChangedTask = TestHelper.GetNextEventOfTypeAsync( + session, + evt => evt.Data.AutoTier == AutoTier.Fast); + + var staged = await session.SetAutoTierAsync(AutoTier.Fast); + Assert.Equal(ModelSwitchAutoTierStatus.Pending, staged.Status); + Assert.Equal(AutoTier.Efficiency, staged.EffectiveAutoTier); + Assert.Equal(AutoTier.Fast, staged.PendingAutoTier); + + var beforeTurn = await session.Rpc.Model.GetCurrentAsync(); + Assert.Equal(AutoTier.Efficiency, beforeTurn.AutoTier); + Assert.Equal(AutoTier.Fast, beforeTurn.PendingAutoTier); + + await session.SendAndWaitAsync(new MessageOptions + { + Prompt = "Reply with exactly AUTO_TIER_FAST_COMMITTED.", + }); + + var modelChanged = await modelChangedTask; + Assert.Equal("auto", modelChanged.Data.PreviousModel); + Assert.Equal("auto", modelChanged.Data.NewModel); + Assert.Equal(AutoTier.Efficiency, modelChanged.Data.PreviousAutoTier); + Assert.Equal(AutoTier.Fast, modelChanged.Data.AutoTier); + + var committed = await session.Rpc.Model.GetCurrentAsync(); + Assert.Equal(AutoTier.Fast, committed.AutoTier); + Assert.Null(committed.PendingAutoTier); + Assert.Null(committed.ActivatingAutoTier); + } + + [Fact] + public async Task Should_Preserve_Effective_Tier_When_Fast_Activation_Fails() + { + string sessionId; + var initialClient = Ctx.CreateClient(); + + await using (initialClient) + { + await using (var session = await Ctx.CreateSessionAsync(initialClient, new SessionConfig + { + Model = "auto", + OnPermissionRequest = PermissionHandler.ApproveAll, + Capi = new CapiSessionOptions + { + AutoTier = AutoTier.Efficiency, + EnableWebSocketResponses = false, + }, + })) + { + sessionId = session.SessionId; + await session.SendAndWaitAsync(new MessageOptions + { + Prompt = "Reply with exactly AUTO_TIER_INITIAL_READY.", + }); + + var failureTask = TestHelper.GetNextEventOfTypeAsync(session); + var fastCommit = new TaskCompletionSource( + TaskCreationOptions.RunContinuationsAsynchronously); + using var modelChangeSubscription = session.On(evt => + { + if (evt.Data.AutoTier == AutoTier.Fast) + { + fastCommit.TrySetResult(evt); + } + }); + + var staged = await session.SetAutoTierAsync(AutoTier.Fast); + Assert.Equal(ModelSwitchAutoTierStatus.Pending, staged.Status); + Assert.Equal(AutoTier.Efficiency, staged.EffectiveAutoTier); + Assert.Equal(AutoTier.Fast, staged.PendingAutoTier); + + await session.SendAndWaitAsync(new MessageOptions + { + Prompt = "Reply with exactly AUTO_TIER_FAILURE_RECOVERED.", + }); + + var failure = await failureTask; + Assert.True(failure.Ephemeral); + Assert.Equal(AutoTier.Efficiency, failure.Data.EffectiveAutoTier); + Assert.Equal(AutoTier.Fast, failure.Data.RequestedAutoTier); + Assert.Equal(AutoTierSwitchFailureReason.RequestFailed, failure.Data.Reason); + var noFastCommit = await Task.WhenAny(fastCommit.Task, Task.Delay(TimeSpan.FromMilliseconds(100))); + Assert.NotSame(fastCommit.Task, noFastCommit); + + var current = await session.Rpc.Model.GetCurrentAsync(); + Assert.Equal(AutoTier.Efficiency, current.AutoTier); + Assert.Null(current.PendingAutoTier); + Assert.Null(current.ActivatingAutoTier); + } + + await initialClient.StopAsync(); + } + + await using var resumedClient = Ctx.CreateClient(); + await using var resumed = await Ctx.ResumeSessionAsync( + resumedClient, + sessionId, + new ResumeSessionConfig { OnPermissionRequest = PermissionHandler.ApproveAll }); + + var resumedCurrent = await resumed.Rpc.Model.GetCurrentAsync(); + Assert.Equal(AutoTier.Efficiency, resumedCurrent.AutoTier); + Assert.Null(resumedCurrent.PendingAutoTier); + Assert.Null(resumedCurrent.ActivatingAutoTier); + + var persisted = await resumed.Rpc.EventLog.ReadAsync(max: 100, waitMs: TimeSpan.Zero); + Assert.DoesNotContain(persisted.Events, evt => evt is SessionAutoTierSwitchFailedEvent); + } } diff --git a/dotnet/test/Unit/ClientSessionLifetimeTests.cs b/dotnet/test/Unit/ClientSessionLifetimeTests.cs index 88de6abd0d..3e4973a192 100644 --- a/dotnet/test/Unit/ClientSessionLifetimeTests.cs +++ b/dotnet/test/Unit/ClientSessionLifetimeTests.cs @@ -471,9 +471,11 @@ public async Task CreateSessionAsync_Omits_CustomAgent_ReasoningEffort_When_Unse { AutoTier.Efficiency, "efficiency", null }, { AutoTier.Balance, "balance", null }, { AutoTier.Intelligence, "intelligence", null }, + { AutoTier.Fast, "fast", null }, { AutoTier.Efficiency, "efficiency", false }, { AutoTier.Balance, "balance", false }, { AutoTier.Intelligence, "intelligence", false }, + { AutoTier.Fast, "fast", false }, }; [Theory] @@ -517,6 +519,7 @@ public async Task SessionRequests_Serialize_CapiAutoTier(AutoTier tier, string e [InlineData("efficiency")] [InlineData("balance")] [InlineData("intelligence")] + [InlineData("fast")] public async Task SetModelAsync_Serializes_AutoTier(string expectedTier) { await using var server = await FakeCopilotServer.StartAsync(); @@ -593,10 +596,10 @@ public async Task SetAutoTierAsync_Serializes_Tier_And_Returns_Snapshot() OnPermissionRequest = PermissionHandler.ApproveAll }); - var result = await session.SetAutoTierAsync(AutoTier.Intelligence); + var result = await session.SetAutoTierAsync(AutoTier.Fast); var request = Assert.Single(server.Requests, request => request.Method == "session.model.switchAutoTier"); - Assert.Equal("intelligence", request.Params.GetProperty("autoTier").GetString()); + Assert.Equal("fast", request.Params.GetProperty("autoTier").GetString()); Assert.Equal(ModelSwitchAutoTierStatus.Pending, result.Status); Assert.Equal(AutoTier.Balance, result.EffectiveAutoTier); } diff --git a/dotnet/test/Unit/SessionEventSerializationTests.cs b/dotnet/test/Unit/SessionEventSerializationTests.cs index 405ffb379f..6f1b5283e0 100644 --- a/dotnet/test/Unit/SessionEventSerializationTests.cs +++ b/dotnet/test/Unit/SessionEventSerializationTests.cs @@ -14,6 +14,7 @@ public class SessionEventSerializationTests { AutoTier.Efficiency, "efficiency" }, { AutoTier.Balance, "balance" }, { AutoTier.Intelligence, "intelligence" }, + { AutoTier.Fast, "fast" }, { null, null }, }; @@ -102,7 +103,7 @@ public void SessionEvent_Deserializes_AutoTierSwitchFailed(string wireReason) "type": "session.auto_tier_switch_failed", "data": { "effectiveAutoTier": "balance", - "requestedAutoTier": "intelligence", + "requestedAutoTier": "fast", "reason": "{{wireReason}}" } } @@ -113,7 +114,7 @@ public void SessionEvent_Deserializes_AutoTierSwitchFailed(string wireReason) var data = Assert.IsType(sessionEvent).Data; Assert.Equal(new AutoTierSwitchFailureReason(wireReason), data.Reason); Assert.Equal(AutoTier.Balance, data.EffectiveAutoTier); - Assert.Equal(AutoTier.Intelligence, data.RequestedAutoTier); + Assert.Equal(AutoTier.Fast, data.RequestedAutoTier); } [Fact] diff --git a/go/README.md b/go/README.md index 2eb2c720fc..441689a901 100644 --- a/go/README.md +++ b/go/README.md @@ -361,9 +361,13 @@ Unknown section IDs are handled gracefully: content from `replace`/`append`/`pre ## Auto routing tiers +The canonical values are `AutoTierEfficiency`, `AutoTierBalance`, `AutoTierIntelligence`, and `AutoTierFast`, which send `efficiency`, `balance`, `intelligence`, and `fast` on the wire. Fast is an integrator-only latency preset, not a fourth first-party GitHub Copilot preference, and requires Copilot CLI `1.0.84-0` or later. The SDK forwards the requested value without deciding eligibility or inspecting client identity. An externally supplied older runtime returns its native runtime or JSON-RPC error; the SDK does not downgrade or silently ignore the request. + +Omitting the tier on create uses the runtime default rather than Balance. A cold resume restores the persisted tier unless the resume request supplies an explicit override. + Change the Auto routing preference without changing the selected model. The runtime does not apply the preference immediately: it records the request and commits it only when a later user turn using the `auto` model successfully obtains a usable model from the provider, so a `pending` status confirms acceptance rather than effect. Only the most recent request survives. -Watch for the outcome through the `session.model_change` event on success or the ephemeral `session.auto_tier_switch_failed` event on failure. Read the authoritative committed, pending, and activating preferences at any time through the session's `model.getCurrent` RPC method. +Watch for the outcome through the `session.model_change` event on success or the ephemeral `session.auto_tier_switch_failed` event on failure. A failed activation leaves the incumbent effective tier unchanged. Read the authoritative committed, pending, and activating preferences at any time through the session's `model.getCurrent` RPC method. ```go tier := copilot.AutoTierIntelligence diff --git a/go/client_test.go b/go/client_test.go index 52587c8462..1613c3b394 100644 --- a/go/client_test.go +++ b/go/client_test.go @@ -635,9 +635,11 @@ func TestClient_ForwardsCapiOptionsToSessionRequests(t *testing.T) { {"efficiency", &CapiSessionOptions{AutoTier: AutoTierEfficiency}, map[string]any{"autoTier": "efficiency"}}, {"balance", &CapiSessionOptions{AutoTier: AutoTierBalance}, map[string]any{"autoTier": "balance"}}, {"intelligence", &CapiSessionOptions{AutoTier: AutoTierIntelligence}, map[string]any{"autoTier": "intelligence"}}, + {"fast", &CapiSessionOptions{AutoTier: AutoTierFast}, map[string]any{"autoTier": "fast"}}, {"efficiency with websocket", &CapiSessionOptions{AutoTier: AutoTierEfficiency, EnableWebSocketResponses: Bool(false)}, map[string]any{"autoTier": "efficiency", "enableWebSocketResponses": false}}, {"balance with websocket", &CapiSessionOptions{AutoTier: AutoTierBalance, EnableWebSocketResponses: Bool(false)}, map[string]any{"autoTier": "balance", "enableWebSocketResponses": false}}, {"intelligence with websocket", &CapiSessionOptions{AutoTier: AutoTierIntelligence, EnableWebSocketResponses: Bool(false)}, map[string]any{"autoTier": "intelligence", "enableWebSocketResponses": false}}, + {"fast with websocket", &CapiSessionOptions{AutoTier: AutoTierFast, EnableWebSocketResponses: Bool(false)}, map[string]any{"autoTier": "fast", "enableWebSocketResponses": false}}, } for _, tt := range tests { t.Run(tt.name, func(t *testing.T) { diff --git a/go/internal/e2e/auto_tier_e2e_test.go b/go/internal/e2e/auto_tier_e2e_test.go index e974f95927..c8689ea904 100644 --- a/go/internal/e2e/auto_tier_e2e_test.go +++ b/go/internal/e2e/auto_tier_e2e_test.go @@ -1,7 +1,9 @@ package e2e import ( + "sync/atomic" "testing" + "time" copilot "github.com/github/copilot-sdk/go" "github.com/github/copilot-sdk/go/internal/e2e/testharness" @@ -41,16 +43,21 @@ func TestAutoTierE2E(t *testing.T) { } } - newAutoSession := func(t *testing.T) *copilot.Session { + newAutoClient := func(t *testing.T) (*testharness.TestContext, *copilot.Client) { t.Helper() ctx := testharness.NewTestContext(t) + ctx.ConfigureForTest(t) client := ctx.NewClient() t.Cleanup(func() { client.ForceStop() }) if err := client.Start(t.Context()); err != nil { t.Fatalf("Failed to start client: %v", err) } - ctx.ConfigureForTest(t) + return ctx, client + } + newAutoSession := func(t *testing.T) *copilot.Session { + t.Helper() + _, client := newAutoClient(t) session, err := client.CreateSession(t.Context(), &copilot.SessionConfig{ Model: "auto", OnPermissionRequest: copilot.PermissionHandler.ApproveAll, @@ -78,9 +85,9 @@ func TestAutoTierE2E(t *testing.T) { assertPending(t, session, rpc.AutoTierEfficiency) // A second request replaces the first and reports the one it displaced. - superseded, err := session.SetAutoTier(t.Context(), autoTier(copilot.AutoTierIntelligence)) + superseded, err := session.SetAutoTier(t.Context(), autoTier(copilot.AutoTierFast)) if err != nil { - t.Fatalf("SetAutoTier(intelligence) failed: %v", err) + t.Fatalf("SetAutoTier(fast) failed: %v", err) } if superseded.Status != rpc.ModelSwitchAutoTierStatusPending { t.Fatalf("Expected status pending, got %q", superseded.Status) @@ -88,6 +95,18 @@ func TestAutoTierE2E(t *testing.T) { if superseded.SupersededAutoTier == nil || *superseded.SupersededAutoTier != rpc.AutoTierEfficiency { t.Fatalf("Expected superseded efficiency, got %+v", superseded) } + assertPending(t, session, rpc.AutoTierFast) + + replacedFast, err := session.SetAutoTier(t.Context(), autoTier(copilot.AutoTierIntelligence)) + if err != nil { + t.Fatalf("SetAutoTier(intelligence) failed: %v", err) + } + if replacedFast.Status != rpc.ModelSwitchAutoTierStatusPending { + t.Fatalf("Expected status pending, got %q", replacedFast.Status) + } + if replacedFast.SupersededAutoTier == nil || *replacedFast.SupersededAutoTier != rpc.AutoTierFast { + t.Fatalf("Expected superseded fast, got %+v", replacedFast) + } assertPending(t, session, rpc.AutoTierIntelligence) // A nil tier returns the session to provider-default routing. The status is @@ -122,11 +141,11 @@ func TestAutoTierE2E(t *testing.T) { // Supplying a tier replaces it. if err := session.SetModel(t.Context(), "auto", &copilot.SetModelOptions{ - AutoTier: autoTier(copilot.AutoTierIntelligence), + AutoTier: autoTier(copilot.AutoTierFast), }); err != nil { t.Fatalf("SetModel with AutoTier failed: %v", err) } - assertPending(t, session, rpc.AutoTierIntelligence) + assertPending(t, session, rpc.AutoTierFast) // ResetAutoTier clears it. Omission, a value, and a reset are three distinct // outcomes, which is why a single nillable field cannot express the request. @@ -137,4 +156,325 @@ func TestAutoTierE2E(t *testing.T) { } assertNoPending(t, session) }) + + t.Run("should restore and override fast auto tier on cold resume", func(t *testing.T) { + ctx, client := newAutoClient(t) + fastSession, err := client.CreateSession(t.Context(), &copilot.SessionConfig{ + Model: "auto", + OnPermissionRequest: copilot.PermissionHandler.ApproveAll, + Capi: &copilot.CapiSessionOptions{ + AutoTier: copilot.AutoTierFast, + EnableWebSocketResponses: copilot.Bool(false), + }, + }) + if err != nil { + t.Fatalf("CreateSession(fast) failed: %v", err) + } + tierlessSession, err := client.CreateSession(t.Context(), &copilot.SessionConfig{ + Model: "auto", + OnPermissionRequest: copilot.PermissionHandler.ApproveAll, + Capi: &copilot.CapiSessionOptions{ + EnableWebSocketResponses: copilot.Bool(false), + }, + }) + if err != nil { + t.Fatalf("CreateSession(tierless) failed: %v", err) + } + fastSessionID := fastSession.SessionID + tierlessSessionID := tierlessSession.SessionID + + if _, err := fastSession.SendAndWait(t.Context(), copilot.MessageOptions{ + Prompt: "Reply with exactly AUTO_TIER_COLD_RESUME_READY.", + }); err != nil { + t.Fatalf("Fast session turn failed: %v", err) + } + if _, err := tierlessSession.SendAndWait(t.Context(), copilot.MessageOptions{ + Prompt: "Reply with exactly AUTO_TIER_TIERLESS_READY.", + }); err != nil { + t.Fatalf("Tierless session turn failed: %v", err) + } + fastCurrent, err := fastSession.RPC.Model.GetCurrent(t.Context()) + if err != nil { + t.Fatalf("GetCurrent(fast) failed: %v", err) + } + if fastCurrent.AutoTier == nil || *fastCurrent.AutoTier != rpc.AutoTierFast { + t.Fatalf("Expected effective fast tier, got %+v", fastCurrent) + } + tierlessCurrent, err := tierlessSession.RPC.Model.GetCurrent(t.Context()) + if err != nil { + t.Fatalf("GetCurrent(tierless) failed: %v", err) + } + if tierlessCurrent.AutoTier != nil { + t.Fatalf("Expected no effective tier, got %+v", tierlessCurrent) + } + + fastSession.Disconnect() + tierlessSession.Disconnect() + if err := client.Stop(); err != nil { + t.Fatalf("Stop initial client failed: %v", err) + } + + restoredClient := ctx.NewClient() + t.Cleanup(func() { restoredClient.ForceStop() }) + if err := restoredClient.Start(t.Context()); err != nil { + t.Fatalf("Start restored client failed: %v", err) + } + restoredFast, err := restoredClient.ResumeSession(t.Context(), fastSessionID, &copilot.ResumeSessionConfig{ + OnPermissionRequest: copilot.PermissionHandler.ApproveAll, + }) + if err != nil { + t.Fatalf("ResumeSession(fast) failed: %v", err) + } + restoredTierless, err := restoredClient.ResumeSession(t.Context(), tierlessSessionID, &copilot.ResumeSessionConfig{ + OnPermissionRequest: copilot.PermissionHandler.ApproveAll, + }) + if err != nil { + t.Fatalf("ResumeSession(tierless) failed: %v", err) + } + restoredFastCurrent, err := restoredFast.RPC.Model.GetCurrent(t.Context()) + if err != nil { + t.Fatalf("GetCurrent(restored fast) failed: %v", err) + } + if restoredFastCurrent.AutoTier == nil || *restoredFastCurrent.AutoTier != rpc.AutoTierFast { + t.Fatalf("Expected restored fast tier, got %+v", restoredFastCurrent) + } + restoredTierlessCurrent, err := restoredTierless.RPC.Model.GetCurrent(t.Context()) + if err != nil { + t.Fatalf("GetCurrent(restored tierless) failed: %v", err) + } + if restoredTierlessCurrent.AutoTier != nil { + t.Fatalf("Expected restored tierless state, got %+v", restoredTierlessCurrent) + } + restoredFast.Disconnect() + restoredTierless.Disconnect() + if err := restoredClient.Stop(); err != nil { + t.Fatalf("Stop restored client failed: %v", err) + } + + overrideClient := ctx.NewClient() + t.Cleanup(func() { overrideClient.ForceStop() }) + if err := overrideClient.Start(t.Context()); err != nil { + t.Fatalf("Start override client failed: %v", err) + } + overridden, err := overrideClient.ResumeSession(t.Context(), fastSessionID, &copilot.ResumeSessionConfig{ + Model: "auto", + OnPermissionRequest: copilot.PermissionHandler.ApproveAll, + Capi: &copilot.CapiSessionOptions{ + AutoTier: copilot.AutoTierBalance, + EnableWebSocketResponses: copilot.Bool(false), + }, + }) + if err != nil { + t.Fatalf("ResumeSession(override) failed: %v", err) + } + overriddenCurrent, err := overridden.RPC.Model.GetCurrent(t.Context()) + if err != nil { + t.Fatalf("GetCurrent(overridden) failed: %v", err) + } + if overriddenCurrent.AutoTier == nil || *overriddenCurrent.AutoTier != rpc.AutoTierBalance { + t.Fatalf("Expected balance override, got %+v", overriddenCurrent) + } + overridden.Disconnect() + if err := overrideClient.Stop(); err != nil { + t.Fatalf("Stop override client failed: %v", err) + } + }) + + t.Run("should commit fast auto tier after successful turn", func(t *testing.T) { + _, client := newAutoClient(t) + session, err := client.CreateSession(t.Context(), &copilot.SessionConfig{ + Model: "auto", + OnPermissionRequest: copilot.PermissionHandler.ApproveAll, + Capi: &copilot.CapiSessionOptions{ + AutoTier: copilot.AutoTierEfficiency, + EnableWebSocketResponses: copilot.Bool(false), + }, + }) + if err != nil { + t.Fatalf("CreateSession failed: %v", err) + } + + modelChanged := make(chan *copilot.SessionModelChangeData, 1) + unsubscribe := session.On(func(event copilot.SessionEvent) { + if data, ok := event.Data.(*copilot.SessionModelChangeData); ok && data.AutoTier != nil && *data.AutoTier == copilot.AutoTierFast { + select { + case modelChanged <- data: + default: + } + } + }) + defer unsubscribe() + + staged, err := session.SetAutoTier(t.Context(), autoTier(copilot.AutoTierFast)) + if err != nil { + t.Fatalf("SetAutoTier(fast) failed: %v", err) + } + if staged.Status != rpc.ModelSwitchAutoTierStatusPending || + staged.EffectiveAutoTier == nil || *staged.EffectiveAutoTier != rpc.AutoTierEfficiency || + staged.PendingAutoTier == nil || *staged.PendingAutoTier != rpc.AutoTierFast { + t.Fatalf("Unexpected staged result: %+v", staged) + } + + beforeTurn, err := session.RPC.Model.GetCurrent(t.Context()) + if err != nil { + t.Fatalf("GetCurrent before turn failed: %v", err) + } + if beforeTurn.AutoTier == nil || *beforeTurn.AutoTier != rpc.AutoTierEfficiency || + beforeTurn.PendingAutoTier == nil || *beforeTurn.PendingAutoTier != rpc.AutoTierFast { + t.Fatalf("Unexpected state before turn: %+v", beforeTurn) + } + + if _, err := session.SendAndWait(t.Context(), copilot.MessageOptions{ + Prompt: "Reply with exactly AUTO_TIER_FAST_COMMITTED.", + }); err != nil { + t.Fatalf("SendAndWait failed: %v", err) + } + + select { + case data := <-modelChanged: + if data.PreviousModel == nil || *data.PreviousModel != "auto" || + data.NewModel != "auto" || + data.PreviousAutoTier == nil || *data.PreviousAutoTier != copilot.AutoTierEfficiency { + t.Fatalf("Unexpected model change: %+v", data) + } + case <-time.After(30 * time.Second): + t.Fatal("Timed out waiting for Fast model change") + } + + committed, err := session.RPC.Model.GetCurrent(t.Context()) + if err != nil { + t.Fatalf("GetCurrent after turn failed: %v", err) + } + if committed.AutoTier == nil || *committed.AutoTier != rpc.AutoTierFast || + committed.PendingAutoTier != nil || committed.ActivatingAutoTier != nil { + t.Fatalf("Unexpected committed state: %+v", committed) + } + }) + + t.Run("should preserve effective tier when fast activation fails", func(t *testing.T) { + ctx, client := newAutoClient(t) + session, err := client.CreateSession(t.Context(), &copilot.SessionConfig{ + Model: "auto", + OnPermissionRequest: copilot.PermissionHandler.ApproveAll, + Capi: &copilot.CapiSessionOptions{ + AutoTier: copilot.AutoTierEfficiency, + EnableWebSocketResponses: copilot.Bool(false), + }, + }) + if err != nil { + t.Fatalf("CreateSession failed: %v", err) + } + sessionID := session.SessionID + if _, err := session.SendAndWait(t.Context(), copilot.MessageOptions{ + Prompt: "Reply with exactly AUTO_TIER_INITIAL_READY.", + }); err != nil { + t.Fatalf("Initial SendAndWait failed: %v", err) + } + + failureObserved := make(chan struct { + data *copilot.SessionAutoTierSwitchFailedData + ephemeral bool + }, 1) + var fastCommitted atomic.Bool + unsubscribe := session.On(func(event copilot.SessionEvent) { + switch data := event.Data.(type) { + case *copilot.SessionAutoTierSwitchFailedData: + select { + case failureObserved <- struct { + data *copilot.SessionAutoTierSwitchFailedData + ephemeral bool + }{data: data, ephemeral: event.Ephemeral != nil && *event.Ephemeral}: + default: + } + case *copilot.SessionModelChangeData: + if data.AutoTier != nil && *data.AutoTier == copilot.AutoTierFast { + fastCommitted.Store(true) + } + } + }) + + staged, err := session.SetAutoTier(t.Context(), autoTier(copilot.AutoTierFast)) + if err != nil { + t.Fatalf("SetAutoTier(fast) failed: %v", err) + } + if staged.Status != rpc.ModelSwitchAutoTierStatusPending || + staged.EffectiveAutoTier == nil || *staged.EffectiveAutoTier != rpc.AutoTierEfficiency || + staged.PendingAutoTier == nil || *staged.PendingAutoTier != rpc.AutoTierFast { + t.Fatalf("Unexpected staged result: %+v", staged) + } + + if _, err := session.SendAndWait(t.Context(), copilot.MessageOptions{ + Prompt: "Reply with exactly AUTO_TIER_FAILURE_RECOVERED.", + }); err != nil { + t.Fatalf("Failure-path SendAndWait failed: %v", err) + } + + select { + case observed := <-failureObserved: + if !observed.ephemeral || + observed.data.EffectiveAutoTier == nil || *observed.data.EffectiveAutoTier != copilot.AutoTierEfficiency || + observed.data.RequestedAutoTier == nil || *observed.data.RequestedAutoTier != copilot.AutoTierFast || + observed.data.Reason != copilot.AutoTierSwitchFailureReasonRequestFailed { + t.Fatalf("Unexpected failure event: %+v", observed) + } + case <-time.After(30 * time.Second): + t.Fatal("Timed out waiting for Fast activation failure") + } + if fastCommitted.Load() { + t.Fatal("Fast tier committed after failed activation") + } + + current, err := session.RPC.Model.GetCurrent(t.Context()) + if err != nil { + t.Fatalf("GetCurrent after failure failed: %v", err) + } + if current.AutoTier == nil || *current.AutoTier != rpc.AutoTierEfficiency || + current.PendingAutoTier != nil || current.ActivatingAutoTier != nil { + t.Fatalf("Unexpected state after failure: %+v", current) + } + + unsubscribe() + session.Disconnect() + if err := client.Stop(); err != nil { + t.Fatalf("Stop failed client failed: %v", err) + } + + resumedClient := ctx.NewClient() + t.Cleanup(func() { resumedClient.ForceStop() }) + if err := resumedClient.Start(t.Context()); err != nil { + t.Fatalf("Start resumed client failed: %v", err) + } + resumed, err := resumedClient.ResumeSession(t.Context(), sessionID, &copilot.ResumeSessionConfig{ + OnPermissionRequest: copilot.PermissionHandler.ApproveAll, + }) + if err != nil { + t.Fatalf("ResumeSession failed: %v", err) + } + resumedCurrent, err := resumed.RPC.Model.GetCurrent(t.Context()) + if err != nil { + t.Fatalf("GetCurrent after resume failed: %v", err) + } + if resumedCurrent.AutoTier == nil || *resumedCurrent.AutoTier != rpc.AutoTierEfficiency || + resumedCurrent.PendingAutoTier != nil || resumedCurrent.ActivatingAutoTier != nil { + t.Fatalf("Unexpected resumed state: %+v", resumedCurrent) + } + max := int64(100) + waitMs := int32(0) + persisted, err := resumed.RPC.EventLog.Read(t.Context(), &rpc.EventLogReadRequest{ + Max: &max, + WaitMs: &waitMs, + }) + if err != nil { + t.Fatalf("EventLog.Read failed: %v", err) + } + for _, event := range persisted.Events { + if event.Type() == copilot.SessionEventTypeSessionAutoTierSwitchFailed { + t.Fatal("Ephemeral failure event was replayed after cold resume") + } + } + resumed.Disconnect() + if err := resumedClient.Stop(); err != nil { + t.Fatalf("Stop resumed client failed: %v", err) + } + }) } diff --git a/go/session_event_serialization_test.go b/go/session_event_serialization_test.go index c64b6ce598..4941abc811 100644 --- a/go/session_event_serialization_test.go +++ b/go/session_event_serialization_test.go @@ -16,7 +16,7 @@ var _ EmbeddedTextResourceContents = rpc.EmbeddedTextResourceContents{} func TestSessionEventAutoTier(t *testing.T) { for _, eventType := range []string{"session.start", "session.resume"} { - for _, tier := range []AutoTier{"", AutoTierEfficiency, AutoTierBalance, AutoTierIntelligence} { + for _, tier := range []AutoTier{"", AutoTierEfficiency, AutoTierBalance, AutoTierIntelligence, AutoTierFast} { t.Run(eventType+"/"+string(tier), func(t *testing.T) { data := map[string]any{ "sessionId": "test-session", "version": 1, @@ -317,7 +317,7 @@ func TestSessionAutoTierSwitchFailedEvent(t *testing.T) { "type": "session.auto_tier_switch_failed", "data": map[string]any{ "effectiveAutoTier": AutoTierBalance, - "requestedAutoTier": AutoTierIntelligence, + "requestedAutoTier": AutoTierFast, "reason": reason, }, }) @@ -338,8 +338,8 @@ func TestSessionAutoTierSwitchFailedEvent(t *testing.T) { if data.EffectiveAutoTier == nil || *data.EffectiveAutoTier != AutoTierBalance { t.Fatalf("expected effective tier %q, got %v", AutoTierBalance, data.EffectiveAutoTier) } - if data.RequestedAutoTier == nil || *data.RequestedAutoTier != AutoTierIntelligence { - t.Fatalf("expected requested tier %q, got %v", AutoTierIntelligence, data.RequestedAutoTier) + if data.RequestedAutoTier == nil || *data.RequestedAutoTier != AutoTierFast { + t.Fatalf("expected requested tier %q, got %v", AutoTierFast, data.RequestedAutoTier) } }) } diff --git a/go/session_test.go b/go/session_test.go index bdf14887a3..c3ab144afe 100644 --- a/go/session_test.go +++ b/go/session_test.go @@ -122,14 +122,14 @@ func TestSession_SetModelOmitsContextTierWhenUnset(t *testing.T) { } func TestSession_SetModelForwardsAutoTier(t *testing.T) { - tier := AutoTierIntelligence + tier := AutoTierFast params := captureSetModelRequestForModel(t, "auto", &SetModelOptions{AutoTier: &tier}) if params["modelId"] != "auto" { t.Fatalf("expected modelId auto, got %v", params["modelId"]) } - if params["autoTier"] != "intelligence" { - t.Fatalf("expected autoTier intelligence, got %v", params["autoTier"]) + if params["autoTier"] != "fast" { + t.Fatalf("expected autoTier fast, got %v", params["autoTier"]) } } @@ -161,14 +161,14 @@ func TestSession_SetModelRejectsConflictingAutoTierOptions(t *testing.T) { } func TestSession_SetAutoTierForwardsTier(t *testing.T) { - tier := AutoTierEfficiency + tier := AutoTierFast params := captureSetAutoTierRequest(t, &tier) if params["sessionId"] != "session-1" { t.Fatalf("expected sessionId session-1, got %v", params["sessionId"]) } - if params["autoTier"] != "efficiency" { - t.Fatalf("expected autoTier efficiency, got %v", params["autoTier"]) + if params["autoTier"] != "fast" { + t.Fatalf("expected autoTier fast, got %v", params["autoTier"]) } } diff --git a/java/README.md b/java/README.md index af13c3ec16..31da76ea2b 100644 --- a/java/README.md +++ b/java/README.md @@ -369,9 +369,13 @@ For design context and decision rationale, see [ADR-006](docs/adr/adr-006-tool-d ## Auto routing tiers Use `CapiSessionOptions.setAutoTier(...)` to select `AutoTier.EFFICIENCY`, -`AutoTier.BALANCE`, or `AutoTier.INTELLIGENCE`. This option is meaningful only -with model `auto` (Auto mode V2). -It requires a runtime version that supports `capi.autoTier`. +`AutoTier.BALANCE`, `AutoTier.INTELLIGENCE`, or `AutoTier.FAST`. This option is +meaningful only with model `auto` (Auto mode V2). +It requires a runtime version that supports `capi.autoTier`; `AutoTier.FAST` +additionally requires Copilot CLI `1.0.84-0` or later. +`AutoTier.FAST` is an integrator-only latency preset, not a first-party GitHub +Copilot product preference — the SDK does not decide Fast eligibility or apply +it implicitly. ```java import com.github.copilot.rpc.AutoTier; @@ -398,7 +402,7 @@ for the lifecycle rules. Change the Auto routing preference without changing the selected model. The runtime does not apply the preference immediately: it records the request and commits it only when a later user turn using the `auto` model successfully obtains a usable model from the provider, so a `pending` status confirms acceptance rather than effect. Only the most recent request survives. -Watch for the outcome through the `session.model_change` event on success or the ephemeral `session.auto_tier_switch_failed` event on failure. Read the authoritative committed, pending, and activating preferences at any time through the session's `model.getCurrent` RPC method. +Watch for the outcome through the `session.model_change` event on success or the ephemeral `session.auto_tier_switch_failed` event on failure. A failed activation leaves the incumbent effective tier unchanged. Read the authoritative committed, pending, and activating preferences at any time through the session's `model.getCurrent` RPC method. ```java var result = session.setAutoTier(AutoTier.INTELLIGENCE).get(); diff --git a/java/sdk/src/main/java/com/github/copilot/rpc/AutoTier.java b/java/sdk/src/main/java/com/github/copilot/rpc/AutoTier.java index f9117abfb2..5a9864682f 100644 --- a/java/sdk/src/main/java/com/github/copilot/rpc/AutoTier.java +++ b/java/sdk/src/main/java/com/github/copilot/rpc/AutoTier.java @@ -21,7 +21,13 @@ public enum AutoTier { BALANCE("balance"), /** Prioritize intelligence. */ - INTELLIGENCE("intelligence"); + INTELLIGENCE("intelligence"), + + /** + * Integrator-only preset that optimizes for latency. Not a first-party GitHub + * Copilot product preference; requires Copilot CLI 1.0.84-0 or later. + */ + FAST("fast"); private final String value; diff --git a/java/sdk/src/test/java/com/github/copilot/AutoTierIT.java b/java/sdk/src/test/java/com/github/copilot/AutoTierIT.java index d486ce1c2c..d93cbf995b 100644 --- a/java/sdk/src/test/java/com/github/copilot/AutoTierIT.java +++ b/java/sdk/src/test/java/com/github/copilot/AutoTierIT.java @@ -1,17 +1,28 @@ package com.github.copilot; import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; import static org.junit.jupiter.api.Assertions.assertNull; +import static org.junit.jupiter.api.Assertions.assertTrue; +import java.util.concurrent.CompletableFuture; import java.util.concurrent.TimeUnit; +import java.util.concurrent.atomic.AtomicBoolean; import org.junit.jupiter.api.AfterAll; import org.junit.jupiter.api.BeforeAll; import org.junit.jupiter.api.Test; +import com.github.copilot.generated.AutoTierSwitchFailureReason; +import com.github.copilot.generated.SessionAutoTierSwitchFailedEvent; +import com.github.copilot.generated.SessionModelChangeEvent; import com.github.copilot.generated.rpc.ModelSwitchAutoTierStatus; +import com.github.copilot.generated.rpc.SessionEventLogReadParams; import com.github.copilot.rpc.AutoTier; +import com.github.copilot.rpc.CapiSessionOptions; +import com.github.copilot.rpc.MessageOptions; import com.github.copilot.rpc.PermissionHandler; +import com.github.copilot.rpc.ResumeSessionConfig; import com.github.copilot.rpc.SessionConfig; import com.github.copilot.rpc.SetModelOptions; @@ -69,10 +80,16 @@ void shouldStageAndResetAutoTierPreference() throws Exception { assertEquals(com.github.copilot.generated.rpc.AutoTier.EFFICIENCY, pendingAutoTier(session)); // A second request replaces the first and reports the one it displaced. - var superseded = session.setAutoTier(AutoTier.INTELLIGENCE).get(30, TimeUnit.SECONDS); + var superseded = session.setAutoTier(AutoTier.FAST).get(30, TimeUnit.SECONDS); assertEquals(ModelSwitchAutoTierStatus.PENDING, superseded.status()); - assertEquals(com.github.copilot.generated.rpc.AutoTier.INTELLIGENCE, superseded.pendingAutoTier()); + assertEquals(com.github.copilot.generated.rpc.AutoTier.FAST, superseded.pendingAutoTier()); assertEquals(com.github.copilot.generated.rpc.AutoTier.EFFICIENCY, superseded.supersededAutoTier()); + assertEquals(com.github.copilot.generated.rpc.AutoTier.FAST, pendingAutoTier(session)); + + var replacedFast = session.setAutoTier(AutoTier.INTELLIGENCE).get(30, TimeUnit.SECONDS); + assertEquals(ModelSwitchAutoTierStatus.PENDING, replacedFast.status()); + assertEquals(com.github.copilot.generated.rpc.AutoTier.INTELLIGENCE, replacedFast.pendingAutoTier()); + assertEquals(com.github.copilot.generated.rpc.AutoTier.FAST, replacedFast.supersededAutoTier()); assertEquals(com.github.copilot.generated.rpc.AutoTier.INTELLIGENCE, pendingAutoTier(session)); // A null tier returns the session to provider-default routing. The status @@ -103,9 +120,9 @@ void shouldPreserveAutoTierWhenSetModelOmitsIt() throws Exception { assertEquals(com.github.copilot.generated.rpc.AutoTier.BALANCE, pendingAutoTier(session)); // Supplying a tier replaces it. - session.setModel(new SetModelOptions().setModel(MODEL_ID).setAutoTier(AutoTier.INTELLIGENCE)).get(30, + session.setModel(new SetModelOptions().setModel(MODEL_ID).setAutoTier(AutoTier.FAST)).get(30, TimeUnit.SECONDS); - assertEquals(com.github.copilot.generated.rpc.AutoTier.INTELLIGENCE, pendingAutoTier(session)); + assertEquals(com.github.copilot.generated.rpc.AutoTier.FAST, pendingAutoTier(session)); // Requesting a reset clears it. Omission, an explicit tier, and a reset // are three distinct outcomes. @@ -117,4 +134,175 @@ void shouldPreserveAutoTierWhenSetModelOmitsIt() throws Exception { } } } + + @Test + void shouldRestoreAndOverrideFastAutoTierOnColdResume() throws Exception { + ctx.configureForTest("auto_tier", "should_restore_and_override_fast_auto_tier_on_cold_resume"); + + String fastSessionId; + String tierlessSessionId; + try (CopilotClient client = ctx.createClient()) { + try (CopilotSession fastSession = client.createSession( + new SessionConfig().setModel(MODEL_ID).setOnPermissionRequest(PermissionHandler.APPROVE_ALL) + .setCapi(new CapiSessionOptions().setAutoTier(AutoTier.FAST) + .setEnableWebSocketResponses(false))) + .get(30, TimeUnit.SECONDS); + CopilotSession tierlessSession = client + .createSession(new SessionConfig().setModel(MODEL_ID) + .setOnPermissionRequest(PermissionHandler.APPROVE_ALL) + .setCapi(new CapiSessionOptions().setEnableWebSocketResponses(false))) + .get(30, TimeUnit.SECONDS)) { + fastSessionId = fastSession.getSessionId(); + tierlessSessionId = tierlessSession.getSessionId(); + + fastSession + .sendAndWait(new MessageOptions().setPrompt("Reply with exactly AUTO_TIER_COLD_RESUME_READY.")) + .get(30, TimeUnit.SECONDS); + tierlessSession + .sendAndWait(new MessageOptions().setPrompt("Reply with exactly AUTO_TIER_TIERLESS_READY.")) + .get(30, TimeUnit.SECONDS); + + assertEquals(com.github.copilot.generated.rpc.AutoTier.FAST, + fastSession.getRpc().model.getCurrent().get(30, TimeUnit.SECONDS).autoTier()); + assertNull(tierlessSession.getRpc().model.getCurrent().get(30, TimeUnit.SECONDS).autoTier()); + } + client.stop().get(30, TimeUnit.SECONDS); + } + + try (CopilotClient restoredClient = ctx.createClient()) { + try (CopilotSession restoredFast = restoredClient + .resumeSession(fastSessionId, + new ResumeSessionConfig().setOnPermissionRequest(PermissionHandler.APPROVE_ALL)) + .get(30, TimeUnit.SECONDS); + CopilotSession restoredTierless = restoredClient + .resumeSession(tierlessSessionId, + new ResumeSessionConfig().setOnPermissionRequest(PermissionHandler.APPROVE_ALL)) + .get(30, TimeUnit.SECONDS)) { + assertEquals(com.github.copilot.generated.rpc.AutoTier.FAST, + restoredFast.getRpc().model.getCurrent().get(30, TimeUnit.SECONDS).autoTier()); + assertNull(restoredTierless.getRpc().model.getCurrent().get(30, TimeUnit.SECONDS).autoTier()); + } + restoredClient.stop().get(30, TimeUnit.SECONDS); + } + + try (CopilotClient overrideClient = ctx.createClient(); + CopilotSession overridden = overrideClient + .resumeSession(fastSessionId, new ResumeSessionConfig().setModel(MODEL_ID) + .setOnPermissionRequest(PermissionHandler.APPROVE_ALL).setCapi(new CapiSessionOptions() + .setAutoTier(AutoTier.BALANCE).setEnableWebSocketResponses(false))) + .get(30, TimeUnit.SECONDS)) { + assertEquals(com.github.copilot.generated.rpc.AutoTier.BALANCE, + overridden.getRpc().model.getCurrent().get(30, TimeUnit.SECONDS).autoTier()); + } + } + + @Test + void shouldCommitFastAutoTierAfterSuccessfulTurn() throws Exception { + ctx.configureForTest("auto_tier", "should_commit_fast_auto_tier_after_successful_turn"); + + try (CopilotClient client = ctx.createClient(); + CopilotSession session = client + .createSession(new SessionConfig().setModel(MODEL_ID) + .setOnPermissionRequest(PermissionHandler.APPROVE_ALL).setCapi(new CapiSessionOptions() + .setAutoTier(AutoTier.EFFICIENCY).setEnableWebSocketResponses(false))) + .get(30, TimeUnit.SECONDS)) { + var modelChangedFuture = new CompletableFuture(); + try (var subscription = session.on(SessionModelChangeEvent.class, event -> { + if (event.getData().autoTier() == com.github.copilot.generated.AutoTier.FAST) { + modelChangedFuture.complete(event); + } + })) { + var staged = session.setAutoTier(AutoTier.FAST).get(30, TimeUnit.SECONDS); + assertEquals(ModelSwitchAutoTierStatus.PENDING, staged.status()); + assertEquals(com.github.copilot.generated.rpc.AutoTier.EFFICIENCY, staged.effectiveAutoTier()); + assertEquals(com.github.copilot.generated.rpc.AutoTier.FAST, staged.pendingAutoTier()); + + var beforeTurn = session.getRpc().model.getCurrent().get(30, TimeUnit.SECONDS); + assertEquals(com.github.copilot.generated.rpc.AutoTier.EFFICIENCY, beforeTurn.autoTier()); + assertEquals(com.github.copilot.generated.rpc.AutoTier.FAST, beforeTurn.pendingAutoTier()); + + session.sendAndWait(new MessageOptions().setPrompt("Reply with exactly AUTO_TIER_FAST_COMMITTED.")) + .get(30, TimeUnit.SECONDS); + + var modelChanged = modelChangedFuture.get(30, TimeUnit.SECONDS); + assertEquals(MODEL_ID, modelChanged.getData().previousModel()); + assertEquals(MODEL_ID, modelChanged.getData().newModel()); + assertEquals(com.github.copilot.generated.AutoTier.EFFICIENCY, + modelChanged.getData().previousAutoTier()); + assertEquals(com.github.copilot.generated.AutoTier.FAST, modelChanged.getData().autoTier()); + + var committed = session.getRpc().model.getCurrent().get(30, TimeUnit.SECONDS); + assertEquals(com.github.copilot.generated.rpc.AutoTier.FAST, committed.autoTier()); + assertNull(committed.pendingAutoTier()); + assertNull(committed.activatingAutoTier()); + } + } + } + + @Test + void shouldPreserveEffectiveTierWhenFastActivationFails() throws Exception { + ctx.configureForTest("auto_tier", "should_preserve_effective_tier_when_fast_activation_fails"); + + String sessionId; + try (CopilotClient client = ctx.createClient()) { + try (CopilotSession session = client + .createSession(new SessionConfig().setModel(MODEL_ID) + .setOnPermissionRequest(PermissionHandler.APPROVE_ALL).setCapi(new CapiSessionOptions() + .setAutoTier(AutoTier.EFFICIENCY).setEnableWebSocketResponses(false))) + .get(30, TimeUnit.SECONDS)) { + sessionId = session.getSessionId(); + session.sendAndWait(new MessageOptions().setPrompt("Reply with exactly AUTO_TIER_INITIAL_READY.")) + .get(30, TimeUnit.SECONDS); + + var failureFuture = new CompletableFuture(); + var fastCommitted = new AtomicBoolean(); + try (var failureSubscription = session.on(SessionAutoTierSwitchFailedEvent.class, + failureFuture::complete); + var modelChangeSubscription = session.on(SessionModelChangeEvent.class, event -> { + if (event.getData().autoTier() == com.github.copilot.generated.AutoTier.FAST) { + fastCommitted.set(true); + } + })) { + var staged = session.setAutoTier(AutoTier.FAST).get(30, TimeUnit.SECONDS); + assertEquals(ModelSwitchAutoTierStatus.PENDING, staged.status()); + assertEquals(com.github.copilot.generated.rpc.AutoTier.EFFICIENCY, staged.effectiveAutoTier()); + assertEquals(com.github.copilot.generated.rpc.AutoTier.FAST, staged.pendingAutoTier()); + + session.sendAndWait( + new MessageOptions().setPrompt("Reply with exactly AUTO_TIER_FAILURE_RECOVERED.")) + .get(30, TimeUnit.SECONDS); + + var failure = failureFuture.get(30, TimeUnit.SECONDS); + assertTrue(failure.getEphemeral()); + assertEquals(com.github.copilot.generated.AutoTier.EFFICIENCY, + failure.getData().effectiveAutoTier()); + assertEquals(com.github.copilot.generated.AutoTier.FAST, failure.getData().requestedAutoTier()); + assertEquals(AutoTierSwitchFailureReason.REQUEST_FAILED, failure.getData().reason()); + assertFalse(fastCommitted.get()); + + var current = session.getRpc().model.getCurrent().get(30, TimeUnit.SECONDS); + assertEquals(com.github.copilot.generated.rpc.AutoTier.EFFICIENCY, current.autoTier()); + assertNull(current.pendingAutoTier()); + assertNull(current.activatingAutoTier()); + } + } + client.stop().get(30, TimeUnit.SECONDS); + } + + try (CopilotClient resumedClient = ctx.createClient(); + CopilotSession resumed = resumedClient + .resumeSession(sessionId, + new ResumeSessionConfig().setOnPermissionRequest(PermissionHandler.APPROVE_ALL)) + .get(30, TimeUnit.SECONDS)) { + var resumedCurrent = resumed.getRpc().model.getCurrent().get(30, TimeUnit.SECONDS); + assertEquals(com.github.copilot.generated.rpc.AutoTier.EFFICIENCY, resumedCurrent.autoTier()); + assertNull(resumedCurrent.pendingAutoTier()); + assertNull(resumedCurrent.activatingAutoTier()); + + var persisted = resumed.getRpc().eventLog + .read(new SessionEventLogReadParams(null, null, 100L, 0L, null, null, null, null, false)) + .get(30, TimeUnit.SECONDS); + assertFalse(persisted.events().stream().anyMatch(SessionAutoTierSwitchFailedEvent.class::isInstance)); + } + } } diff --git a/java/sdk/src/test/java/com/github/copilot/CapiSessionOptionsTest.java b/java/sdk/src/test/java/com/github/copilot/CapiSessionOptionsTest.java index dccb4e9add..aeb4ffe0ad 100644 --- a/java/sdk/src/test/java/com/github/copilot/CapiSessionOptionsTest.java +++ b/java/sdk/src/test/java/com/github/copilot/CapiSessionOptionsTest.java @@ -68,7 +68,7 @@ void omitsUnsetEnableWebSocketResponses() { } @ParameterizedTest - @CsvSource({"EFFICIENCY,efficiency", "BALANCE,balance", "INTELLIGENCE,intelligence"}) + @CsvSource({"EFFICIENCY,efficiency", "BALANCE,balance", "INTELLIGENCE,intelligence", "FAST,fast"}) void autoTierCanonicalValuesRoundTripAndForward(AutoTier tier, String value) throws Exception { var mapper = JsonRpcClient.getObjectMapper(); var capi = new CapiSessionOptions().setAutoTier(tier); diff --git a/java/sdk/src/test/java/com/github/copilot/SessionAutoTierEventTest.java b/java/sdk/src/test/java/com/github/copilot/SessionAutoTierEventTest.java index 25e356a8d1..4378c105cd 100644 --- a/java/sdk/src/test/java/com/github/copilot/SessionAutoTierEventTest.java +++ b/java/sdk/src/test/java/com/github/copilot/SessionAutoTierEventTest.java @@ -31,8 +31,9 @@ class SessionAutoTierEventTest { @ParameterizedTest @CsvSource({"session.start,EFFICIENCY,efficiency", "session.start,BALANCE,balance", - "session.start,INTELLIGENCE,intelligence", "session.resume,EFFICIENCY,efficiency", - "session.resume,BALANCE,balance", "session.resume,INTELLIGENCE,intelligence"}) + "session.start,INTELLIGENCE,intelligence", "session.start,FAST,fast", + "session.resume,EFFICIENCY,efficiency", "session.resume,BALANCE,balance", + "session.resume,INTELLIGENCE,intelligence", "session.resume,FAST,fast"}) void canonicalAutoTierRoundTrips(String type, AutoTier tier, String value) throws Exception { String json = """ {"type":"%s","data":{"selectedModel":"auto","autoTier":"%s"}} @@ -74,14 +75,14 @@ void autoTierSwitchFailedEventDecodesEveryReason(String value, AutoTierSwitchFai throws Exception { String json = """ {"type":"session.auto_tier_switch_failed","data":{"effectiveAutoTier":"balance", - "requestedAutoTier":"intelligence","reason":"%s"}} + "requestedAutoTier":"fast","reason":"%s"}} """.formatted(value); var event = MAPPER.readValue(json, SessionEvent.class); var data = assertInstanceOf(SessionAutoTierSwitchFailedEvent.class, event).getData(); assertEquals(AutoTier.BALANCE, data.effectiveAutoTier()); - assertEquals(AutoTier.INTELLIGENCE, data.requestedAutoTier()); + assertEquals(AutoTier.FAST, data.requestedAutoTier()); assertEquals(reason, data.reason()); } diff --git a/java/sdk/src/test/java/com/github/copilot/SessionAutoTierSwitchTest.java b/java/sdk/src/test/java/com/github/copilot/SessionAutoTierSwitchTest.java index adeeca2c6b..74e02c66f9 100644 --- a/java/sdk/src/test/java/com/github/copilot/SessionAutoTierSwitchTest.java +++ b/java/sdk/src/test/java/com/github/copilot/SessionAutoTierSwitchTest.java @@ -52,13 +52,13 @@ void setModel_sends_requested_autoTier() throws Exception { var session = new CopilotSession("sess-2", sockets.client()); var stub = sockets.stubServer(); - session.setModel(new SetModelOptions().setModel("auto").setAutoTier(AutoTier.INTELLIGENCE) - .setReasoningEffort("high")); + session.setModel( + new SetModelOptions().setModel("auto").setAutoTier(AutoTier.FAST).setReasoningEffort("high")); var sent = stub.readOneMessage(); assertEquals("session.model.switchTo", sent.get("method").asText()); var params = sent.get("params"); - assertEquals("intelligence", params.get("autoTier").asText()); + assertEquals("fast", params.get("autoTier").asText()); assertEquals("high", params.get("reasoningEffort").asText()); assertEquals("sess-2", params.get("sessionId").asText()); } @@ -108,12 +108,12 @@ void setAutoTier_sends_the_requested_tier() throws Exception { var session = new CopilotSession("sess-6", sockets.client()); var stub = sockets.stubServer(); - session.setAutoTier(AutoTier.EFFICIENCY); + session.setAutoTier(AutoTier.FAST); var sent = stub.readOneMessage(); assertEquals("session.model.switchAutoTier", sent.get("method").asText()); var params = sent.get("params"); - assertEquals("efficiency", params.get("autoTier").asText()); + assertEquals("fast", params.get("autoTier").asText()); assertEquals("sess-6", params.get("sessionId").asText()); } } @@ -141,7 +141,7 @@ void switchAutoTier_result_deserializes_every_field() throws Exception { { "status": "pending", "effectiveAutoTier": "balance", - "pendingAutoTier": "intelligence", + "pendingAutoTier": "fast", "activatingAutoTier": null, "supersededAutoTier": "efficiency" } @@ -151,7 +151,7 @@ void switchAutoTier_result_deserializes_every_field() throws Exception { assertEquals(ModelSwitchAutoTierStatus.PENDING, result.status()); assertEquals(com.github.copilot.generated.rpc.AutoTier.BALANCE, result.effectiveAutoTier()); - assertEquals(com.github.copilot.generated.rpc.AutoTier.INTELLIGENCE, result.pendingAutoTier()); + assertEquals(com.github.copilot.generated.rpc.AutoTier.FAST, result.pendingAutoTier()); assertNull(result.activatingAutoTier()); assertEquals(com.github.copilot.generated.rpc.AutoTier.EFFICIENCY, result.supersededAutoTier()); } diff --git a/nodejs/README.md b/nodejs/README.md index 2fb98ce972..b454c35a69 100644 --- a/nodejs/README.md +++ b/nodejs/README.md @@ -151,7 +151,7 @@ Create a new conversation session. - `sessionId?: string` - Custom session ID. - `model?: string` - Model to use ("gpt-5", "claude-sonnet-4.5", etc.). **Required when using custom provider.** -- `capi?: CapiSessionOptions` - Copilot API options. With `model: "auto"`, set `autoTier` to `"efficiency"`, `"balance"`, or `"intelligence"` to choose a routing preference. Requires a runtime with Auto tier support and V2 Auto routing. Omission preserves default behavior. See [Auto tier persistence](../docs/features/session-persistence.md#auto-tier-persistence) for resume semantics. +- `capi?: CapiSessionOptions` - Copilot API options. With `model: "auto"`, set `autoTier` to `"efficiency"`, `"balance"`, `"intelligence"`, or `"fast"` to choose a routing preference. `"fast"` is an integrator-only latency preset, not a first-party GitHub Copilot product preference, and requires Copilot CLI `1.0.84-0` or later. Requires a runtime with Auto tier support and V2 Auto routing. Omission preserves default behavior. See [Auto tier persistence](../docs/features/session-persistence.md#auto-tier-persistence) for resume semantics. - `reasoningEffort?: "low" | "medium" | "high" | "xhigh" | "max"` - Reasoning effort level for models that support it. Use `listModels()` to check which models support this option. - `tools?: Tool[]` - Custom tools exposed to the CLI. Tools without `handler` are declaration-only and must be resolved via pending tool-call RPCs. - `systemMessage?: SystemMessageConfig` - System message customization (see below) @@ -354,7 +354,7 @@ Change the Auto routing preference without changing the selected model. Pass `nu The runtime does not apply the preference immediately. It records the request and commits it only when a later user turn using the `auto` model successfully obtains a usable model from the provider, so a `pending` status confirms acceptance rather than effect. Only the most recent request survives. -Watch for the outcome through the `session.model_change` event on success or the ephemeral `session.auto_tier_switch_failed` event on failure, and read the authoritative state at any time with `session.rpc.model.getCurrent()`. +Watch for the outcome through the `session.model_change` event on success or the ephemeral `session.auto_tier_switch_failed` event on failure. A failed activation leaves the incumbent effective tier unchanged. Read the authoritative state at any time with `session.rpc.model.getCurrent()`. ```typescript const result = await session.setAutoTier("intelligence"); diff --git a/nodejs/test/client.test.ts b/nodejs/test/client.test.ts index 24adb5cb6e..f8f3848b11 100644 --- a/nodejs/test/client.test.ts +++ b/nodejs/test/client.test.ts @@ -1432,6 +1432,7 @@ describe("CopilotClient", () => { { autoTier: "efficiency" }, { autoTier: "balance" }, { autoTier: "intelligence" }, + { autoTier: "fast" }, { autoTier: "balance", enableWebSocketResponses: false }, ] satisfies (CapiSessionOptions | undefined)[])( "forwards capi options %j in session.create and session.resume", @@ -2676,12 +2677,12 @@ describe("CopilotClient", () => { throw new Error(`Unexpected method: ${method}`); }); - await session.setModel("auto", { autoTier: "intelligence" }); + await session.setModel("auto", { autoTier: "fast" }); expect(spy).toHaveBeenCalledWith("session.model.switchTo", { sessionId: session.sessionId, modelId: "auto", - autoTier: "intelligence", + autoTier: "fast", }); spy.mockRestore(); @@ -2726,7 +2727,7 @@ describe("CopilotClient", () => { return { status: "pending", effectiveAutoTier: "balance", - pendingAutoTier: "intelligence", + pendingAutoTier: "fast", activatingAutoTier: null, supersededAutoTier: null, }; @@ -2734,15 +2735,15 @@ describe("CopilotClient", () => { throw new Error(`Unexpected method: ${method}`); }); - const result = await session.setAutoTier("intelligence"); + const result = await session.setAutoTier("fast"); expect(spy).toHaveBeenCalledWith("session.model.switchAutoTier", { sessionId: session.sessionId, - autoTier: "intelligence", + autoTier: "fast", }); expect(result.status).toBe("pending"); expect(result.effectiveAutoTier).toBe("balance"); - expect(result.pendingAutoTier).toBe("intelligence"); + expect(result.pendingAutoTier).toBe("fast"); expect(result.activatingAutoTier).toBeNull(); spy.mockRestore(); diff --git a/nodejs/test/e2e/auto_tier.e2e.test.ts b/nodejs/test/e2e/auto_tier.e2e.test.ts index 0cb2a1a266..f2b5717e60 100644 --- a/nodejs/test/e2e/auto_tier.e2e.test.ts +++ b/nodejs/test/e2e/auto_tier.e2e.test.ts @@ -5,6 +5,7 @@ import { describe, expect, it } from "vitest"; import { approveAll } from "../../src/index.js"; import { createSdkTestContext } from "./harness/sdkTestContext.js"; +import { getNextEventOfType } from "./harness/sdkTestHelper.js"; /** * The runtime stages an Auto routing preference instead of applying it immediately: a @@ -13,7 +14,7 @@ import { createSdkTestContext } from "./harness/sdkTestContext.js"; * they assert what the runtime actually recorded rather than what the SDK serialized. */ describe("Auto tier switching", async () => { - const { copilotClient: client } = await createSdkTestContext(); + const { copilotClient: client, createClient } = await createSdkTestContext(); it("should stage and reset auto tier preference", async () => { const session = await client.createSession({ @@ -29,10 +30,16 @@ describe("Auto tier switching", async () => { expect((await session.rpc.model.getCurrent()).pendingAutoTier).toBe("efficiency"); // A second request replaces the first and reports the one it displaced. - const superseded = await session.setAutoTier("intelligence"); + const superseded = await session.setAutoTier("fast"); expect(superseded.status).toBe("pending"); - expect(superseded.pendingAutoTier).toBe("intelligence"); + expect(superseded.pendingAutoTier).toBe("fast"); expect(superseded.supersededAutoTier).toBe("efficiency"); + expect((await session.rpc.model.getCurrent()).pendingAutoTier).toBe("fast"); + + const replacedFast = await session.setAutoTier("intelligence"); + expect(replacedFast.status).toBe("pending"); + expect(replacedFast.pendingAutoTier).toBe("intelligence"); + expect(replacedFast.supersededAutoTier).toBe("fast"); expect((await session.rpc.model.getCurrent()).pendingAutoTier).toBe("intelligence"); // Passing null returns the session to provider-default routing. The status is @@ -60,8 +67,8 @@ describe("Auto tier switching", async () => { expect((await session.rpc.model.getCurrent()).pendingAutoTier).toBe("balance"); // Supplying a tier replaces it. - await session.setModel("auto", { autoTier: "intelligence" }); - expect((await session.rpc.model.getCurrent()).pendingAutoTier).toBe("intelligence"); + await session.setModel("auto", { autoTier: "fast" }); + expect((await session.rpc.model.getCurrent()).pendingAutoTier).toBe("fast"); // Supplying null clears it. Omission, a value, and null are three distinct // outcomes, which is why the option cannot collapse to a plain optional field. @@ -70,4 +77,150 @@ describe("Auto tier switching", async () => { await session.disconnect(); }); + + it("should restore and override fast auto tier on cold resume", async () => { + const fastSession = await client.createSession({ + onPermissionRequest: approveAll, + model: "auto", + capi: { autoTier: "fast", enableWebSocketResponses: false }, + }); + const tierlessSession = await client.createSession({ + onPermissionRequest: approveAll, + model: "auto", + capi: { enableWebSocketResponses: false }, + }); + const fastSessionId = fastSession.sessionId; + const tierlessSessionId = tierlessSession.sessionId; + + await fastSession.sendAndWait({ + prompt: "Reply with exactly AUTO_TIER_COLD_RESUME_READY.", + }); + await tierlessSession.sendAndWait({ + prompt: "Reply with exactly AUTO_TIER_TIERLESS_READY.", + }); + expect((await fastSession.rpc.model.getCurrent()).autoTier).toBe("fast"); + expect((await tierlessSession.rpc.model.getCurrent()).autoTier).toBeUndefined(); + + await fastSession.disconnect(); + await tierlessSession.disconnect(); + await client.stop(); + + const restoredClient = createClient(); + const restoredFast = await restoredClient.resumeSession(fastSessionId, { + onPermissionRequest: approveAll, + }); + const restoredTierless = await restoredClient.resumeSession(tierlessSessionId, { + onPermissionRequest: approveAll, + }); + expect((await restoredFast.rpc.model.getCurrent()).autoTier).toBe("fast"); + expect((await restoredTierless.rpc.model.getCurrent()).autoTier).toBeUndefined(); + await restoredFast.disconnect(); + await restoredTierless.disconnect(); + await restoredClient.stop(); + + const overrideClient = createClient(); + const overridden = await overrideClient.resumeSession(fastSessionId, { + onPermissionRequest: approveAll, + model: "auto", + capi: { autoTier: "balance", enableWebSocketResponses: false }, + }); + expect((await overridden.rpc.model.getCurrent()).autoTier).toBe("balance"); + await overridden.disconnect(); + await overrideClient.stop(); + }, 120_000); + + it("should commit fast auto tier after successful turn", async () => { + const session = await client.createSession({ + onPermissionRequest: approveAll, + model: "auto", + capi: { autoTier: "efficiency", enableWebSocketResponses: false }, + }); + + const modelChangePromise = getNextEventOfType(session, "session.model_change"); + const staged = await session.setAutoTier("fast"); + expect(staged.status).toBe("pending"); + expect(staged.effectiveAutoTier).toBe("efficiency"); + expect(staged.pendingAutoTier).toBe("fast"); + + const beforeTurn = await session.rpc.model.getCurrent(); + expect(beforeTurn.autoTier).toBe("efficiency"); + expect(beforeTurn.pendingAutoTier).toBe("fast"); + + await session.sendAndWait({ + prompt: "Reply with exactly AUTO_TIER_FAST_COMMITTED.", + }); + + const modelChange = await modelChangePromise; + expect(modelChange.data.previousModel).toBe("auto"); + expect(modelChange.data.newModel).toBe("auto"); + expect(modelChange.data.previousAutoTier).toBe("efficiency"); + expect(modelChange.data.autoTier).toBe("fast"); + + const committed = await session.rpc.model.getCurrent(); + expect(committed.autoTier).toBe("fast"); + expect(committed.pendingAutoTier).toBeUndefined(); + expect(committed.activatingAutoTier).toBeUndefined(); + + await session.disconnect(); + }, 120_000); + + it("should preserve effective tier when fast activation fails", async () => { + const session = await client.createSession({ + onPermissionRequest: approveAll, + model: "auto", + capi: { autoTier: "efficiency", enableWebSocketResponses: false }, + }); + const sessionId = session.sessionId; + + await session.sendAndWait({ + prompt: "Reply with exactly AUTO_TIER_INITIAL_READY.", + }); + + let fastCommitted = false; + const unsubscribeModelChange = session.on("session.model_change", (event) => { + fastCommitted ||= event.data.autoTier === "fast"; + }); + const failurePromise = getNextEventOfType(session, "session.auto_tier_switch_failed"); + + const staged = await session.setAutoTier("fast"); + expect(staged.status).toBe("pending"); + expect(staged.effectiveAutoTier).toBe("efficiency"); + expect(staged.pendingAutoTier).toBe("fast"); + + await session.sendAndWait({ + prompt: "Reply with exactly AUTO_TIER_FAILURE_RECOVERED.", + }); + + const failure = await failurePromise; + expect(failure.ephemeral).toBe(true); + expect(failure.data.effectiveAutoTier).toBe("efficiency"); + expect(failure.data.requestedAutoTier).toBe("fast"); + expect(failure.data.reason).toBe("request_failed"); + expect(fastCommitted).toBe(false); + + const current = await session.rpc.model.getCurrent(); + expect(current.autoTier).toBe("efficiency"); + expect(current.pendingAutoTier).toBeUndefined(); + expect(current.activatingAutoTier).toBeUndefined(); + + unsubscribeModelChange(); + await session.disconnect(); + await client.stop(); + + const resumedClient = createClient(); + const resumed = await resumedClient.resumeSession(sessionId, { + onPermissionRequest: approveAll, + }); + const resumedCurrent = await resumed.rpc.model.getCurrent(); + expect(resumedCurrent.autoTier).toBe("efficiency"); + expect(resumedCurrent.pendingAutoTier).toBeUndefined(); + expect(resumedCurrent.activatingAutoTier).toBeUndefined(); + + const persisted = await resumed.rpc.eventLog.read({ max: 100, waitMs: 0 }); + expect( + persisted.events.some((event) => event.type === "session.auto_tier_switch_failed") + ).toBe(false); + await resumed.disconnect(); + await resumedClient.stop(); + }, 120_000); }); diff --git a/nodejs/test/session-event-types.test.ts b/nodejs/test/session-event-types.test.ts index d20f3caaf6..cc5cb12bf1 100644 --- a/nodejs/test/session-event-types.test.ts +++ b/nodejs/test/session-event-types.test.ts @@ -133,7 +133,7 @@ type _PermissionRequestedEventStaysAlignedWithSessionEventUnion = _AssertEqual< const _permissionRequestedEventAlignmentCheck: _PermissionRequestedEventStaysAlignedWithSessionEventUnion = true; describe("Session event type exports (#1156)", () => { - it.each(["efficiency", "balance", "intelligence", undefined] satisfies ( + it.each(["efficiency", "balance", "intelligence", "fast", undefined] satisfies ( | AutoTier | undefined )[])("exposes Auto tier %s on start and resume data", (autoTier) => { @@ -169,7 +169,7 @@ describe("Session event type exports (#1156)", () => { (reason) => { const data: AutoTierSwitchFailedData = { reason, - requestedAutoTier: "intelligence", + requestedAutoTier: "fast", effectiveAutoTier: "balance", }; const event: AutoTierSwitchFailedEvent = { @@ -186,7 +186,7 @@ describe("Session event type exports (#1156)", () => { const asSessionEvent: SessionEvent = event; expect(asSessionEvent.type).toBe("session.auto_tier_switch_failed"); expect(data.reason).toBe(reason); - expect(data.requestedAutoTier).toBe("intelligence"); + expect(data.requestedAutoTier).toBe("fast"); } ); diff --git a/python/README.md b/python/README.md index 3b7d937c44..603907b12c 100644 --- a/python/README.md +++ b/python/README.md @@ -306,7 +306,7 @@ finally: These are passed as keyword arguments to `create_session()`: - `model` (str): Model to use ("gpt-5", "claude-sonnet-4.5", etc.). **Required when using custom provider.** -- `capi` (CapiSessionOptions): Copilot API options. With `model="auto"`, set `auto_tier` to `"efficiency"`, `"balance"`, or `"intelligence"` to choose a routing preference. Requires a runtime with Auto tier support and V2 Auto routing. Omission preserves default behavior. See [Auto tier persistence](../docs/features/session-persistence.md#auto-tier-persistence) for resume semantics. +- `capi` (CapiSessionOptions): Copilot API options. With `model="auto"`, set `auto_tier` to `"efficiency"`, `"balance"`, `"intelligence"`, or `"fast"` to choose a routing preference. `"fast"` is an integrator-only latency preset, not a first-party GitHub Copilot product preference, and requires Copilot CLI `1.0.84-0` or later. Requires a runtime with Auto tier support and V2 Auto routing. Omission preserves default behavior. See [Auto tier persistence](../docs/features/session-persistence.md#auto-tier-persistence) for resume semantics. - `reasoning_effort` (str): Reasoning effort level for models that support it ("low", "medium", "high", "xhigh", "max"). Use `list_models()` to check which models support this option. - `session_id` (str): Custom session ID - `tools` (list): Custom tools exposed to the CLI. Tools with `handler=None` are declaration-only and must be resolved via pending tool-call RPCs. @@ -492,7 +492,7 @@ async def lookup_issue(params: LookupParams) -> str: Change the Auto routing preference without changing the selected model. The runtime does not apply the preference immediately: it records the request and commits it only when a later user turn using the `auto` model successfully obtains a usable model from the provider, so a `pending` status confirms acceptance rather than effect. Only the most recent request survives. -Watch for the outcome through the `session.model_change` event on success or the ephemeral `session.auto_tier_switch_failed` event on failure. Read the authoritative committed, pending, and activating preferences at any time through the session's `model.getCurrent` RPC method. +Watch for the outcome through the `session.model_change` event on success or the ephemeral `session.auto_tier_switch_failed` event on failure. A failed activation leaves the incumbent effective tier unchanged. Read the authoritative committed, pending, and activating preferences at any time through the session's `model.getCurrent` RPC method. ```python result = await session.set_auto_tier("intelligence") diff --git a/python/copilot/client.py b/python/copilot/client.py index d8d2f7e3f4..4d9d4dfb76 100644 --- a/python/copilot/client.py +++ b/python/copilot/client.py @@ -2385,8 +2385,10 @@ async def create_session( working_directory: Working directory for the session. provider: Provider configuration for Azure or custom endpoints. capi: CAPI provider-scoped options. Set ``auto_tier`` to ``efficiency``, - ``balance``, or ``intelligence`` to select an Auto routing preference - on a runtime with Auto tier support. WebSocket transport is the + ``balance``, ``intelligence``, or ``fast`` to select an Auto routing + preference on a runtime with Auto tier support. ``fast`` is an + integrator-only latency preset, not a first-party GitHub Copilot + product preference. WebSocket transport is the default for the CAPI Responses API whenever the model advertises the ``ws:/responses`` endpoint. Set ``enable_web_socket_responses=False`` to force the HTTP diff --git a/python/copilot/session.py b/python/copilot/session.py index f298ce7802..4149aa5893 100644 --- a/python/copilot/session.py +++ b/python/copilot/session.py @@ -180,7 +180,7 @@ def _capabilities_to_dict(caps: ModelCapabilitiesOverride) -> dict: ReasoningEffort = Literal["low", "medium", "high", "xhigh", "max"] ReasoningSummary = Literal["none", "concise", "detailed"] ContextTier = Literal["default", "long_context"] -AutoTier = Literal["efficiency", "balance", "intelligence"] +AutoTier = Literal["efficiency", "balance", "intelligence", "fast"] SessionFsConventions = Literal["posix", "windows"] diff --git a/python/e2e/test_auto_tier_e2e.py b/python/e2e/test_auto_tier_e2e.py index 5a878c4655..3a9223cce6 100644 --- a/python/e2e/test_auto_tier_e2e.py +++ b/python/e2e/test_auto_tier_e2e.py @@ -12,11 +12,16 @@ import pytest -from copilot.rpc import ModelSwitchAutoTierStatus +from copilot import CopilotClient, RuntimeConnection +from copilot.rpc import EventLogReadRequest, ModelSwitchAutoTierStatus from copilot.session import PermissionHandler -from copilot.session_events import AutoTier +from copilot.session_events import ( + AutoTier, + SessionAutoTierSwitchFailedData, + SessionModelChangeData, +) -from .testharness import E2ETestContext +from .testharness import DEFAULT_GITHUB_TOKEN, E2ETestContext, get_next_event_of_type pytestmark = pytest.mark.asyncio(loop_scope="module") @@ -25,6 +30,15 @@ async def pending_auto_tier(session) -> AutoTier | None: return (await session.rpc.model.get_current()).pending_auto_tier +def create_auto_client(ctx: E2ETestContext) -> CopilotClient: + return CopilotClient( + connection=RuntimeConnection.for_stdio(path=ctx.cli_path), + working_directory=ctx.work_dir, + env=ctx.get_env(), + github_token=DEFAULT_GITHUB_TOKEN, + ) + + class TestAutoTier: async def test_should_stage_and_reset_auto_tier_preference(self, ctx: E2ETestContext): session = await ctx.client.create_session( @@ -40,10 +54,16 @@ async def test_should_stage_and_reset_auto_tier_preference(self, ctx: E2ETestCon assert await pending_auto_tier(session) == AutoTier.EFFICIENCY # A second request replaces the first and reports the one it displaced. - superseded = await session.set_auto_tier("intelligence") + superseded = await session.set_auto_tier("fast") assert superseded.status == ModelSwitchAutoTierStatus.PENDING - assert superseded.pending_auto_tier == AutoTier.INTELLIGENCE + assert superseded.pending_auto_tier == AutoTier.FAST assert superseded.superseded_auto_tier == AutoTier.EFFICIENCY + assert await pending_auto_tier(session) == AutoTier.FAST + + replaced_fast = await session.set_auto_tier("intelligence") + assert replaced_fast.status == ModelSwitchAutoTierStatus.PENDING + assert replaced_fast.pending_auto_tier == AutoTier.INTELLIGENCE + assert replaced_fast.superseded_auto_tier == AutoTier.FAST assert await pending_auto_tier(session) == AutoTier.INTELLIGENCE # Passing None returns the session to provider-default routing. The status is @@ -70,8 +90,8 @@ async def test_should_preserve_auto_tier_when_set_model_omits_it(self, ctx: E2ET assert await pending_auto_tier(session) == AutoTier.BALANCE # Supplying a tier replaces it. - await session.set_model("auto", auto_tier="intelligence") - assert await pending_auto_tier(session) == AutoTier.INTELLIGENCE + await session.set_model("auto", auto_tier="fast") + assert await pending_auto_tier(session) == AutoTier.FAST # Supplying None clears it. Omission, a value, and None are three distinct # outcomes, which is why the argument cannot collapse to a plain optional. @@ -79,3 +99,158 @@ async def test_should_preserve_auto_tier_when_set_model_omits_it(self, ctx: E2ET assert await pending_auto_tier(session) is None finally: await session.disconnect() + + async def test_should_restore_and_override_fast_auto_tier_on_cold_resume( + self, ctx: E2ETestContext + ): + client = create_auto_client(ctx) + fast_session = await client.create_session( + on_permission_request=PermissionHandler.approve_all, + model="auto", + capi={"auto_tier": "fast", "enable_web_socket_responses": False}, + ) + tierless_session = await client.create_session( + on_permission_request=PermissionHandler.approve_all, + model="auto", + capi={"enable_web_socket_responses": False}, + ) + fast_session_id = fast_session.session_id + tierless_session_id = tierless_session.session_id + + await fast_session.send_and_wait("Reply with exactly AUTO_TIER_COLD_RESUME_READY.") + await tierless_session.send_and_wait("Reply with exactly AUTO_TIER_TIERLESS_READY.") + assert (await fast_session.rpc.model.get_current()).auto_tier == AutoTier.FAST + assert (await tierless_session.rpc.model.get_current()).auto_tier is None + + await fast_session.disconnect() + await tierless_session.disconnect() + await client.stop() + + restored_client = create_auto_client(ctx) + restored_fast = await restored_client.resume_session( + fast_session_id, + on_permission_request=PermissionHandler.approve_all, + ) + restored_tierless = await restored_client.resume_session( + tierless_session_id, + on_permission_request=PermissionHandler.approve_all, + ) + assert (await restored_fast.rpc.model.get_current()).auto_tier == AutoTier.FAST + assert (await restored_tierless.rpc.model.get_current()).auto_tier is None + await restored_fast.disconnect() + await restored_tierless.disconnect() + await restored_client.stop() + + override_client = create_auto_client(ctx) + overridden = await override_client.resume_session( + fast_session_id, + on_permission_request=PermissionHandler.approve_all, + model="auto", + capi={"auto_tier": "balance", "enable_web_socket_responses": False}, + ) + assert (await overridden.rpc.model.get_current()).auto_tier == AutoTier.BALANCE + await overridden.disconnect() + await override_client.stop() + + async def test_should_commit_fast_auto_tier_after_successful_turn(self, ctx: E2ETestContext): + client = create_auto_client(ctx) + session = await client.create_session( + on_permission_request=PermissionHandler.approve_all, + model="auto", + capi={"auto_tier": "efficiency", "enable_web_socket_responses": False}, + ) + try: + model_change_task = get_next_event_of_type(session, "session.model_change") + staged = await session.set_auto_tier("fast") + assert staged.status == ModelSwitchAutoTierStatus.PENDING + assert staged.effective_auto_tier == AutoTier.EFFICIENCY + assert staged.pending_auto_tier == AutoTier.FAST + + before_turn = await session.rpc.model.get_current() + assert before_turn.auto_tier == AutoTier.EFFICIENCY + assert before_turn.pending_auto_tier == AutoTier.FAST + + await session.send_and_wait("Reply with exactly AUTO_TIER_FAST_COMMITTED.") + + model_change = await model_change_task + assert isinstance(model_change.data, SessionModelChangeData) + assert model_change.data.previous_model == "auto" + assert model_change.data.new_model == "auto" + assert model_change.data.previous_auto_tier == AutoTier.EFFICIENCY + assert model_change.data.auto_tier == AutoTier.FAST + + committed = await session.rpc.model.get_current() + assert committed.auto_tier == AutoTier.FAST + assert committed.pending_auto_tier is None + assert committed.activating_auto_tier is None + finally: + await session.disconnect() + await client.stop() + + async def test_should_preserve_effective_tier_when_fast_activation_fails( + self, ctx: E2ETestContext + ): + client = create_auto_client(ctx) + session = await client.create_session( + on_permission_request=PermissionHandler.approve_all, + model="auto", + capi={"auto_tier": "efficiency", "enable_web_socket_responses": False}, + ) + session_id = session.session_id + + await session.send_and_wait("Reply with exactly AUTO_TIER_INITIAL_READY.") + + fast_committed = False + + def on_event(event): + nonlocal fast_committed + if isinstance(event.data, SessionModelChangeData): + fast_committed |= event.data.auto_tier == AutoTier.FAST + + unsubscribe = session.on(on_event) + failure_task = get_next_event_of_type( + session, "session.auto_tier_switch_failed" + ) + + staged = await session.set_auto_tier("fast") + assert staged.status == ModelSwitchAutoTierStatus.PENDING + assert staged.effective_auto_tier == AutoTier.EFFICIENCY + assert staged.pending_auto_tier == AutoTier.FAST + + await session.send_and_wait("Reply with exactly AUTO_TIER_FAILURE_RECOVERED.") + + failure = await failure_task + assert failure.ephemeral is True + assert isinstance(failure.data, SessionAutoTierSwitchFailedData) + assert failure.data.effective_auto_tier == AutoTier.EFFICIENCY + assert failure.data.requested_auto_tier == AutoTier.FAST + assert failure.data.reason.value == "request_failed" + assert fast_committed is False + + current = await session.rpc.model.get_current() + assert current.auto_tier == AutoTier.EFFICIENCY + assert current.pending_auto_tier is None + assert current.activating_auto_tier is None + + unsubscribe() + await session.disconnect() + await client.stop() + + resumed_client = create_auto_client(ctx) + try: + resumed = await resumed_client.resume_session( + session_id, + on_permission_request=PermissionHandler.approve_all, + ) + resumed_current = await resumed.rpc.model.get_current() + assert resumed_current.auto_tier == AutoTier.EFFICIENCY + assert resumed_current.pending_auto_tier is None + assert resumed_current.activating_auto_tier is None + + persisted = await resumed.rpc.event_log.read(EventLogReadRequest(max=100, wait_ms=0)) + assert not any( + event.type.value == "session.auto_tier_switch_failed" for event in persisted.events + ) + await resumed.disconnect() + finally: + await resumed_client.stop() diff --git a/python/test_client.py b/python/test_client.py index 2e3868ef1c..775d54a5db 100644 --- a/python/test_client.py +++ b/python/test_client.py @@ -1246,6 +1246,12 @@ async def mock_request(method, params, **kwargs): {"autoTier": "intelligence"}, {"autoTier": "intelligence"}, ), + ( + {"auto_tier": "fast"}, + {"auto_tier": "fast"}, + {"autoTier": "fast"}, + {"autoTier": "fast"}, + ), ( {"auto_tier": "balance", "enable_web_socket_responses": False}, {"auto_tier": "balance", "enable_web_socket_responses": True}, @@ -2673,10 +2679,10 @@ async def mock_request(method, params, **kwargs): return await original_request(method, params, **kwargs) client._client.request = mock_request - await session.set_model("auto", auto_tier="intelligence") + await session.set_model("auto", auto_tier="fast") assert captured["session.model.switchTo"]["sessionId"] == session.session_id assert captured["session.model.switchTo"]["modelId"] == "auto" - assert captured["session.model.switchTo"]["autoTier"] == "intelligence" + assert captured["session.model.switchTo"]["autoTier"] == "fast" finally: await client.force_stop() @@ -2711,7 +2717,7 @@ async def mock_request(method, params, **kwargs): class TestSetAutoTier: @pytest.mark.asyncio - @pytest.mark.parametrize("auto_tier", ["efficiency", "balance", "intelligence", None]) + @pytest.mark.parametrize("auto_tier", ["efficiency", "balance", "intelligence", "fast", None]) async def test_set_auto_tier_sends_correct_rpc(self, auto_tier): client = CopilotClient(connection=RuntimeConnection.for_stdio(path=CLI_PATH)) await client.start() @@ -2766,16 +2772,16 @@ async def test_set_auto_tier_accepts_the_enum_it_returns(self): async def mock_request(method, params, **kwargs): captured[method] = params if method == "session.model.switchAutoTier": - return {"status": "pending", "effectiveAutoTier": "intelligence"} + return {"status": "pending", "effectiveAutoTier": "fast"} return await original_request(method, params, **kwargs) client._client.request = mock_request - await session.set_auto_tier(AutoTierEnum.INTELLIGENCE) + await session.set_auto_tier(AutoTierEnum.FAST) params = captured["session.model.switchAutoTier"] # The value must be a plain string; the JSON-RPC encoder cannot # serialize an enum. - assert params["autoTier"] == "intelligence" + assert params["autoTier"] == "fast" assert isinstance(params["autoTier"], str) json.dumps(params) finally: diff --git a/python/test_event_forward_compatibility.py b/python/test_event_forward_compatibility.py index 65e39a80ba..4d60d63b87 100644 --- a/python/test_event_forward_compatibility.py +++ b/python/test_event_forward_compatibility.py @@ -40,7 +40,7 @@ class TestEventForwardCompatibility: """Test forward compatibility for unknown event types.""" @pytest.mark.parametrize("event_type", ["session.start", "session.resume"]) - @pytest.mark.parametrize("tier", ["efficiency", "balance", "intelligence", None]) + @pytest.mark.parametrize("tier", ["efficiency", "balance", "intelligence", "fast", None]) def test_auto_tier_lifecycle_events_round_trip(self, event_type, tier): timestamp = "2026-08-28T00:00:00Z" data = ( @@ -87,7 +87,7 @@ def test_auto_tier_switch_failed_event_decodes_every_reason(self, reason): "type": "session.auto_tier_switch_failed", "data": { "effectiveAutoTier": "balance", - "requestedAutoTier": "intelligence", + "requestedAutoTier": "fast", "reason": reason, }, } @@ -95,7 +95,7 @@ def test_auto_tier_switch_failed_event_decodes_every_reason(self, reason): assert isinstance(event.data, SessionAutoTierSwitchFailedData) assert event.data.reason == AutoTierSwitchFailureReason(reason) assert event.data.effective_auto_tier == AutoTier.BALANCE - assert event.data.requested_auto_tier == AutoTier.INTELLIGENCE + assert event.data.requested_auto_tier == AutoTier.FAST def test_auto_tier_switch_failed_event_allows_null_requested_tier(self): # A null requested tier means the attempt to return to provider-default diff --git a/rust/README.md b/rust/README.md index d50dc96ec8..a504b3f935 100644 --- a/rust/README.md +++ b/rust/README.md @@ -365,9 +365,13 @@ next credential-consuming operation; there is no background refresh timer. ### Auto routing tiers Use `CapiSessionOptions::with_auto_tier` to select `AutoTier::Efficiency`, -`AutoTier::Balance`, or `AutoTier::Intelligence`. This option is meaningful only -with model `auto` (Auto mode V2). -It requires a runtime version that supports `capi.autoTier`. +`AutoTier::Balance`, `AutoTier::Intelligence`, or `AutoTier::Fast`. This option +is meaningful only with model `auto` (Auto mode V2). +It requires a runtime version that supports `capi.autoTier`; `AutoTier::Fast` +additionally requires Copilot CLI `1.0.84-0` or later. +`AutoTier::Fast` is an integrator-only latency preset, not a first-party +GitHub Copilot product preference — the SDK does not decide Fast eligibility +or apply it implicitly. ```rust use github_copilot_sdk::{AutoTier, CapiSessionOptions, SessionConfig}; @@ -388,7 +392,7 @@ resume succeeds; it cannot change a turn that is already in flight. The SDK does Change the Auto routing preference without changing the selected model. The runtime does not apply the preference immediately: it records the request and commits it only when a later user turn using the `auto` model successfully obtains a usable model from the provider, so a `pending` status confirms acceptance rather than effect. Only the most recent request survives. -Watch for the outcome through the `session.model_change` event on success or the ephemeral `session.auto_tier_switch_failed` event on failure. Read the authoritative committed, pending, and activating preferences at any time through the session's `model.getCurrent` RPC method. +Watch for the outcome through the `session.model_change` event on success or the ephemeral `session.auto_tier_switch_failed` event on failure. A failed activation leaves the incumbent effective tier unchanged. Read the authoritative committed, pending, and activating preferences at any time through the session's `model.getCurrent` RPC method. ```rust,ignore use github_copilot_sdk::{AutoTier, ModelSwitchAutoTierStatus}; diff --git a/rust/src/types.rs b/rust/src/types.rs index 8e4051d287..9b8eb8357d 100644 --- a/rust/src/types.rs +++ b/rust/src/types.rs @@ -24,6 +24,9 @@ use crate::generated::api_types::{CurrentToolMetadata, OpenCanvasInstance}; /// Acknowledgement and Auto preference snapshot returned by an Auto tier switch. pub use crate::generated::api_types::{ModelSwitchAutoTierResult, ModelSwitchAutoTierStatus}; /// Routing tier for the `auto` model with Auto mode V2. +/// +/// [`AutoTier::Fast`] is an integrator-only latency preset, not a first-party +/// GitHub Copilot product preference. pub use crate::generated::session_events::AutoTier; use crate::generated::session_events::ReasoningSummary; /// Context window tier for models that support tiered context windows. @@ -1422,7 +1425,8 @@ impl ProviderConfig { #[non_exhaustive] pub struct CapiSessionOptions { /// Routing tier, meaningful only with model `auto` (Auto mode V2). - /// Requires a runtime version that supports `capi.autoTier`. + /// Requires a runtime version that supports `capi.autoTier`. Fast + /// requires Copilot CLI 1.0.84-0 or later. /// /// When omitted, the runtime chooses its default on create and restores /// the last committed tier on cold resume. On resident resume, a different @@ -7644,6 +7648,7 @@ mod tests { (AutoTier::Efficiency, "efficiency"), (AutoTier::Balance, "balance"), (AutoTier::Intelligence, "intelligence"), + (AutoTier::Fast, "fast"), ] { let exported: crate::AutoTier = tier.clone(); let capi = CapiSessionOptions::new().with_auto_tier(exported); diff --git a/rust/tests/api_types_test.rs b/rust/tests/api_types_test.rs index 4d51340925..94f233f19a 100644 --- a/rust/tests/api_types_test.rs +++ b/rust/tests/api_types_test.rs @@ -22,6 +22,7 @@ fn session_events_deserialize_auto_tier() { (Some(AutoTier::Efficiency), Some("efficiency")), (Some(AutoTier::Balance), Some("balance")), (Some(AutoTier::Intelligence), Some("intelligence")), + (Some(AutoTier::Fast), Some("fast")), (None, None), ] { let mut wire = serde_json::json!({ @@ -279,6 +280,7 @@ fn switch_auto_tier_request_serializes_each_tier() { (AutoTier::Efficiency, "efficiency"), (AutoTier::Balance, "balance"), (AutoTier::Intelligence, "intelligence"), + (AutoTier::Fast, "fast"), ] { let request = ModelSwitchAutoTierRequest { auto_tier: Some(tier), @@ -294,16 +296,17 @@ fn switch_auto_tier_result_deserializes_full_snapshot() { let result: ModelSwitchAutoTierResult = serde_json::from_value(serde_json::json!({ "status": "pending", "effectiveAutoTier": "balance", - "pendingAutoTier": "intelligence", + "pendingAutoTier": "fast", "activatingAutoTier": null, - "supersededAutoTier": null + "supersededAutoTier": "efficiency" })) .unwrap(); assert_eq!(result.status, ModelSwitchAutoTierStatus::Pending); assert_eq!(result.effective_auto_tier, Some(AutoTier::Balance)); - assert_eq!(result.pending_auto_tier, Some(AutoTier::Intelligence)); + assert_eq!(result.pending_auto_tier, Some(AutoTier::Fast)); assert_eq!(result.activating_auto_tier, None); + assert_eq!(result.superseded_auto_tier, Some(AutoTier::Efficiency)); } #[test] diff --git a/rust/tests/e2e/auto_tier.rs b/rust/tests/e2e/auto_tier.rs index 85c70dd460..baaa1698c3 100644 --- a/rust/tests/e2e/auto_tier.rs +++ b/rust/tests/e2e/auto_tier.rs @@ -1,9 +1,20 @@ -use github_copilot_sdk::SetModelOptions; -use github_copilot_sdk::rpc::ModelSwitchAutoTierStatus; +use std::sync::Arc; +use std::time::Duration; + +use github_copilot_sdk::handler::ApproveAllHandler; +use github_copilot_sdk::rpc::{EventLogReadRequest, ModelSwitchAutoTierStatus}; use github_copilot_sdk::session::Session; -use github_copilot_sdk::session_events::AutoTier; +use github_copilot_sdk::session_events::{ + AutoTier, AutoTierSwitchFailureReason, SessionAutoTierSwitchFailedData, SessionEventType, + SessionModelChangeData, +}; +use github_copilot_sdk::{ + CapiSessionOptions, MessageOptions, ResumeSessionConfig, SessionConfig, SessionId, + SetModelOptions, +}; +use serde_json::json; -use super::support::with_dedicated_e2e_context; +use super::support::{DEFAULT_TEST_TOKEN, wait_for_event, with_dedicated_e2e_context}; const MODEL_ID: &str = "auto"; @@ -24,6 +35,32 @@ async fn pending_auto_tier(session: &Session) -> Option { .pending_auto_tier } +fn auto_session_config(tier: Option) -> SessionConfig { + let capi = CapiSessionOptions::new().with_enable_web_socket_responses(false); + let capi = match tier { + Some(tier) => capi.with_auto_tier(tier), + None => capi, + }; + SessionConfig::default() + .with_model(MODEL_ID) + .with_capi(capi) + .with_permission_handler(Arc::new(ApproveAllHandler)) + .with_github_token(DEFAULT_TEST_TOKEN) +} + +fn auto_resume_config(session_id: SessionId) -> ResumeSessionConfig { + ResumeSessionConfig::new(session_id) + .with_permission_handler(Arc::new(ApproveAllHandler)) + .with_github_token(DEFAULT_TEST_TOKEN) +} + +async fn send_prompt(session: &Session, prompt: &str) { + session + .send_and_wait(MessageOptions::new(prompt).with_wait_timeout(Duration::from_secs(120))) + .await + .expect("send prompt"); +} + #[tokio::test] async fn should_stage_and_reset_auto_tier_preference() { with_dedicated_e2e_context( @@ -53,12 +90,24 @@ async fn should_stage_and_reset_auto_tier_preference() { // A second request replaces the first and reports the one it displaced. let superseded = session - .set_auto_tier(Some(AutoTier::Intelligence)) + .set_auto_tier(Some(AutoTier::Fast)) .await - .expect("stage intelligence"); + .expect("stage fast"); assert_eq!(superseded.status, ModelSwitchAutoTierStatus::Pending); - assert_eq!(superseded.pending_auto_tier, Some(AutoTier::Intelligence)); + assert_eq!(superseded.pending_auto_tier, Some(AutoTier::Fast)); assert_eq!(superseded.superseded_auto_tier, Some(AutoTier::Efficiency)); + assert_eq!(pending_auto_tier(&session).await, Some(AutoTier::Fast)); + + let replaced_fast = session + .set_auto_tier(Some(AutoTier::Intelligence)) + .await + .expect("stage intelligence"); + assert_eq!(replaced_fast.status, ModelSwitchAutoTierStatus::Pending); + assert_eq!( + replaced_fast.pending_auto_tier, + Some(AutoTier::Intelligence) + ); + assert_eq!(replaced_fast.superseded_auto_tier, Some(AutoTier::Fast)); assert_eq!( pending_auto_tier(&session).await, Some(AutoTier::Intelligence) @@ -111,14 +160,11 @@ async fn should_preserve_auto_tier_when_set_model_omits_it() { session .set_model( MODEL_ID, - Some(SetModelOptions::default().with_auto_tier(AutoTier::Intelligence)), + Some(SetModelOptions::default().with_auto_tier(AutoTier::Fast)), ) .await .expect("set model with a tier"); - assert_eq!( - pending_auto_tier(&session).await, - Some(AutoTier::Intelligence) - ); + assert_eq!(pending_auto_tier(&session).await, Some(AutoTier::Fast)); // Requesting a reset clears it. Omission, a tier, and a reset are three // distinct outcomes, which `AutoTierPreference` makes explicit. @@ -138,3 +184,311 @@ async fn should_preserve_auto_tier_when_set_model_omits_it() { ) .await; } + +#[tokio::test] +async fn should_restore_and_override_fast_auto_tier_on_cold_resume() { + with_dedicated_e2e_context( + "auto_tier", + "should_restore_and_override_fast_auto_tier_on_cold_resume", + |ctx| { + Box::pin(async move { + ctx.set_default_copilot_user(); + let client = ctx.start_client().await; + let fast_session = client + .create_session(auto_session_config(Some(AutoTier::Fast))) + .await + .expect("create fast session"); + let tierless_session = client + .create_session(auto_session_config(None)) + .await + .expect("create tierless session"); + let fast_session_id = fast_session.id().clone(); + let tierless_session_id = tierless_session.id().clone(); + + send_prompt( + &fast_session, + "Reply with exactly AUTO_TIER_COLD_RESUME_READY.", + ) + .await; + send_prompt( + &tierless_session, + "Reply with exactly AUTO_TIER_TIERLESS_READY.", + ) + .await; + + let fast_current = fast_session + .rpc() + .model() + .get_current() + .await + .expect("get fast current model"); + assert_eq!(fast_current.auto_tier, Some(AutoTier::Fast)); + let tierless_current = tierless_session + .rpc() + .model() + .get_current() + .await + .expect("get tierless current model"); + assert_eq!(tierless_current.auto_tier, None); + + fast_session + .disconnect() + .await + .expect("disconnect fast session"); + tierless_session + .disconnect() + .await + .expect("disconnect tierless session"); + client.stop().await.expect("stop initial client"); + + let restored_client = ctx.start_client().await; + let restored_fast = restored_client + .resume_session(auto_resume_config(fast_session_id.clone())) + .await + .expect("resume fast session"); + let restored_tierless = restored_client + .resume_session(auto_resume_config(tierless_session_id)) + .await + .expect("resume tierless session"); + let restored_fast_current = restored_fast + .rpc() + .model() + .get_current() + .await + .expect("get restored fast current model"); + assert_eq!(restored_fast_current.auto_tier, Some(AutoTier::Fast)); + let restored_tierless_current = restored_tierless + .rpc() + .model() + .get_current() + .await + .expect("get restored tierless current model"); + assert_eq!(restored_tierless_current.auto_tier, None); + + restored_fast + .disconnect() + .await + .expect("disconnect restored fast session"); + restored_tierless + .disconnect() + .await + .expect("disconnect restored tierless session"); + restored_client.stop().await.expect("stop restored client"); + + let override_client = ctx.start_client().await; + let overridden = override_client + .resume_session( + auto_resume_config(fast_session_id) + .with_model(MODEL_ID) + .with_capi( + CapiSessionOptions::new() + .with_auto_tier(AutoTier::Balance) + .with_enable_web_socket_responses(false), + ), + ) + .await + .expect("resume session with balance override"); + let overridden_current = overridden + .rpc() + .model() + .get_current() + .await + .expect("get overridden current model"); + assert_eq!(overridden_current.auto_tier, Some(AutoTier::Balance)); + + overridden + .disconnect() + .await + .expect("disconnect overridden session"); + override_client.stop().await.expect("stop override client"); + }) + }, + ) + .await; +} + +#[tokio::test] +async fn should_commit_fast_auto_tier_after_successful_turn() { + with_dedicated_e2e_context( + "auto_tier", + "should_commit_fast_auto_tier_after_successful_turn", + |ctx| { + Box::pin(async move { + ctx.set_default_copilot_user(); + let client = ctx.start_client().await; + let session = client + .create_session(auto_session_config(Some(AutoTier::Efficiency))) + .await + .expect("create session"); + + let model_change = wait_for_event( + session.subscribe(), + "Fast Auto tier model change", + |event| { + event.parsed_type() == SessionEventType::SessionModelChange + && event + .typed_data::() + .is_some_and(|data| data.auto_tier == Some(AutoTier::Fast)) + }, + ); + + let staged = session + .set_auto_tier(Some(AutoTier::Fast)) + .await + .expect("stage fast"); + assert_eq!(staged.status, ModelSwitchAutoTierStatus::Pending); + assert_eq!(staged.effective_auto_tier, Some(AutoTier::Efficiency)); + assert_eq!(staged.pending_auto_tier, Some(AutoTier::Fast)); + + let before_turn = session + .rpc() + .model() + .get_current() + .await + .expect("get current model before turn"); + assert_eq!(before_turn.auto_tier, Some(AutoTier::Efficiency)); + assert_eq!(before_turn.pending_auto_tier, Some(AutoTier::Fast)); + + send_prompt(&session, "Reply with exactly AUTO_TIER_FAST_COMMITTED.").await; + + let model_change = model_change.await; + let data = model_change + .typed_data::() + .expect("typed model change data"); + assert_eq!(data.previous_model.as_deref(), Some(MODEL_ID)); + assert_eq!(data.new_model, MODEL_ID); + assert_eq!(data.previous_auto_tier, Some(AutoTier::Efficiency)); + assert_eq!(data.auto_tier, Some(AutoTier::Fast)); + + let committed = session + .rpc() + .model() + .get_current() + .await + .expect("get current model after turn"); + assert_eq!(committed.auto_tier, Some(AutoTier::Fast)); + assert_eq!(committed.pending_auto_tier, None); + assert_eq!(committed.activating_auto_tier, None); + + session.disconnect().await.expect("disconnect session"); + client.stop().await.expect("stop client"); + }) + }, + ) + .await; +} + +#[tokio::test] +async fn should_preserve_effective_tier_when_fast_activation_fails() { + with_dedicated_e2e_context( + "auto_tier", + "should_preserve_effective_tier_when_fast_activation_fails", + |ctx| { + Box::pin(async move { + ctx.set_default_copilot_user(); + let client = ctx.start_client().await; + let session = client + .create_session(auto_session_config(Some(AutoTier::Efficiency))) + .await + .expect("create session"); + let session_id = session.id().clone(); + send_prompt(&session, "Reply with exactly AUTO_TIER_INITIAL_READY.").await; + + let failure = wait_for_event( + session.subscribe(), + "Fast Auto tier activation failure", + |event| event.parsed_type() == SessionEventType::SessionAutoTierSwitchFailed, + ); + let mut model_changes = session.subscribe(); + + let staged = session + .set_auto_tier(Some(AutoTier::Fast)) + .await + .expect("stage fast"); + assert_eq!(staged.status, ModelSwitchAutoTierStatus::Pending); + assert_eq!(staged.effective_auto_tier, Some(AutoTier::Efficiency)); + assert_eq!(staged.pending_auto_tier, Some(AutoTier::Fast)); + + send_prompt(&session, "Reply with exactly AUTO_TIER_FAILURE_RECOVERED.").await; + + let failure = failure.await; + assert_eq!(failure.ephemeral, Some(true)); + let data = failure + .typed_data::() + .expect("typed Auto tier failure data"); + assert_eq!(data.effective_auto_tier, Some(AutoTier::Efficiency)); + assert_eq!(data.requested_auto_tier, Some(AutoTier::Fast)); + assert_eq!(data.reason, AutoTierSwitchFailureReason::RequestFailed); + + while let Ok(Ok(event)) = + tokio::time::timeout(Duration::from_millis(25), model_changes.recv()).await + { + assert!( + event.parsed_type() != SessionEventType::SessionModelChange + || !event + .typed_data::() + .is_some_and(|data| data.auto_tier == Some(AutoTier::Fast)), + "Fast tier committed after failed activation" + ); + } + + let current = session + .rpc() + .model() + .get_current() + .await + .expect("get current model after failure"); + assert_eq!(current.auto_tier, Some(AutoTier::Efficiency)); + assert_eq!(current.pending_auto_tier, None); + assert_eq!(current.activating_auto_tier, None); + + session.disconnect().await.expect("disconnect session"); + client.stop().await.expect("stop initial client"); + + let resumed_client = ctx.start_client().await; + let resumed = resumed_client + .resume_session(auto_resume_config(session_id)) + .await + .expect("resume session"); + let resumed_current = resumed + .rpc() + .model() + .get_current() + .await + .expect("get current model after resume"); + assert_eq!(resumed_current.auto_tier, Some(AutoTier::Efficiency)); + assert_eq!(resumed_current.pending_auto_tier, None); + assert_eq!(resumed_current.activating_auto_tier, None); + + let persisted = resumed + .rpc() + .event_log() + .read(EventLogReadRequest { + agent_ids: None, + agent_scope: None, + cursor: None, + direction: None, + include_ephemeral: Some(false), + max: Some(100), + types: Some(json!("*")), + wait_ms: Some(0), + }) + .await + .expect("read event log"); + assert!( + persisted.events.iter().all(|event| { + event.parsed_type() != SessionEventType::SessionAutoTierSwitchFailed + }), + "ephemeral failure event was replayed after cold resume" + ); + + resumed + .disconnect() + .await + .expect("disconnect resumed session"); + resumed_client.stop().await.expect("stop resumed client"); + }) + }, + ) + .await; +} diff --git a/test/harness/replayingCapiProxy.test.ts b/test/harness/replayingCapiProxy.test.ts index 062fe89ae9..80bcd6ed24 100644 --- a/test/harness/replayingCapiProxy.test.ts +++ b/test/harness/replayingCapiProxy.test.ts @@ -1698,5 +1698,55 @@ Always include PINEAPPLE_COCONUT_42. await proxy.stop(); } }); + + test("returns cached Auto responses in order", async () => { + const cachePath = path.join(tempDir, "cache.yaml"); + const autoResponses = [ + { + body: { + session_token: "first-token", + selected_model: { id: "test-model" }, + }, + }, + { + statusCode: 500, + body: { + session_token: "unused-token", + selected_model: { id: "unused-model" }, + }, + }, + ]; + await writeFile( + cachePath, + yaml.stringify({ + models: ["test-model"], + autoResponses, + conversations: [], + } satisfies NormalizedData), + ); + + const proxy = new ReplayingCapiProxy( + "http://localhost:9999", + cachePath, + workDir, + ); + const proxyUrl = await proxy.start(); + + try { + const success = await makeRequest(proxyUrl, "/auto", { + body: { prompt: "first" }, + }); + expect(success.status).toBe(200); + expect(JSON.parse(success.body)).toEqual(autoResponses[0].body); + + const failure = await makeRequest(proxyUrl, "/auto", { + body: { prompt: "second" }, + }); + expect(failure.status).toBe(500); + expect(JSON.parse(failure.body)).toEqual(autoResponses[1].body); + } finally { + await proxy.stop(); + } + }); }); }); diff --git a/test/harness/replayingCapiProxy.ts b/test/harness/replayingCapiProxy.ts index 9bcafcdad4..4ecbcdc52d 100644 --- a/test/harness/replayingCapiProxy.ts +++ b/test/harness/replayingCapiProxy.ts @@ -165,6 +165,7 @@ export class ReplayingCapiProxy extends CapturingHttpProxy { workDir, testInfo, backend: "capi", + autoResponseIndex: 0, toolResultNormalizers: [...this.defaultToolResultNormalizers], }; } @@ -199,6 +200,7 @@ export class ReplayingCapiProxy extends CapturingHttpProxy { workDir: config.workDir, testInfo: config.testInfo, backend: parseReplayBackend(config.backend), + autoResponseIndex: 0, toolResultNormalizers: [...this.defaultToolResultNormalizers], }; @@ -431,6 +433,29 @@ export class ReplayingCapiProxy extends CapturingHttpProxy { return; } + // Deterministic Auto-tier fixture responses for CAPI's `/auto` + // routing endpoint. Consumed one per call in fixture order so + // Auto-tier lifecycle scenarios (cold resume, successful + // activation, failed activation) can control what model `/auto` + // selects and, via `statusCode`, simulate an `/auto` failure. + if ( + options.requestOptions.path === "/auto" && + options.requestOptions.method === "POST" + ) { + const response = + state.storedData?.autoResponses?.[state.autoResponseIndex]; + if (response) { + state.autoResponseIndex++; + options.onResponseStart(response.statusCode ?? 200, { + "content-type": "application/json", + ...commonResponseHeaders, + }); + options.onData(Buffer.from(JSON.stringify(response.body))); + options.onResponseEnd(); + return; + } + } + // Keep GitHub MCP tests hermetic while still capturing the request at // the CAPI proxy. The tests only need a successful transport handshake; // no fake tools are exposed. @@ -698,6 +723,9 @@ async function writeCapturesToDisk( ]), ]; } + if (state.storedData?.autoResponses?.length) { + data.autoResponses = state.storedData.autoResponses; + } if (data.conversations.length > 0) { let yamlText = yaml.stringify(data, { lineWidth: 120 }); @@ -2093,6 +2121,7 @@ type ReplayingCapiProxyState = { testInfo?: { file: string; line?: number }; backend: ReplayBackend; storedData?: NormalizedData | undefined; + autoResponseIndex: number; toolResultNormalizers: ToolResultNormalizer[]; }; @@ -2130,6 +2159,35 @@ export interface NormalizedData { models: string[]; errors?: NormalizedErrorResponse[]; conversations: NormalizedConversation[]; + /** + * Ordered fixture responses for CAPI's `POST /auto` routing endpoint, + * consumed one per call in array order. Used by deterministic Auto-tier + * lifecycle scenarios (cold resume, successful activation, failed + * activation) that need to control what model `/auto` selects without + * depending on live, non-deterministic Auto routing. + */ + autoResponses?: AutoResponseStub[]; +} + +/** + * A single deterministic `/auto` fixture response. `statusCode` defaults to + * 200; a non-2xx value (e.g. 500) simulates an `/auto` request failure so + * tests can exercise failed Auto-tier activation without a real routing + * error. + */ +export interface AutoResponseStub { + statusCode?: number; + body: { + session_token: string; + selected_model: { + id: string; + name?: string; + capabilities?: { + supports: Record; + limits: Record; + }; + }; + }; } function sortJsonKeys(obj: unknown): unknown { diff --git a/test/snapshots/auto_tier/should_commit_fast_auto_tier_after_successful_turn.yaml b/test/snapshots/auto_tier/should_commit_fast_auto_tier_after_successful_turn.yaml new file mode 100644 index 0000000000..c4ea26d401 --- /dev/null +++ b/test/snapshots/auto_tier/should_commit_fast_auto_tier_after_successful_turn.yaml @@ -0,0 +1,21 @@ +models: + - claude-sonnet-5 +conversations: + - messages: + - role: system + content: ${system} + - role: user + content: Reply with exactly AUTO_TIER_FAST_COMMITTED. + - role: assistant + content: AUTO_TIER_FAST_COMMITTED +autoResponses: + - body: + session_token: auto-tier-replay-token-1 + selected_model: + id: claude-sonnet-5 + name: Claude Sonnet 5 + capabilities: + supports: + vision: false + limits: + max_context_window_tokens: 128000 diff --git a/test/snapshots/auto_tier/should_preserve_effective_tier_when_fast_activation_fails.yaml b/test/snapshots/auto_tier/should_preserve_effective_tier_when_fast_activation_fails.yaml new file mode 100644 index 0000000000..44adec410d --- /dev/null +++ b/test/snapshots/auto_tier/should_preserve_effective_tier_when_fast_activation_fails.yaml @@ -0,0 +1,30 @@ +models: + - claude-sonnet-5 +conversations: + - messages: + - role: system + content: ${system} + - role: user + content: Reply with exactly AUTO_TIER_INITIAL_READY. + - role: assistant + content: AUTO_TIER_INITIAL_READY + - role: user + content: Reply with exactly AUTO_TIER_FAILURE_RECOVERED. + - role: assistant + content: AUTO_TIER_FAILURE_RECOVERED +autoResponses: + - body: + session_token: auto-tier-replay-token-1 + selected_model: + id: claude-sonnet-5 + name: Claude Sonnet 5 + capabilities: + supports: + vision: false + limits: + max_context_window_tokens: 128000 + - statusCode: 500 + body: + session_token: unused-auto-tier-token + selected_model: + id: unused-model diff --git a/test/snapshots/auto_tier/should_restore_and_override_fast_auto_tier_on_cold_resume.yaml b/test/snapshots/auto_tier/should_restore_and_override_fast_auto_tier_on_cold_resume.yaml new file mode 100644 index 0000000000..ca831aaa51 --- /dev/null +++ b/test/snapshots/auto_tier/should_restore_and_override_fast_auto_tier_on_cold_resume.yaml @@ -0,0 +1,38 @@ +models: + - claude-sonnet-5 +conversations: + - messages: + - role: system + content: ${system} + - role: user + content: Reply with exactly AUTO_TIER_COLD_RESUME_READY. + - role: assistant + content: AUTO_TIER_COLD_RESUME_READY + - messages: + - role: system + content: ${system} + - role: user + content: Reply with exactly AUTO_TIER_TIERLESS_READY. + - role: assistant + content: AUTO_TIER_TIERLESS_READY +autoResponses: + - body: + session_token: auto-tier-replay-token-1 + selected_model: + id: claude-sonnet-5 + name: Claude Sonnet 5 + capabilities: + supports: + vision: false + limits: + max_context_window_tokens: 128000 + - body: + session_token: auto-tier-replay-token-2 + selected_model: + id: claude-sonnet-5 + name: Claude Sonnet 5 + capabilities: + supports: + vision: false + limits: + max_context_window_tokens: 128000 From 4914720f6c45dec54148774aa860957d3cb54994 Mon Sep 17 00:00:00 2001 From: Andy Feller Date: Tue, 15 Sep 2026 15:26:07 -0400 Subject: [PATCH 2/4] Fix Fast Auto-tier test backend classification and formatting Restrict routing-dependent lifecycle tests to CAPI while retaining direct-provider coverage for the other Auto-tier tests. Apply Python formatting and documentation punctuation fixes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- docs/features/session-persistence.md | 2 +- dotnet/test/E2E/AutoTierE2ETests.cs | 2 ++ python/e2e/test_auto_tier_e2e.py | 4 +--- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/features/session-persistence.md b/docs/features/session-persistence.md index c99f71c53f..87c2542786 100644 --- a/docs/features/session-persistence.md +++ b/docs/features/session-persistence.md @@ -259,7 +259,7 @@ When resuming a session, you can optionally reconfigure many settings. This is u With `model: "auto"`, the optional `capi.autoTier` setting selects an Auto routing preference: `efficiency`, `balance`, `intelligence`, or `fast`. In Python, use `capi={"auto_tier": "balance"}`. This requires Copilot CLI `1.0.82-1` or later with V2 Auto routing; V1 Auto requests are unchanged. `fast` additionally requires Copilot CLI `1.0.84-0` or later. -`fast` is an integrator-only latency preset, not a first-party GitHub Copilot product preference. The SDK does not decide Fast eligibility, inspect client identity, choose it as a default, or fall back to another tier when a runtime does not support it — an older runtime returns its native error unchanged. +`fast` is an integrator-only latency preset, not a first-party GitHub Copilot product preference. The SDK does not decide Fast eligibility, inspect client identity, choose it as a default, or fall back to another tier when a runtime does not support it—an older runtime returns its native error unchanged. The runtime persists the selected tier, so applications do not need to resend it on every resume: diff --git a/dotnet/test/E2E/AutoTierE2ETests.cs b/dotnet/test/E2E/AutoTierE2ETests.cs index c04a765038..19b2b7f9bf 100644 --- a/dotnet/test/E2E/AutoTierE2ETests.cs +++ b/dotnet/test/E2E/AutoTierE2ETests.cs @@ -183,6 +183,7 @@ await tierlessSession.SendAndWaitAsync(new MessageOptions } [Fact] + [Trait(E2ETestTraits.Backend, E2ETestTraits.CapiOnly)] public async Task Should_Commit_Fast_Auto_Tier_After_Successful_Turn() { await using var client = Ctx.CreateClient(); @@ -228,6 +229,7 @@ await session.SendAndWaitAsync(new MessageOptions } [Fact] + [Trait(E2ETestTraits.Backend, E2ETestTraits.CapiOnly)] public async Task Should_Preserve_Effective_Tier_When_Fast_Activation_Fails() { string sessionId; diff --git a/python/e2e/test_auto_tier_e2e.py b/python/e2e/test_auto_tier_e2e.py index 3a9223cce6..d5fb12cc5a 100644 --- a/python/e2e/test_auto_tier_e2e.py +++ b/python/e2e/test_auto_tier_e2e.py @@ -208,9 +208,7 @@ def on_event(event): fast_committed |= event.data.auto_tier == AutoTier.FAST unsubscribe = session.on(on_event) - failure_task = get_next_event_of_type( - session, "session.auto_tier_switch_failed" - ) + failure_task = get_next_event_of_type(session, "session.auto_tier_switch_failed") staged = await session.set_auto_tier("fast") assert staged.status == ModelSwitchAutoTierStatus.PENDING From f2f9beb97f71480b5937b786b1b3578b06fb8ea9 Mon Sep 17 00:00:00 2001 From: Andy Feller Date: Tue, 15 Sep 2026 22:18:43 -0400 Subject: [PATCH 3/4] Remove CLI version requirements from Auto-tier documentation Address maintainer feedback consistently across feature documentation, SDK READMEs, the changelog, and API comments while preserving routing and runtime-error guidance. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- CHANGELOG.md | 2 +- docs/features/session-persistence.md | 4 ++-- dotnet/README.md | 2 +- go/README.md | 2 +- java/README.md | 3 +-- java/sdk/src/main/java/com/github/copilot/rpc/AutoTier.java | 2 +- nodejs/README.md | 2 +- python/README.md | 2 +- rust/README.md | 3 +-- rust/src/types.rs | 3 +-- 10 files changed, 11 insertions(+), 14 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index f4cf3a90a4..5c0e7a6579 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,7 +9,7 @@ See [GitHub Releases](https://github.com/github/copilot-sdk/releases) for the fu ### Feature: complete Fast Auto tier support -All six SDKs now expose the canonical `fast` Auto routing tier across session creation, cold resume, resident switching, authoritative state reads, and terminal lifecycle events. Fast is an integrator-only latency preset and requires Copilot CLI `1.0.84-0` or later. +All six SDKs now expose the canonical `fast` Auto routing tier across session creation, cold resume, resident switching, authoritative state reads, and terminal lifecycle events. Fast is an integrator-only latency preset. The SDKs preserve the runtime's behavior: omitted tiers use the runtime default or persisted value, explicit resume tiers override persisted state, resident switches remain pending until a successful Auto turn commits them, and failed activation leaves the incumbent effective tier unchanged. SDKs do not decide Fast eligibility, downgrade unsupported requests, or hide errors from externally supplied older runtimes. diff --git a/docs/features/session-persistence.md b/docs/features/session-persistence.md index 87c2542786..69fda41fba 100644 --- a/docs/features/session-persistence.md +++ b/docs/features/session-persistence.md @@ -257,7 +257,7 @@ When resuming a session, you can optionally reconfigure many settings. This is u ### Auto tier persistence -With `model: "auto"`, the optional `capi.autoTier` setting selects an Auto routing preference: `efficiency`, `balance`, `intelligence`, or `fast`. In Python, use `capi={"auto_tier": "balance"}`. This requires Copilot CLI `1.0.82-1` or later with V2 Auto routing; V1 Auto requests are unchanged. `fast` additionally requires Copilot CLI `1.0.84-0` or later. +With `model: "auto"`, the optional `capi.autoTier` setting selects an Auto routing preference: `efficiency`, `balance`, `intelligence`, or `fast`. In Python, use `capi={"auto_tier": "balance"}`. This setting applies to V2 Auto routing; V1 Auto requests are unchanged. `fast` is an integrator-only latency preset, not a first-party GitHub Copilot product preference. The SDK does not decide Fast eligibility, inspect client identity, choose it as a default, or fall back to another tier when a runtime does not support it—an older runtime returns its native error unchanged. @@ -274,7 +274,7 @@ The `session.start` and `session.resume` events expose the selected tier in thei ### Changing the Auto tier during a session -Call `setAutoTier` to change the routing preference on a live session without changing the selected model. Pass `null` (Python `None`, Go `nil`) to return to the provider's default Auto routing. This requires Copilot CLI `1.0.83-4` or later, which is newer than the `1.0.82-1` needed to select a tier when creating or resuming a session. +Call `setAutoTier` to change the routing preference on a live session without changing the selected model. Pass `null` (Python `None`, Go `nil`) to return to the provider's default Auto routing. ```typescript const result = await session.setAutoTier("intelligence"); diff --git a/dotnet/README.md b/dotnet/README.md index a952b2aae2..f8b7e44f06 100644 --- a/dotnet/README.md +++ b/dotnet/README.md @@ -317,7 +317,7 @@ await session2.DisposeAsync(); ## Auto routing tiers -The canonical values are `AutoTier.Efficiency`, `AutoTier.Balance`, `AutoTier.Intelligence`, and `AutoTier.Fast`, which send `efficiency`, `balance`, `intelligence`, and `fast` on the wire. Fast is an integrator-only latency preset, not a fourth first-party GitHub Copilot preference, and requires Copilot CLI `1.0.84-0` or later. The SDK forwards the requested value without deciding eligibility or inspecting client identity. An externally supplied older runtime returns its native runtime or JSON-RPC error; the SDK does not downgrade or silently ignore the request. +The canonical values are `AutoTier.Efficiency`, `AutoTier.Balance`, `AutoTier.Intelligence`, and `AutoTier.Fast`, which send `efficiency`, `balance`, `intelligence`, and `fast` on the wire. Fast is an integrator-only latency preset, not a fourth first-party GitHub Copilot preference. The SDK forwards the requested value without deciding eligibility or inspecting client identity. An externally supplied older runtime returns its native runtime or JSON-RPC error; the SDK does not downgrade or silently ignore the request. Omitting the tier on create uses the runtime default rather than Balance. A cold resume restores the persisted tier unless the resume request supplies an explicit override. diff --git a/go/README.md b/go/README.md index 441689a901..cbbb84b78c 100644 --- a/go/README.md +++ b/go/README.md @@ -361,7 +361,7 @@ Unknown section IDs are handled gracefully: content from `replace`/`append`/`pre ## Auto routing tiers -The canonical values are `AutoTierEfficiency`, `AutoTierBalance`, `AutoTierIntelligence`, and `AutoTierFast`, which send `efficiency`, `balance`, `intelligence`, and `fast` on the wire. Fast is an integrator-only latency preset, not a fourth first-party GitHub Copilot preference, and requires Copilot CLI `1.0.84-0` or later. The SDK forwards the requested value without deciding eligibility or inspecting client identity. An externally supplied older runtime returns its native runtime or JSON-RPC error; the SDK does not downgrade or silently ignore the request. +The canonical values are `AutoTierEfficiency`, `AutoTierBalance`, `AutoTierIntelligence`, and `AutoTierFast`, which send `efficiency`, `balance`, `intelligence`, and `fast` on the wire. Fast is an integrator-only latency preset, not a fourth first-party GitHub Copilot preference. The SDK forwards the requested value without deciding eligibility or inspecting client identity. An externally supplied older runtime returns its native runtime or JSON-RPC error; the SDK does not downgrade or silently ignore the request. Omitting the tier on create uses the runtime default rather than Balance. A cold resume restores the persisted tier unless the resume request supplies an explicit override. diff --git a/java/README.md b/java/README.md index 31da76ea2b..373a48c1f0 100644 --- a/java/README.md +++ b/java/README.md @@ -371,8 +371,7 @@ For design context and decision rationale, see [ADR-006](docs/adr/adr-006-tool-d Use `CapiSessionOptions.setAutoTier(...)` to select `AutoTier.EFFICIENCY`, `AutoTier.BALANCE`, `AutoTier.INTELLIGENCE`, or `AutoTier.FAST`. This option is meaningful only with model `auto` (Auto mode V2). -It requires a runtime version that supports `capi.autoTier`; `AutoTier.FAST` -additionally requires Copilot CLI `1.0.84-0` or later. +It requires a runtime version that supports `capi.autoTier`. `AutoTier.FAST` is an integrator-only latency preset, not a first-party GitHub Copilot product preference — the SDK does not decide Fast eligibility or apply it implicitly. diff --git a/java/sdk/src/main/java/com/github/copilot/rpc/AutoTier.java b/java/sdk/src/main/java/com/github/copilot/rpc/AutoTier.java index 5a9864682f..092dc73026 100644 --- a/java/sdk/src/main/java/com/github/copilot/rpc/AutoTier.java +++ b/java/sdk/src/main/java/com/github/copilot/rpc/AutoTier.java @@ -25,7 +25,7 @@ public enum AutoTier { /** * Integrator-only preset that optimizes for latency. Not a first-party GitHub - * Copilot product preference; requires Copilot CLI 1.0.84-0 or later. + * Copilot product preference. */ FAST("fast"); diff --git a/nodejs/README.md b/nodejs/README.md index b454c35a69..a6e577be93 100644 --- a/nodejs/README.md +++ b/nodejs/README.md @@ -151,7 +151,7 @@ Create a new conversation session. - `sessionId?: string` - Custom session ID. - `model?: string` - Model to use ("gpt-5", "claude-sonnet-4.5", etc.). **Required when using custom provider.** -- `capi?: CapiSessionOptions` - Copilot API options. With `model: "auto"`, set `autoTier` to `"efficiency"`, `"balance"`, `"intelligence"`, or `"fast"` to choose a routing preference. `"fast"` is an integrator-only latency preset, not a first-party GitHub Copilot product preference, and requires Copilot CLI `1.0.84-0` or later. Requires a runtime with Auto tier support and V2 Auto routing. Omission preserves default behavior. See [Auto tier persistence](../docs/features/session-persistence.md#auto-tier-persistence) for resume semantics. +- `capi?: CapiSessionOptions` - Copilot API options. With `model: "auto"`, set `autoTier` to `"efficiency"`, `"balance"`, `"intelligence"`, or `"fast"` to choose a routing preference. `"fast"` is an integrator-only latency preset, not a first-party GitHub Copilot product preference. Requires a runtime with Auto tier support and V2 Auto routing. Omission preserves default behavior. See [Auto tier persistence](../docs/features/session-persistence.md#auto-tier-persistence) for resume semantics. - `reasoningEffort?: "low" | "medium" | "high" | "xhigh" | "max"` - Reasoning effort level for models that support it. Use `listModels()` to check which models support this option. - `tools?: Tool[]` - Custom tools exposed to the CLI. Tools without `handler` are declaration-only and must be resolved via pending tool-call RPCs. - `systemMessage?: SystemMessageConfig` - System message customization (see below) diff --git a/python/README.md b/python/README.md index 603907b12c..71a6e8ae78 100644 --- a/python/README.md +++ b/python/README.md @@ -306,7 +306,7 @@ finally: These are passed as keyword arguments to `create_session()`: - `model` (str): Model to use ("gpt-5", "claude-sonnet-4.5", etc.). **Required when using custom provider.** -- `capi` (CapiSessionOptions): Copilot API options. With `model="auto"`, set `auto_tier` to `"efficiency"`, `"balance"`, `"intelligence"`, or `"fast"` to choose a routing preference. `"fast"` is an integrator-only latency preset, not a first-party GitHub Copilot product preference, and requires Copilot CLI `1.0.84-0` or later. Requires a runtime with Auto tier support and V2 Auto routing. Omission preserves default behavior. See [Auto tier persistence](../docs/features/session-persistence.md#auto-tier-persistence) for resume semantics. +- `capi` (CapiSessionOptions): Copilot API options. With `model="auto"`, set `auto_tier` to `"efficiency"`, `"balance"`, `"intelligence"`, or `"fast"` to choose a routing preference. `"fast"` is an integrator-only latency preset, not a first-party GitHub Copilot product preference. Requires a runtime with Auto tier support and V2 Auto routing. Omission preserves default behavior. See [Auto tier persistence](../docs/features/session-persistence.md#auto-tier-persistence) for resume semantics. - `reasoning_effort` (str): Reasoning effort level for models that support it ("low", "medium", "high", "xhigh", "max"). Use `list_models()` to check which models support this option. - `session_id` (str): Custom session ID - `tools` (list): Custom tools exposed to the CLI. Tools with `handler=None` are declaration-only and must be resolved via pending tool-call RPCs. diff --git a/rust/README.md b/rust/README.md index a504b3f935..11d9637b22 100644 --- a/rust/README.md +++ b/rust/README.md @@ -367,8 +367,7 @@ next credential-consuming operation; there is no background refresh timer. Use `CapiSessionOptions::with_auto_tier` to select `AutoTier::Efficiency`, `AutoTier::Balance`, `AutoTier::Intelligence`, or `AutoTier::Fast`. This option is meaningful only with model `auto` (Auto mode V2). -It requires a runtime version that supports `capi.autoTier`; `AutoTier::Fast` -additionally requires Copilot CLI `1.0.84-0` or later. +It requires a runtime version that supports `capi.autoTier`. `AutoTier::Fast` is an integrator-only latency preset, not a first-party GitHub Copilot product preference — the SDK does not decide Fast eligibility or apply it implicitly. diff --git a/rust/src/types.rs b/rust/src/types.rs index 9b8eb8357d..6e3af9273f 100644 --- a/rust/src/types.rs +++ b/rust/src/types.rs @@ -1425,8 +1425,7 @@ impl ProviderConfig { #[non_exhaustive] pub struct CapiSessionOptions { /// Routing tier, meaningful only with model `auto` (Auto mode V2). - /// Requires a runtime version that supports `capi.autoTier`. Fast - /// requires Copilot CLI 1.0.84-0 or later. + /// Requires a runtime version that supports `capi.autoTier`. /// /// When omitted, the runtime chooses its default on create and restores /// the last committed tier on cold resume. On resident resume, a different From 5cf136c23684806e3e31271a20a9848aec95d279 Mon Sep 17 00:00:00 2001 From: Stephen Toub Date: Wed, 16 Sep 2026 00:20:24 -0400 Subject: [PATCH 4/4] Apply suggestion from @stephentoub --- CHANGELOG.md | 6 ------ 1 file changed, 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 5c0e7a6579..d71e9b8248 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,12 +7,6 @@ See [GitHub Releases](https://github.com/github/copilot-sdk/releases) for the fu ## [Unreleased] -### Feature: complete Fast Auto tier support - -All six SDKs now expose the canonical `fast` Auto routing tier across session creation, cold resume, resident switching, authoritative state reads, and terminal lifecycle events. Fast is an integrator-only latency preset. - -The SDKs preserve the runtime's behavior: omitted tiers use the runtime default or persisted value, explicit resume tiers override persisted state, resident switches remain pending until a successful Auto turn commits them, and failed activation leaves the incumbent effective tier unchanged. SDKs do not decide Fast eligibility, downgrade unsupported requests, or hide errors from externally supplied older runtimes. - ## [v1.0.13](https://github.com/github/copilot-sdk/releases/tag/v1.0.13) (2026-09-04) ### Feature: cancellation for host-owned external tools