diff --git a/docs/features/session-persistence.md b/docs/features/session-persistence.md index f2e1c7d5d9..69fda41fba 100644 --- a/docs/features/session-persistence.md +++ b/docs/features/session-persistence.md @@ -257,7 +257,9 @@ When resuming a session, you can optionally reconfigure many settings. This is u ### Auto tier persistence -With `model: "auto"`, the optional `capi.autoTier` setting selects an Auto routing preference: `efficiency`, `balance`, or `intelligence`. In Python, use `capi={"auto_tier": "balance"}`. This requires Copilot CLI `1.0.82-1` or later with V2 Auto routing; V1 Auto requests are unchanged. +With `model: "auto"`, the optional `capi.autoTier` setting selects an Auto routing preference: `efficiency`, `balance`, `intelligence`, or `fast`. In Python, use `capi={"auto_tier": "balance"}`. This setting applies to V2 Auto routing; V1 Auto requests are unchanged. + +`fast` is an integrator-only latency preset, not a first-party GitHub Copilot product preference. The SDK does not decide Fast eligibility, inspect client identity, choose it as a default, or fall back to another tier when a runtime does not support it—an older runtime returns its native error unchanged. The runtime persists the selected tier, so applications do not need to resend it on every resume: @@ -272,7 +274,7 @@ The `session.start` and `session.resume` events expose the selected tier in thei ### Changing the Auto tier during a session -Call `setAutoTier` to change the routing preference on a live session without changing the selected model. Pass `null` (Python `None`, Go `nil`) to return to the provider's default Auto routing. This requires Copilot CLI `1.0.83-4` or later, which is newer than the `1.0.82-1` needed to select a tier when creating or resuming a session. +Call `setAutoTier` to change the routing preference on a live session without changing the selected model. Pass `null` (Python `None`, Go `nil`) to return to the provider's default Auto routing. ```typescript const result = await session.setAutoTier("intelligence"); diff --git a/dotnet/README.md b/dotnet/README.md index c518a40326..f8b7e44f06 100644 --- a/dotnet/README.md +++ b/dotnet/README.md @@ -317,9 +317,13 @@ await session2.DisposeAsync(); ## Auto routing tiers +The canonical values are `AutoTier.Efficiency`, `AutoTier.Balance`, `AutoTier.Intelligence`, and `AutoTier.Fast`, which send `efficiency`, `balance`, `intelligence`, and `fast` on the wire. Fast is an integrator-only latency preset, not a fourth first-party GitHub Copilot preference. The SDK forwards the requested value without deciding eligibility or inspecting client identity. An externally supplied older runtime returns its native runtime or JSON-RPC error; the SDK does not downgrade or silently ignore the request. + +Omitting the tier on create uses the runtime default rather than Balance. A cold resume restores the persisted tier unless the resume request supplies an explicit override. + Change the Auto routing preference without changing the selected model. The runtime does not apply the preference immediately: it records the request and commits it only when a later user turn using the `auto` model successfully obtains a usable model from the provider, so a `pending` status confirms acceptance rather than effect. Only the most recent request survives. -Watch for the outcome through the `session.model_change` event on success or the ephemeral `session.auto_tier_switch_failed` event on failure. Read the authoritative committed, pending, and activating preferences at any time through the session's `model.getCurrent` RPC method. +Watch for the outcome through the `session.model_change` event on success or the ephemeral `session.auto_tier_switch_failed` event on failure. A failed activation leaves the incumbent effective tier unchanged. Read the authoritative committed, pending, and activating preferences at any time through the session's `model.getCurrent` RPC method. ```csharp var result = await session.SetAutoTierAsync(AutoTier.Intelligence); diff --git a/dotnet/test/E2E/AutoTierE2ETests.cs b/dotnet/test/E2E/AutoTierE2ETests.cs index 3758eee0f5..19b2b7f9bf 100644 --- a/dotnet/test/E2E/AutoTierE2ETests.cs +++ b/dotnet/test/E2E/AutoTierE2ETests.cs @@ -45,10 +45,16 @@ public async Task Should_Stage_And_Reset_Auto_Tier_Preference() await AssertPendingAutoTierAsync(session, AutoTier.Efficiency); // A second request replaces the first and reports the one it displaced. - var superseded = await session.SetAutoTierAsync(AutoTier.Intelligence); + var superseded = await session.SetAutoTierAsync(AutoTier.Fast); Assert.Equal(ModelSwitchAutoTierStatus.Pending, superseded.Status); - Assert.Equal(AutoTier.Intelligence, superseded.PendingAutoTier); + Assert.Equal(AutoTier.Fast, superseded.PendingAutoTier); Assert.Equal(AutoTier.Efficiency, superseded.SupersededAutoTier); + await AssertPendingAutoTierAsync(session, AutoTier.Fast); + + var replacedFast = await session.SetAutoTierAsync(AutoTier.Intelligence); + Assert.Equal(ModelSwitchAutoTierStatus.Pending, replacedFast.Status); + Assert.Equal(AutoTier.Intelligence, replacedFast.PendingAutoTier); + Assert.Equal(AutoTier.Fast, replacedFast.SupersededAutoTier); await AssertPendingAutoTierAsync(session, AutoTier.Intelligence); // A null tier returns the session to provider-default routing. The status is @@ -77,12 +83,227 @@ public async Task Should_Preserve_Auto_Tier_When_Set_Model_Omits_It() await AssertPendingAutoTierAsync(session, AutoTier.Balance); // Supplying a tier replaces it. - await session.SetModelAsync("auto", new SetModelOptions { AutoTier = AutoTier.Intelligence }); - await AssertPendingAutoTierAsync(session, AutoTier.Intelligence); + await session.SetModelAsync("auto", new SetModelOptions { AutoTier = AutoTier.Fast }); + await AssertPendingAutoTierAsync(session, AutoTier.Fast); // ResetAutoTier clears it. Omission, a value, and a reset are three distinct // outcomes, which is why a single nullable property cannot express the request. await session.SetModelAsync("auto", new SetModelOptions { ResetAutoTier = true }); await AssertPendingAutoTierAsync(session, null); } + + [Fact] + public async Task Should_Restore_And_Override_Fast_Auto_Tier_On_Cold_Resume() + { + var initialClient = Ctx.CreateClient(); + string fastSessionId; + string tierlessSessionId; + + await using (initialClient) + { + await using (var fastSession = await Ctx.CreateSessionAsync(initialClient, new SessionConfig + { + Model = "auto", + OnPermissionRequest = PermissionHandler.ApproveAll, + Capi = new CapiSessionOptions + { + AutoTier = AutoTier.Fast, + EnableWebSocketResponses = false, + }, + })) + await using (var tierlessSession = await Ctx.CreateSessionAsync(initialClient, new SessionConfig + { + Model = "auto", + OnPermissionRequest = PermissionHandler.ApproveAll, + Capi = new CapiSessionOptions { EnableWebSocketResponses = false }, + })) + { + fastSessionId = fastSession.SessionId; + tierlessSessionId = tierlessSession.SessionId; + + await fastSession.SendAndWaitAsync(new MessageOptions + { + Prompt = "Reply with exactly AUTO_TIER_COLD_RESUME_READY.", + }); + await tierlessSession.SendAndWaitAsync(new MessageOptions + { + Prompt = "Reply with exactly AUTO_TIER_TIERLESS_READY.", + }); + + var fastCurrent = await fastSession.Rpc.Model.GetCurrentAsync(); + Assert.Equal(AutoTier.Fast, fastCurrent.AutoTier); + var tierlessCurrent = await tierlessSession.Rpc.Model.GetCurrentAsync(); + Assert.Null(tierlessCurrent.AutoTier); + } + + await initialClient.StopAsync(); + } + + var restoredClient = Ctx.CreateClient(); + await using (restoredClient) + { + await using (var restoredFast = await Ctx.ResumeSessionAsync( + restoredClient, + fastSessionId, + new ResumeSessionConfig { OnPermissionRequest = PermissionHandler.ApproveAll })) + await using (var restoredTierless = await Ctx.ResumeSessionAsync( + restoredClient, + tierlessSessionId, + new ResumeSessionConfig { OnPermissionRequest = PermissionHandler.ApproveAll })) + { + var restoredFastCurrent = await restoredFast.Rpc.Model.GetCurrentAsync(); + Assert.Equal(AutoTier.Fast, restoredFastCurrent.AutoTier); + var restoredTierlessCurrent = await restoredTierless.Rpc.Model.GetCurrentAsync(); + Assert.Null(restoredTierlessCurrent.AutoTier); + } + + await restoredClient.StopAsync(); + } + + var overrideClient = Ctx.CreateClient(); + await using (overrideClient) + { + await using var overridden = await Ctx.ResumeSessionAsync( + overrideClient, + fastSessionId, + new ResumeSessionConfig + { + Model = "auto", + OnPermissionRequest = PermissionHandler.ApproveAll, + Capi = new CapiSessionOptions + { + AutoTier = AutoTier.Balance, + EnableWebSocketResponses = false, + }, + }); + + var overriddenCurrent = await overridden.Rpc.Model.GetCurrentAsync(); + Assert.Equal(AutoTier.Balance, overriddenCurrent.AutoTier); + } + } + + [Fact] + [Trait(E2ETestTraits.Backend, E2ETestTraits.CapiOnly)] + public async Task Should_Commit_Fast_Auto_Tier_After_Successful_Turn() + { + await using var client = Ctx.CreateClient(); + await using var session = await Ctx.CreateSessionAsync(client, new SessionConfig + { + Model = "auto", + OnPermissionRequest = PermissionHandler.ApproveAll, + Capi = new CapiSessionOptions + { + AutoTier = AutoTier.Efficiency, + EnableWebSocketResponses = false, + }, + }); + + var modelChangedTask = TestHelper.GetNextEventOfTypeAsync( + session, + evt => evt.Data.AutoTier == AutoTier.Fast); + + var staged = await session.SetAutoTierAsync(AutoTier.Fast); + Assert.Equal(ModelSwitchAutoTierStatus.Pending, staged.Status); + Assert.Equal(AutoTier.Efficiency, staged.EffectiveAutoTier); + Assert.Equal(AutoTier.Fast, staged.PendingAutoTier); + + var beforeTurn = await session.Rpc.Model.GetCurrentAsync(); + Assert.Equal(AutoTier.Efficiency, beforeTurn.AutoTier); + Assert.Equal(AutoTier.Fast, beforeTurn.PendingAutoTier); + + await session.SendAndWaitAsync(new MessageOptions + { + Prompt = "Reply with exactly AUTO_TIER_FAST_COMMITTED.", + }); + + var modelChanged = await modelChangedTask; + Assert.Equal("auto", modelChanged.Data.PreviousModel); + Assert.Equal("auto", modelChanged.Data.NewModel); + Assert.Equal(AutoTier.Efficiency, modelChanged.Data.PreviousAutoTier); + Assert.Equal(AutoTier.Fast, modelChanged.Data.AutoTier); + + var committed = await session.Rpc.Model.GetCurrentAsync(); + Assert.Equal(AutoTier.Fast, committed.AutoTier); + Assert.Null(committed.PendingAutoTier); + Assert.Null(committed.ActivatingAutoTier); + } + + [Fact] + [Trait(E2ETestTraits.Backend, E2ETestTraits.CapiOnly)] + public async Task Should_Preserve_Effective_Tier_When_Fast_Activation_Fails() + { + string sessionId; + var initialClient = Ctx.CreateClient(); + + await using (initialClient) + { + await using (var session = await Ctx.CreateSessionAsync(initialClient, new SessionConfig + { + Model = "auto", + OnPermissionRequest = PermissionHandler.ApproveAll, + Capi = new CapiSessionOptions + { + AutoTier = AutoTier.Efficiency, + EnableWebSocketResponses = false, + }, + })) + { + sessionId = session.SessionId; + await session.SendAndWaitAsync(new MessageOptions + { + Prompt = "Reply with exactly AUTO_TIER_INITIAL_READY.", + }); + + var failureTask = TestHelper.GetNextEventOfTypeAsync(session); + var fastCommit = new TaskCompletionSource( + TaskCreationOptions.RunContinuationsAsynchronously); + using var modelChangeSubscription = session.On(evt => + { + if (evt.Data.AutoTier == AutoTier.Fast) + { + fastCommit.TrySetResult(evt); + } + }); + + var staged = await session.SetAutoTierAsync(AutoTier.Fast); + Assert.Equal(ModelSwitchAutoTierStatus.Pending, staged.Status); + Assert.Equal(AutoTier.Efficiency, staged.EffectiveAutoTier); + Assert.Equal(AutoTier.Fast, staged.PendingAutoTier); + + await session.SendAndWaitAsync(new MessageOptions + { + Prompt = "Reply with exactly AUTO_TIER_FAILURE_RECOVERED.", + }); + + var failure = await failureTask; + Assert.True(failure.Ephemeral); + Assert.Equal(AutoTier.Efficiency, failure.Data.EffectiveAutoTier); + Assert.Equal(AutoTier.Fast, failure.Data.RequestedAutoTier); + Assert.Equal(AutoTierSwitchFailureReason.RequestFailed, failure.Data.Reason); + var noFastCommit = await Task.WhenAny(fastCommit.Task, Task.Delay(TimeSpan.FromMilliseconds(100))); + Assert.NotSame(fastCommit.Task, noFastCommit); + + var current = await session.Rpc.Model.GetCurrentAsync(); + Assert.Equal(AutoTier.Efficiency, current.AutoTier); + Assert.Null(current.PendingAutoTier); + Assert.Null(current.ActivatingAutoTier); + } + + await initialClient.StopAsync(); + } + + await using var resumedClient = Ctx.CreateClient(); + await using var resumed = await Ctx.ResumeSessionAsync( + resumedClient, + sessionId, + new ResumeSessionConfig { OnPermissionRequest = PermissionHandler.ApproveAll }); + + var resumedCurrent = await resumed.Rpc.Model.GetCurrentAsync(); + Assert.Equal(AutoTier.Efficiency, resumedCurrent.AutoTier); + Assert.Null(resumedCurrent.PendingAutoTier); + Assert.Null(resumedCurrent.ActivatingAutoTier); + + var persisted = await resumed.Rpc.EventLog.ReadAsync(max: 100, waitMs: TimeSpan.Zero); + Assert.DoesNotContain(persisted.Events, evt => evt is SessionAutoTierSwitchFailedEvent); + } } diff --git a/dotnet/test/Unit/ClientSessionLifetimeTests.cs b/dotnet/test/Unit/ClientSessionLifetimeTests.cs index 88de6abd0d..3e4973a192 100644 --- a/dotnet/test/Unit/ClientSessionLifetimeTests.cs +++ b/dotnet/test/Unit/ClientSessionLifetimeTests.cs @@ -471,9 +471,11 @@ public async Task CreateSessionAsync_Omits_CustomAgent_ReasoningEffort_When_Unse { AutoTier.Efficiency, "efficiency", null }, { AutoTier.Balance, "balance", null }, { AutoTier.Intelligence, "intelligence", null }, + { AutoTier.Fast, "fast", null }, { AutoTier.Efficiency, "efficiency", false }, { AutoTier.Balance, "balance", false }, { AutoTier.Intelligence, "intelligence", false }, + { AutoTier.Fast, "fast", false }, }; [Theory] @@ -517,6 +519,7 @@ public async Task SessionRequests_Serialize_CapiAutoTier(AutoTier tier, string e [InlineData("efficiency")] [InlineData("balance")] [InlineData("intelligence")] + [InlineData("fast")] public async Task SetModelAsync_Serializes_AutoTier(string expectedTier) { await using var server = await FakeCopilotServer.StartAsync(); @@ -593,10 +596,10 @@ public async Task SetAutoTierAsync_Serializes_Tier_And_Returns_Snapshot() OnPermissionRequest = PermissionHandler.ApproveAll }); - var result = await session.SetAutoTierAsync(AutoTier.Intelligence); + var result = await session.SetAutoTierAsync(AutoTier.Fast); var request = Assert.Single(server.Requests, request => request.Method == "session.model.switchAutoTier"); - Assert.Equal("intelligence", request.Params.GetProperty("autoTier").GetString()); + Assert.Equal("fast", request.Params.GetProperty("autoTier").GetString()); Assert.Equal(ModelSwitchAutoTierStatus.Pending, result.Status); Assert.Equal(AutoTier.Balance, result.EffectiveAutoTier); } diff --git a/dotnet/test/Unit/SessionEventSerializationTests.cs b/dotnet/test/Unit/SessionEventSerializationTests.cs index 405ffb379f..6f1b5283e0 100644 --- a/dotnet/test/Unit/SessionEventSerializationTests.cs +++ b/dotnet/test/Unit/SessionEventSerializationTests.cs @@ -14,6 +14,7 @@ public class SessionEventSerializationTests { AutoTier.Efficiency, "efficiency" }, { AutoTier.Balance, "balance" }, { AutoTier.Intelligence, "intelligence" }, + { AutoTier.Fast, "fast" }, { null, null }, }; @@ -102,7 +103,7 @@ public void SessionEvent_Deserializes_AutoTierSwitchFailed(string wireReason) "type": "session.auto_tier_switch_failed", "data": { "effectiveAutoTier": "balance", - "requestedAutoTier": "intelligence", + "requestedAutoTier": "fast", "reason": "{{wireReason}}" } } @@ -113,7 +114,7 @@ public void SessionEvent_Deserializes_AutoTierSwitchFailed(string wireReason) var data = Assert.IsType(sessionEvent).Data; Assert.Equal(new AutoTierSwitchFailureReason(wireReason), data.Reason); Assert.Equal(AutoTier.Balance, data.EffectiveAutoTier); - Assert.Equal(AutoTier.Intelligence, data.RequestedAutoTier); + Assert.Equal(AutoTier.Fast, data.RequestedAutoTier); } [Fact] diff --git a/go/README.md b/go/README.md index 2eb2c720fc..cbbb84b78c 100644 --- a/go/README.md +++ b/go/README.md @@ -361,9 +361,13 @@ Unknown section IDs are handled gracefully: content from `replace`/`append`/`pre ## Auto routing tiers +The canonical values are `AutoTierEfficiency`, `AutoTierBalance`, `AutoTierIntelligence`, and `AutoTierFast`, which send `efficiency`, `balance`, `intelligence`, and `fast` on the wire. Fast is an integrator-only latency preset, not a fourth first-party GitHub Copilot preference. The SDK forwards the requested value without deciding eligibility or inspecting client identity. An externally supplied older runtime returns its native runtime or JSON-RPC error; the SDK does not downgrade or silently ignore the request. + +Omitting the tier on create uses the runtime default rather than Balance. A cold resume restores the persisted tier unless the resume request supplies an explicit override. + Change the Auto routing preference without changing the selected model. The runtime does not apply the preference immediately: it records the request and commits it only when a later user turn using the `auto` model successfully obtains a usable model from the provider, so a `pending` status confirms acceptance rather than effect. Only the most recent request survives. -Watch for the outcome through the `session.model_change` event on success or the ephemeral `session.auto_tier_switch_failed` event on failure. Read the authoritative committed, pending, and activating preferences at any time through the session's `model.getCurrent` RPC method. +Watch for the outcome through the `session.model_change` event on success or the ephemeral `session.auto_tier_switch_failed` event on failure. A failed activation leaves the incumbent effective tier unchanged. Read the authoritative committed, pending, and activating preferences at any time through the session's `model.getCurrent` RPC method. ```go tier := copilot.AutoTierIntelligence diff --git a/go/client_test.go b/go/client_test.go index 52587c8462..1613c3b394 100644 --- a/go/client_test.go +++ b/go/client_test.go @@ -635,9 +635,11 @@ func TestClient_ForwardsCapiOptionsToSessionRequests(t *testing.T) { {"efficiency", &CapiSessionOptions{AutoTier: AutoTierEfficiency}, map[string]any{"autoTier": "efficiency"}}, {"balance", &CapiSessionOptions{AutoTier: AutoTierBalance}, map[string]any{"autoTier": "balance"}}, {"intelligence", &CapiSessionOptions{AutoTier: AutoTierIntelligence}, map[string]any{"autoTier": "intelligence"}}, + {"fast", &CapiSessionOptions{AutoTier: AutoTierFast}, map[string]any{"autoTier": "fast"}}, {"efficiency with websocket", &CapiSessionOptions{AutoTier: AutoTierEfficiency, EnableWebSocketResponses: Bool(false)}, map[string]any{"autoTier": "efficiency", "enableWebSocketResponses": false}}, {"balance with websocket", &CapiSessionOptions{AutoTier: AutoTierBalance, EnableWebSocketResponses: Bool(false)}, map[string]any{"autoTier": "balance", "enableWebSocketResponses": false}}, {"intelligence with websocket", &CapiSessionOptions{AutoTier: AutoTierIntelligence, EnableWebSocketResponses: Bool(false)}, map[string]any{"autoTier": "intelligence", "enableWebSocketResponses": false}}, + {"fast with websocket", &CapiSessionOptions{AutoTier: AutoTierFast, EnableWebSocketResponses: Bool(false)}, map[string]any{"autoTier": "fast", "enableWebSocketResponses": false}}, } for _, tt := range tests { t.Run(tt.name, func(t *testing.T) { diff --git a/go/internal/e2e/auto_tier_e2e_test.go b/go/internal/e2e/auto_tier_e2e_test.go index e974f95927..c8689ea904 100644 --- a/go/internal/e2e/auto_tier_e2e_test.go +++ b/go/internal/e2e/auto_tier_e2e_test.go @@ -1,7 +1,9 @@ package e2e import ( + "sync/atomic" "testing" + "time" copilot "github.com/github/copilot-sdk/go" "github.com/github/copilot-sdk/go/internal/e2e/testharness" @@ -41,16 +43,21 @@ func TestAutoTierE2E(t *testing.T) { } } - newAutoSession := func(t *testing.T) *copilot.Session { + newAutoClient := func(t *testing.T) (*testharness.TestContext, *copilot.Client) { t.Helper() ctx := testharness.NewTestContext(t) + ctx.ConfigureForTest(t) client := ctx.NewClient() t.Cleanup(func() { client.ForceStop() }) if err := client.Start(t.Context()); err != nil { t.Fatalf("Failed to start client: %v", err) } - ctx.ConfigureForTest(t) + return ctx, client + } + newAutoSession := func(t *testing.T) *copilot.Session { + t.Helper() + _, client := newAutoClient(t) session, err := client.CreateSession(t.Context(), &copilot.SessionConfig{ Model: "auto", OnPermissionRequest: copilot.PermissionHandler.ApproveAll, @@ -78,9 +85,9 @@ func TestAutoTierE2E(t *testing.T) { assertPending(t, session, rpc.AutoTierEfficiency) // A second request replaces the first and reports the one it displaced. - superseded, err := session.SetAutoTier(t.Context(), autoTier(copilot.AutoTierIntelligence)) + superseded, err := session.SetAutoTier(t.Context(), autoTier(copilot.AutoTierFast)) if err != nil { - t.Fatalf("SetAutoTier(intelligence) failed: %v", err) + t.Fatalf("SetAutoTier(fast) failed: %v", err) } if superseded.Status != rpc.ModelSwitchAutoTierStatusPending { t.Fatalf("Expected status pending, got %q", superseded.Status) @@ -88,6 +95,18 @@ func TestAutoTierE2E(t *testing.T) { if superseded.SupersededAutoTier == nil || *superseded.SupersededAutoTier != rpc.AutoTierEfficiency { t.Fatalf("Expected superseded efficiency, got %+v", superseded) } + assertPending(t, session, rpc.AutoTierFast) + + replacedFast, err := session.SetAutoTier(t.Context(), autoTier(copilot.AutoTierIntelligence)) + if err != nil { + t.Fatalf("SetAutoTier(intelligence) failed: %v", err) + } + if replacedFast.Status != rpc.ModelSwitchAutoTierStatusPending { + t.Fatalf("Expected status pending, got %q", replacedFast.Status) + } + if replacedFast.SupersededAutoTier == nil || *replacedFast.SupersededAutoTier != rpc.AutoTierFast { + t.Fatalf("Expected superseded fast, got %+v", replacedFast) + } assertPending(t, session, rpc.AutoTierIntelligence) // A nil tier returns the session to provider-default routing. The status is @@ -122,11 +141,11 @@ func TestAutoTierE2E(t *testing.T) { // Supplying a tier replaces it. if err := session.SetModel(t.Context(), "auto", &copilot.SetModelOptions{ - AutoTier: autoTier(copilot.AutoTierIntelligence), + AutoTier: autoTier(copilot.AutoTierFast), }); err != nil { t.Fatalf("SetModel with AutoTier failed: %v", err) } - assertPending(t, session, rpc.AutoTierIntelligence) + assertPending(t, session, rpc.AutoTierFast) // ResetAutoTier clears it. Omission, a value, and a reset are three distinct // outcomes, which is why a single nillable field cannot express the request. @@ -137,4 +156,325 @@ func TestAutoTierE2E(t *testing.T) { } assertNoPending(t, session) }) + + t.Run("should restore and override fast auto tier on cold resume", func(t *testing.T) { + ctx, client := newAutoClient(t) + fastSession, err := client.CreateSession(t.Context(), &copilot.SessionConfig{ + Model: "auto", + OnPermissionRequest: copilot.PermissionHandler.ApproveAll, + Capi: &copilot.CapiSessionOptions{ + AutoTier: copilot.AutoTierFast, + EnableWebSocketResponses: copilot.Bool(false), + }, + }) + if err != nil { + t.Fatalf("CreateSession(fast) failed: %v", err) + } + tierlessSession, err := client.CreateSession(t.Context(), &copilot.SessionConfig{ + Model: "auto", + OnPermissionRequest: copilot.PermissionHandler.ApproveAll, + Capi: &copilot.CapiSessionOptions{ + EnableWebSocketResponses: copilot.Bool(false), + }, + }) + if err != nil { + t.Fatalf("CreateSession(tierless) failed: %v", err) + } + fastSessionID := fastSession.SessionID + tierlessSessionID := tierlessSession.SessionID + + if _, err := fastSession.SendAndWait(t.Context(), copilot.MessageOptions{ + Prompt: "Reply with exactly AUTO_TIER_COLD_RESUME_READY.", + }); err != nil { + t.Fatalf("Fast session turn failed: %v", err) + } + if _, err := tierlessSession.SendAndWait(t.Context(), copilot.MessageOptions{ + Prompt: "Reply with exactly AUTO_TIER_TIERLESS_READY.", + }); err != nil { + t.Fatalf("Tierless session turn failed: %v", err) + } + fastCurrent, err := fastSession.RPC.Model.GetCurrent(t.Context()) + if err != nil { + t.Fatalf("GetCurrent(fast) failed: %v", err) + } + if fastCurrent.AutoTier == nil || *fastCurrent.AutoTier != rpc.AutoTierFast { + t.Fatalf("Expected effective fast tier, got %+v", fastCurrent) + } + tierlessCurrent, err := tierlessSession.RPC.Model.GetCurrent(t.Context()) + if err != nil { + t.Fatalf("GetCurrent(tierless) failed: %v", err) + } + if tierlessCurrent.AutoTier != nil { + t.Fatalf("Expected no effective tier, got %+v", tierlessCurrent) + } + + fastSession.Disconnect() + tierlessSession.Disconnect() + if err := client.Stop(); err != nil { + t.Fatalf("Stop initial client failed: %v", err) + } + + restoredClient := ctx.NewClient() + t.Cleanup(func() { restoredClient.ForceStop() }) + if err := restoredClient.Start(t.Context()); err != nil { + t.Fatalf("Start restored client failed: %v", err) + } + restoredFast, err := restoredClient.ResumeSession(t.Context(), fastSessionID, &copilot.ResumeSessionConfig{ + OnPermissionRequest: copilot.PermissionHandler.ApproveAll, + }) + if err != nil { + t.Fatalf("ResumeSession(fast) failed: %v", err) + } + restoredTierless, err := restoredClient.ResumeSession(t.Context(), tierlessSessionID, &copilot.ResumeSessionConfig{ + OnPermissionRequest: copilot.PermissionHandler.ApproveAll, + }) + if err != nil { + t.Fatalf("ResumeSession(tierless) failed: %v", err) + } + restoredFastCurrent, err := restoredFast.RPC.Model.GetCurrent(t.Context()) + if err != nil { + t.Fatalf("GetCurrent(restored fast) failed: %v", err) + } + if restoredFastCurrent.AutoTier == nil || *restoredFastCurrent.AutoTier != rpc.AutoTierFast { + t.Fatalf("Expected restored fast tier, got %+v", restoredFastCurrent) + } + restoredTierlessCurrent, err := restoredTierless.RPC.Model.GetCurrent(t.Context()) + if err != nil { + t.Fatalf("GetCurrent(restored tierless) failed: %v", err) + } + if restoredTierlessCurrent.AutoTier != nil { + t.Fatalf("Expected restored tierless state, got %+v", restoredTierlessCurrent) + } + restoredFast.Disconnect() + restoredTierless.Disconnect() + if err := restoredClient.Stop(); err != nil { + t.Fatalf("Stop restored client failed: %v", err) + } + + overrideClient := ctx.NewClient() + t.Cleanup(func() { overrideClient.ForceStop() }) + if err := overrideClient.Start(t.Context()); err != nil { + t.Fatalf("Start override client failed: %v", err) + } + overridden, err := overrideClient.ResumeSession(t.Context(), fastSessionID, &copilot.ResumeSessionConfig{ + Model: "auto", + OnPermissionRequest: copilot.PermissionHandler.ApproveAll, + Capi: &copilot.CapiSessionOptions{ + AutoTier: copilot.AutoTierBalance, + EnableWebSocketResponses: copilot.Bool(false), + }, + }) + if err != nil { + t.Fatalf("ResumeSession(override) failed: %v", err) + } + overriddenCurrent, err := overridden.RPC.Model.GetCurrent(t.Context()) + if err != nil { + t.Fatalf("GetCurrent(overridden) failed: %v", err) + } + if overriddenCurrent.AutoTier == nil || *overriddenCurrent.AutoTier != rpc.AutoTierBalance { + t.Fatalf("Expected balance override, got %+v", overriddenCurrent) + } + overridden.Disconnect() + if err := overrideClient.Stop(); err != nil { + t.Fatalf("Stop override client failed: %v", err) + } + }) + + t.Run("should commit fast auto tier after successful turn", func(t *testing.T) { + _, client := newAutoClient(t) + session, err := client.CreateSession(t.Context(), &copilot.SessionConfig{ + Model: "auto", + OnPermissionRequest: copilot.PermissionHandler.ApproveAll, + Capi: &copilot.CapiSessionOptions{ + AutoTier: copilot.AutoTierEfficiency, + EnableWebSocketResponses: copilot.Bool(false), + }, + }) + if err != nil { + t.Fatalf("CreateSession failed: %v", err) + } + + modelChanged := make(chan *copilot.SessionModelChangeData, 1) + unsubscribe := session.On(func(event copilot.SessionEvent) { + if data, ok := event.Data.(*copilot.SessionModelChangeData); ok && data.AutoTier != nil && *data.AutoTier == copilot.AutoTierFast { + select { + case modelChanged <- data: + default: + } + } + }) + defer unsubscribe() + + staged, err := session.SetAutoTier(t.Context(), autoTier(copilot.AutoTierFast)) + if err != nil { + t.Fatalf("SetAutoTier(fast) failed: %v", err) + } + if staged.Status != rpc.ModelSwitchAutoTierStatusPending || + staged.EffectiveAutoTier == nil || *staged.EffectiveAutoTier != rpc.AutoTierEfficiency || + staged.PendingAutoTier == nil || *staged.PendingAutoTier != rpc.AutoTierFast { + t.Fatalf("Unexpected staged result: %+v", staged) + } + + beforeTurn, err := session.RPC.Model.GetCurrent(t.Context()) + if err != nil { + t.Fatalf("GetCurrent before turn failed: %v", err) + } + if beforeTurn.AutoTier == nil || *beforeTurn.AutoTier != rpc.AutoTierEfficiency || + beforeTurn.PendingAutoTier == nil || *beforeTurn.PendingAutoTier != rpc.AutoTierFast { + t.Fatalf("Unexpected state before turn: %+v", beforeTurn) + } + + if _, err := session.SendAndWait(t.Context(), copilot.MessageOptions{ + Prompt: "Reply with exactly AUTO_TIER_FAST_COMMITTED.", + }); err != nil { + t.Fatalf("SendAndWait failed: %v", err) + } + + select { + case data := <-modelChanged: + if data.PreviousModel == nil || *data.PreviousModel != "auto" || + data.NewModel != "auto" || + data.PreviousAutoTier == nil || *data.PreviousAutoTier != copilot.AutoTierEfficiency { + t.Fatalf("Unexpected model change: %+v", data) + } + case <-time.After(30 * time.Second): + t.Fatal("Timed out waiting for Fast model change") + } + + committed, err := session.RPC.Model.GetCurrent(t.Context()) + if err != nil { + t.Fatalf("GetCurrent after turn failed: %v", err) + } + if committed.AutoTier == nil || *committed.AutoTier != rpc.AutoTierFast || + committed.PendingAutoTier != nil || committed.ActivatingAutoTier != nil { + t.Fatalf("Unexpected committed state: %+v", committed) + } + }) + + t.Run("should preserve effective tier when fast activation fails", func(t *testing.T) { + ctx, client := newAutoClient(t) + session, err := client.CreateSession(t.Context(), &copilot.SessionConfig{ + Model: "auto", + OnPermissionRequest: copilot.PermissionHandler.ApproveAll, + Capi: &copilot.CapiSessionOptions{ + AutoTier: copilot.AutoTierEfficiency, + EnableWebSocketResponses: copilot.Bool(false), + }, + }) + if err != nil { + t.Fatalf("CreateSession failed: %v", err) + } + sessionID := session.SessionID + if _, err := session.SendAndWait(t.Context(), copilot.MessageOptions{ + Prompt: "Reply with exactly AUTO_TIER_INITIAL_READY.", + }); err != nil { + t.Fatalf("Initial SendAndWait failed: %v", err) + } + + failureObserved := make(chan struct { + data *copilot.SessionAutoTierSwitchFailedData + ephemeral bool + }, 1) + var fastCommitted atomic.Bool + unsubscribe := session.On(func(event copilot.SessionEvent) { + switch data := event.Data.(type) { + case *copilot.SessionAutoTierSwitchFailedData: + select { + case failureObserved <- struct { + data *copilot.SessionAutoTierSwitchFailedData + ephemeral bool + }{data: data, ephemeral: event.Ephemeral != nil && *event.Ephemeral}: + default: + } + case *copilot.SessionModelChangeData: + if data.AutoTier != nil && *data.AutoTier == copilot.AutoTierFast { + fastCommitted.Store(true) + } + } + }) + + staged, err := session.SetAutoTier(t.Context(), autoTier(copilot.AutoTierFast)) + if err != nil { + t.Fatalf("SetAutoTier(fast) failed: %v", err) + } + if staged.Status != rpc.ModelSwitchAutoTierStatusPending || + staged.EffectiveAutoTier == nil || *staged.EffectiveAutoTier != rpc.AutoTierEfficiency || + staged.PendingAutoTier == nil || *staged.PendingAutoTier != rpc.AutoTierFast { + t.Fatalf("Unexpected staged result: %+v", staged) + } + + if _, err := session.SendAndWait(t.Context(), copilot.MessageOptions{ + Prompt: "Reply with exactly AUTO_TIER_FAILURE_RECOVERED.", + }); err != nil { + t.Fatalf("Failure-path SendAndWait failed: %v", err) + } + + select { + case observed := <-failureObserved: + if !observed.ephemeral || + observed.data.EffectiveAutoTier == nil || *observed.data.EffectiveAutoTier != copilot.AutoTierEfficiency || + observed.data.RequestedAutoTier == nil || *observed.data.RequestedAutoTier != copilot.AutoTierFast || + observed.data.Reason != copilot.AutoTierSwitchFailureReasonRequestFailed { + t.Fatalf("Unexpected failure event: %+v", observed) + } + case <-time.After(30 * time.Second): + t.Fatal("Timed out waiting for Fast activation failure") + } + if fastCommitted.Load() { + t.Fatal("Fast tier committed after failed activation") + } + + current, err := session.RPC.Model.GetCurrent(t.Context()) + if err != nil { + t.Fatalf("GetCurrent after failure failed: %v", err) + } + if current.AutoTier == nil || *current.AutoTier != rpc.AutoTierEfficiency || + current.PendingAutoTier != nil || current.ActivatingAutoTier != nil { + t.Fatalf("Unexpected state after failure: %+v", current) + } + + unsubscribe() + session.Disconnect() + if err := client.Stop(); err != nil { + t.Fatalf("Stop failed client failed: %v", err) + } + + resumedClient := ctx.NewClient() + t.Cleanup(func() { resumedClient.ForceStop() }) + if err := resumedClient.Start(t.Context()); err != nil { + t.Fatalf("Start resumed client failed: %v", err) + } + resumed, err := resumedClient.ResumeSession(t.Context(), sessionID, &copilot.ResumeSessionConfig{ + OnPermissionRequest: copilot.PermissionHandler.ApproveAll, + }) + if err != nil { + t.Fatalf("ResumeSession failed: %v", err) + } + resumedCurrent, err := resumed.RPC.Model.GetCurrent(t.Context()) + if err != nil { + t.Fatalf("GetCurrent after resume failed: %v", err) + } + if resumedCurrent.AutoTier == nil || *resumedCurrent.AutoTier != rpc.AutoTierEfficiency || + resumedCurrent.PendingAutoTier != nil || resumedCurrent.ActivatingAutoTier != nil { + t.Fatalf("Unexpected resumed state: %+v", resumedCurrent) + } + max := int64(100) + waitMs := int32(0) + persisted, err := resumed.RPC.EventLog.Read(t.Context(), &rpc.EventLogReadRequest{ + Max: &max, + WaitMs: &waitMs, + }) + if err != nil { + t.Fatalf("EventLog.Read failed: %v", err) + } + for _, event := range persisted.Events { + if event.Type() == copilot.SessionEventTypeSessionAutoTierSwitchFailed { + t.Fatal("Ephemeral failure event was replayed after cold resume") + } + } + resumed.Disconnect() + if err := resumedClient.Stop(); err != nil { + t.Fatalf("Stop resumed client failed: %v", err) + } + }) } diff --git a/go/session_event_serialization_test.go b/go/session_event_serialization_test.go index c64b6ce598..4941abc811 100644 --- a/go/session_event_serialization_test.go +++ b/go/session_event_serialization_test.go @@ -16,7 +16,7 @@ var _ EmbeddedTextResourceContents = rpc.EmbeddedTextResourceContents{} func TestSessionEventAutoTier(t *testing.T) { for _, eventType := range []string{"session.start", "session.resume"} { - for _, tier := range []AutoTier{"", AutoTierEfficiency, AutoTierBalance, AutoTierIntelligence} { + for _, tier := range []AutoTier{"", AutoTierEfficiency, AutoTierBalance, AutoTierIntelligence, AutoTierFast} { t.Run(eventType+"/"+string(tier), func(t *testing.T) { data := map[string]any{ "sessionId": "test-session", "version": 1, @@ -317,7 +317,7 @@ func TestSessionAutoTierSwitchFailedEvent(t *testing.T) { "type": "session.auto_tier_switch_failed", "data": map[string]any{ "effectiveAutoTier": AutoTierBalance, - "requestedAutoTier": AutoTierIntelligence, + "requestedAutoTier": AutoTierFast, "reason": reason, }, }) @@ -338,8 +338,8 @@ func TestSessionAutoTierSwitchFailedEvent(t *testing.T) { if data.EffectiveAutoTier == nil || *data.EffectiveAutoTier != AutoTierBalance { t.Fatalf("expected effective tier %q, got %v", AutoTierBalance, data.EffectiveAutoTier) } - if data.RequestedAutoTier == nil || *data.RequestedAutoTier != AutoTierIntelligence { - t.Fatalf("expected requested tier %q, got %v", AutoTierIntelligence, data.RequestedAutoTier) + if data.RequestedAutoTier == nil || *data.RequestedAutoTier != AutoTierFast { + t.Fatalf("expected requested tier %q, got %v", AutoTierFast, data.RequestedAutoTier) } }) } diff --git a/go/session_test.go b/go/session_test.go index bdf14887a3..c3ab144afe 100644 --- a/go/session_test.go +++ b/go/session_test.go @@ -122,14 +122,14 @@ func TestSession_SetModelOmitsContextTierWhenUnset(t *testing.T) { } func TestSession_SetModelForwardsAutoTier(t *testing.T) { - tier := AutoTierIntelligence + tier := AutoTierFast params := captureSetModelRequestForModel(t, "auto", &SetModelOptions{AutoTier: &tier}) if params["modelId"] != "auto" { t.Fatalf("expected modelId auto, got %v", params["modelId"]) } - if params["autoTier"] != "intelligence" { - t.Fatalf("expected autoTier intelligence, got %v", params["autoTier"]) + if params["autoTier"] != "fast" { + t.Fatalf("expected autoTier fast, got %v", params["autoTier"]) } } @@ -161,14 +161,14 @@ func TestSession_SetModelRejectsConflictingAutoTierOptions(t *testing.T) { } func TestSession_SetAutoTierForwardsTier(t *testing.T) { - tier := AutoTierEfficiency + tier := AutoTierFast params := captureSetAutoTierRequest(t, &tier) if params["sessionId"] != "session-1" { t.Fatalf("expected sessionId session-1, got %v", params["sessionId"]) } - if params["autoTier"] != "efficiency" { - t.Fatalf("expected autoTier efficiency, got %v", params["autoTier"]) + if params["autoTier"] != "fast" { + t.Fatalf("expected autoTier fast, got %v", params["autoTier"]) } } diff --git a/java/README.md b/java/README.md index af13c3ec16..373a48c1f0 100644 --- a/java/README.md +++ b/java/README.md @@ -369,9 +369,12 @@ For design context and decision rationale, see [ADR-006](docs/adr/adr-006-tool-d ## Auto routing tiers Use `CapiSessionOptions.setAutoTier(...)` to select `AutoTier.EFFICIENCY`, -`AutoTier.BALANCE`, or `AutoTier.INTELLIGENCE`. This option is meaningful only -with model `auto` (Auto mode V2). +`AutoTier.BALANCE`, `AutoTier.INTELLIGENCE`, or `AutoTier.FAST`. This option is +meaningful only with model `auto` (Auto mode V2). It requires a runtime version that supports `capi.autoTier`. +`AutoTier.FAST` is an integrator-only latency preset, not a first-party GitHub +Copilot product preference — the SDK does not decide Fast eligibility or apply +it implicitly. ```java import com.github.copilot.rpc.AutoTier; @@ -398,7 +401,7 @@ for the lifecycle rules. Change the Auto routing preference without changing the selected model. The runtime does not apply the preference immediately: it records the request and commits it only when a later user turn using the `auto` model successfully obtains a usable model from the provider, so a `pending` status confirms acceptance rather than effect. Only the most recent request survives. -Watch for the outcome through the `session.model_change` event on success or the ephemeral `session.auto_tier_switch_failed` event on failure. Read the authoritative committed, pending, and activating preferences at any time through the session's `model.getCurrent` RPC method. +Watch for the outcome through the `session.model_change` event on success or the ephemeral `session.auto_tier_switch_failed` event on failure. A failed activation leaves the incumbent effective tier unchanged. Read the authoritative committed, pending, and activating preferences at any time through the session's `model.getCurrent` RPC method. ```java var result = session.setAutoTier(AutoTier.INTELLIGENCE).get(); diff --git a/java/sdk/src/main/java/com/github/copilot/rpc/AutoTier.java b/java/sdk/src/main/java/com/github/copilot/rpc/AutoTier.java index f9117abfb2..092dc73026 100644 --- a/java/sdk/src/main/java/com/github/copilot/rpc/AutoTier.java +++ b/java/sdk/src/main/java/com/github/copilot/rpc/AutoTier.java @@ -21,7 +21,13 @@ public enum AutoTier { BALANCE("balance"), /** Prioritize intelligence. */ - INTELLIGENCE("intelligence"); + INTELLIGENCE("intelligence"), + + /** + * Integrator-only preset that optimizes for latency. Not a first-party GitHub + * Copilot product preference. + */ + FAST("fast"); private final String value; diff --git a/java/sdk/src/test/java/com/github/copilot/AutoTierIT.java b/java/sdk/src/test/java/com/github/copilot/AutoTierIT.java index d486ce1c2c..d93cbf995b 100644 --- a/java/sdk/src/test/java/com/github/copilot/AutoTierIT.java +++ b/java/sdk/src/test/java/com/github/copilot/AutoTierIT.java @@ -1,17 +1,28 @@ package com.github.copilot; import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; import static org.junit.jupiter.api.Assertions.assertNull; +import static org.junit.jupiter.api.Assertions.assertTrue; +import java.util.concurrent.CompletableFuture; import java.util.concurrent.TimeUnit; +import java.util.concurrent.atomic.AtomicBoolean; import org.junit.jupiter.api.AfterAll; import org.junit.jupiter.api.BeforeAll; import org.junit.jupiter.api.Test; +import com.github.copilot.generated.AutoTierSwitchFailureReason; +import com.github.copilot.generated.SessionAutoTierSwitchFailedEvent; +import com.github.copilot.generated.SessionModelChangeEvent; import com.github.copilot.generated.rpc.ModelSwitchAutoTierStatus; +import com.github.copilot.generated.rpc.SessionEventLogReadParams; import com.github.copilot.rpc.AutoTier; +import com.github.copilot.rpc.CapiSessionOptions; +import com.github.copilot.rpc.MessageOptions; import com.github.copilot.rpc.PermissionHandler; +import com.github.copilot.rpc.ResumeSessionConfig; import com.github.copilot.rpc.SessionConfig; import com.github.copilot.rpc.SetModelOptions; @@ -69,10 +80,16 @@ void shouldStageAndResetAutoTierPreference() throws Exception { assertEquals(com.github.copilot.generated.rpc.AutoTier.EFFICIENCY, pendingAutoTier(session)); // A second request replaces the first and reports the one it displaced. - var superseded = session.setAutoTier(AutoTier.INTELLIGENCE).get(30, TimeUnit.SECONDS); + var superseded = session.setAutoTier(AutoTier.FAST).get(30, TimeUnit.SECONDS); assertEquals(ModelSwitchAutoTierStatus.PENDING, superseded.status()); - assertEquals(com.github.copilot.generated.rpc.AutoTier.INTELLIGENCE, superseded.pendingAutoTier()); + assertEquals(com.github.copilot.generated.rpc.AutoTier.FAST, superseded.pendingAutoTier()); assertEquals(com.github.copilot.generated.rpc.AutoTier.EFFICIENCY, superseded.supersededAutoTier()); + assertEquals(com.github.copilot.generated.rpc.AutoTier.FAST, pendingAutoTier(session)); + + var replacedFast = session.setAutoTier(AutoTier.INTELLIGENCE).get(30, TimeUnit.SECONDS); + assertEquals(ModelSwitchAutoTierStatus.PENDING, replacedFast.status()); + assertEquals(com.github.copilot.generated.rpc.AutoTier.INTELLIGENCE, replacedFast.pendingAutoTier()); + assertEquals(com.github.copilot.generated.rpc.AutoTier.FAST, replacedFast.supersededAutoTier()); assertEquals(com.github.copilot.generated.rpc.AutoTier.INTELLIGENCE, pendingAutoTier(session)); // A null tier returns the session to provider-default routing. The status @@ -103,9 +120,9 @@ void shouldPreserveAutoTierWhenSetModelOmitsIt() throws Exception { assertEquals(com.github.copilot.generated.rpc.AutoTier.BALANCE, pendingAutoTier(session)); // Supplying a tier replaces it. - session.setModel(new SetModelOptions().setModel(MODEL_ID).setAutoTier(AutoTier.INTELLIGENCE)).get(30, + session.setModel(new SetModelOptions().setModel(MODEL_ID).setAutoTier(AutoTier.FAST)).get(30, TimeUnit.SECONDS); - assertEquals(com.github.copilot.generated.rpc.AutoTier.INTELLIGENCE, pendingAutoTier(session)); + assertEquals(com.github.copilot.generated.rpc.AutoTier.FAST, pendingAutoTier(session)); // Requesting a reset clears it. Omission, an explicit tier, and a reset // are three distinct outcomes. @@ -117,4 +134,175 @@ void shouldPreserveAutoTierWhenSetModelOmitsIt() throws Exception { } } } + + @Test + void shouldRestoreAndOverrideFastAutoTierOnColdResume() throws Exception { + ctx.configureForTest("auto_tier", "should_restore_and_override_fast_auto_tier_on_cold_resume"); + + String fastSessionId; + String tierlessSessionId; + try (CopilotClient client = ctx.createClient()) { + try (CopilotSession fastSession = client.createSession( + new SessionConfig().setModel(MODEL_ID).setOnPermissionRequest(PermissionHandler.APPROVE_ALL) + .setCapi(new CapiSessionOptions().setAutoTier(AutoTier.FAST) + .setEnableWebSocketResponses(false))) + .get(30, TimeUnit.SECONDS); + CopilotSession tierlessSession = client + .createSession(new SessionConfig().setModel(MODEL_ID) + .setOnPermissionRequest(PermissionHandler.APPROVE_ALL) + .setCapi(new CapiSessionOptions().setEnableWebSocketResponses(false))) + .get(30, TimeUnit.SECONDS)) { + fastSessionId = fastSession.getSessionId(); + tierlessSessionId = tierlessSession.getSessionId(); + + fastSession + .sendAndWait(new MessageOptions().setPrompt("Reply with exactly AUTO_TIER_COLD_RESUME_READY.")) + .get(30, TimeUnit.SECONDS); + tierlessSession + .sendAndWait(new MessageOptions().setPrompt("Reply with exactly AUTO_TIER_TIERLESS_READY.")) + .get(30, TimeUnit.SECONDS); + + assertEquals(com.github.copilot.generated.rpc.AutoTier.FAST, + fastSession.getRpc().model.getCurrent().get(30, TimeUnit.SECONDS).autoTier()); + assertNull(tierlessSession.getRpc().model.getCurrent().get(30, TimeUnit.SECONDS).autoTier()); + } + client.stop().get(30, TimeUnit.SECONDS); + } + + try (CopilotClient restoredClient = ctx.createClient()) { + try (CopilotSession restoredFast = restoredClient + .resumeSession(fastSessionId, + new ResumeSessionConfig().setOnPermissionRequest(PermissionHandler.APPROVE_ALL)) + .get(30, TimeUnit.SECONDS); + CopilotSession restoredTierless = restoredClient + .resumeSession(tierlessSessionId, + new ResumeSessionConfig().setOnPermissionRequest(PermissionHandler.APPROVE_ALL)) + .get(30, TimeUnit.SECONDS)) { + assertEquals(com.github.copilot.generated.rpc.AutoTier.FAST, + restoredFast.getRpc().model.getCurrent().get(30, TimeUnit.SECONDS).autoTier()); + assertNull(restoredTierless.getRpc().model.getCurrent().get(30, TimeUnit.SECONDS).autoTier()); + } + restoredClient.stop().get(30, TimeUnit.SECONDS); + } + + try (CopilotClient overrideClient = ctx.createClient(); + CopilotSession overridden = overrideClient + .resumeSession(fastSessionId, new ResumeSessionConfig().setModel(MODEL_ID) + .setOnPermissionRequest(PermissionHandler.APPROVE_ALL).setCapi(new CapiSessionOptions() + .setAutoTier(AutoTier.BALANCE).setEnableWebSocketResponses(false))) + .get(30, TimeUnit.SECONDS)) { + assertEquals(com.github.copilot.generated.rpc.AutoTier.BALANCE, + overridden.getRpc().model.getCurrent().get(30, TimeUnit.SECONDS).autoTier()); + } + } + + @Test + void shouldCommitFastAutoTierAfterSuccessfulTurn() throws Exception { + ctx.configureForTest("auto_tier", "should_commit_fast_auto_tier_after_successful_turn"); + + try (CopilotClient client = ctx.createClient(); + CopilotSession session = client + .createSession(new SessionConfig().setModel(MODEL_ID) + .setOnPermissionRequest(PermissionHandler.APPROVE_ALL).setCapi(new CapiSessionOptions() + .setAutoTier(AutoTier.EFFICIENCY).setEnableWebSocketResponses(false))) + .get(30, TimeUnit.SECONDS)) { + var modelChangedFuture = new CompletableFuture(); + try (var subscription = session.on(SessionModelChangeEvent.class, event -> { + if (event.getData().autoTier() == com.github.copilot.generated.AutoTier.FAST) { + modelChangedFuture.complete(event); + } + })) { + var staged = session.setAutoTier(AutoTier.FAST).get(30, TimeUnit.SECONDS); + assertEquals(ModelSwitchAutoTierStatus.PENDING, staged.status()); + assertEquals(com.github.copilot.generated.rpc.AutoTier.EFFICIENCY, staged.effectiveAutoTier()); + assertEquals(com.github.copilot.generated.rpc.AutoTier.FAST, staged.pendingAutoTier()); + + var beforeTurn = session.getRpc().model.getCurrent().get(30, TimeUnit.SECONDS); + assertEquals(com.github.copilot.generated.rpc.AutoTier.EFFICIENCY, beforeTurn.autoTier()); + assertEquals(com.github.copilot.generated.rpc.AutoTier.FAST, beforeTurn.pendingAutoTier()); + + session.sendAndWait(new MessageOptions().setPrompt("Reply with exactly AUTO_TIER_FAST_COMMITTED.")) + .get(30, TimeUnit.SECONDS); + + var modelChanged = modelChangedFuture.get(30, TimeUnit.SECONDS); + assertEquals(MODEL_ID, modelChanged.getData().previousModel()); + assertEquals(MODEL_ID, modelChanged.getData().newModel()); + assertEquals(com.github.copilot.generated.AutoTier.EFFICIENCY, + modelChanged.getData().previousAutoTier()); + assertEquals(com.github.copilot.generated.AutoTier.FAST, modelChanged.getData().autoTier()); + + var committed = session.getRpc().model.getCurrent().get(30, TimeUnit.SECONDS); + assertEquals(com.github.copilot.generated.rpc.AutoTier.FAST, committed.autoTier()); + assertNull(committed.pendingAutoTier()); + assertNull(committed.activatingAutoTier()); + } + } + } + + @Test + void shouldPreserveEffectiveTierWhenFastActivationFails() throws Exception { + ctx.configureForTest("auto_tier", "should_preserve_effective_tier_when_fast_activation_fails"); + + String sessionId; + try (CopilotClient client = ctx.createClient()) { + try (CopilotSession session = client + .createSession(new SessionConfig().setModel(MODEL_ID) + .setOnPermissionRequest(PermissionHandler.APPROVE_ALL).setCapi(new CapiSessionOptions() + .setAutoTier(AutoTier.EFFICIENCY).setEnableWebSocketResponses(false))) + .get(30, TimeUnit.SECONDS)) { + sessionId = session.getSessionId(); + session.sendAndWait(new MessageOptions().setPrompt("Reply with exactly AUTO_TIER_INITIAL_READY.")) + .get(30, TimeUnit.SECONDS); + + var failureFuture = new CompletableFuture(); + var fastCommitted = new AtomicBoolean(); + try (var failureSubscription = session.on(SessionAutoTierSwitchFailedEvent.class, + failureFuture::complete); + var modelChangeSubscription = session.on(SessionModelChangeEvent.class, event -> { + if (event.getData().autoTier() == com.github.copilot.generated.AutoTier.FAST) { + fastCommitted.set(true); + } + })) { + var staged = session.setAutoTier(AutoTier.FAST).get(30, TimeUnit.SECONDS); + assertEquals(ModelSwitchAutoTierStatus.PENDING, staged.status()); + assertEquals(com.github.copilot.generated.rpc.AutoTier.EFFICIENCY, staged.effectiveAutoTier()); + assertEquals(com.github.copilot.generated.rpc.AutoTier.FAST, staged.pendingAutoTier()); + + session.sendAndWait( + new MessageOptions().setPrompt("Reply with exactly AUTO_TIER_FAILURE_RECOVERED.")) + .get(30, TimeUnit.SECONDS); + + var failure = failureFuture.get(30, TimeUnit.SECONDS); + assertTrue(failure.getEphemeral()); + assertEquals(com.github.copilot.generated.AutoTier.EFFICIENCY, + failure.getData().effectiveAutoTier()); + assertEquals(com.github.copilot.generated.AutoTier.FAST, failure.getData().requestedAutoTier()); + assertEquals(AutoTierSwitchFailureReason.REQUEST_FAILED, failure.getData().reason()); + assertFalse(fastCommitted.get()); + + var current = session.getRpc().model.getCurrent().get(30, TimeUnit.SECONDS); + assertEquals(com.github.copilot.generated.rpc.AutoTier.EFFICIENCY, current.autoTier()); + assertNull(current.pendingAutoTier()); + assertNull(current.activatingAutoTier()); + } + } + client.stop().get(30, TimeUnit.SECONDS); + } + + try (CopilotClient resumedClient = ctx.createClient(); + CopilotSession resumed = resumedClient + .resumeSession(sessionId, + new ResumeSessionConfig().setOnPermissionRequest(PermissionHandler.APPROVE_ALL)) + .get(30, TimeUnit.SECONDS)) { + var resumedCurrent = resumed.getRpc().model.getCurrent().get(30, TimeUnit.SECONDS); + assertEquals(com.github.copilot.generated.rpc.AutoTier.EFFICIENCY, resumedCurrent.autoTier()); + assertNull(resumedCurrent.pendingAutoTier()); + assertNull(resumedCurrent.activatingAutoTier()); + + var persisted = resumed.getRpc().eventLog + .read(new SessionEventLogReadParams(null, null, 100L, 0L, null, null, null, null, false)) + .get(30, TimeUnit.SECONDS); + assertFalse(persisted.events().stream().anyMatch(SessionAutoTierSwitchFailedEvent.class::isInstance)); + } + } } diff --git a/java/sdk/src/test/java/com/github/copilot/CapiSessionOptionsTest.java b/java/sdk/src/test/java/com/github/copilot/CapiSessionOptionsTest.java index dccb4e9add..aeb4ffe0ad 100644 --- a/java/sdk/src/test/java/com/github/copilot/CapiSessionOptionsTest.java +++ b/java/sdk/src/test/java/com/github/copilot/CapiSessionOptionsTest.java @@ -68,7 +68,7 @@ void omitsUnsetEnableWebSocketResponses() { } @ParameterizedTest - @CsvSource({"EFFICIENCY,efficiency", "BALANCE,balance", "INTELLIGENCE,intelligence"}) + @CsvSource({"EFFICIENCY,efficiency", "BALANCE,balance", "INTELLIGENCE,intelligence", "FAST,fast"}) void autoTierCanonicalValuesRoundTripAndForward(AutoTier tier, String value) throws Exception { var mapper = JsonRpcClient.getObjectMapper(); var capi = new CapiSessionOptions().setAutoTier(tier); diff --git a/java/sdk/src/test/java/com/github/copilot/SessionAutoTierEventTest.java b/java/sdk/src/test/java/com/github/copilot/SessionAutoTierEventTest.java index 25e356a8d1..4378c105cd 100644 --- a/java/sdk/src/test/java/com/github/copilot/SessionAutoTierEventTest.java +++ b/java/sdk/src/test/java/com/github/copilot/SessionAutoTierEventTest.java @@ -31,8 +31,9 @@ class SessionAutoTierEventTest { @ParameterizedTest @CsvSource({"session.start,EFFICIENCY,efficiency", "session.start,BALANCE,balance", - "session.start,INTELLIGENCE,intelligence", "session.resume,EFFICIENCY,efficiency", - "session.resume,BALANCE,balance", "session.resume,INTELLIGENCE,intelligence"}) + "session.start,INTELLIGENCE,intelligence", "session.start,FAST,fast", + "session.resume,EFFICIENCY,efficiency", "session.resume,BALANCE,balance", + "session.resume,INTELLIGENCE,intelligence", "session.resume,FAST,fast"}) void canonicalAutoTierRoundTrips(String type, AutoTier tier, String value) throws Exception { String json = """ {"type":"%s","data":{"selectedModel":"auto","autoTier":"%s"}} @@ -74,14 +75,14 @@ void autoTierSwitchFailedEventDecodesEveryReason(String value, AutoTierSwitchFai throws Exception { String json = """ {"type":"session.auto_tier_switch_failed","data":{"effectiveAutoTier":"balance", - "requestedAutoTier":"intelligence","reason":"%s"}} + "requestedAutoTier":"fast","reason":"%s"}} """.formatted(value); var event = MAPPER.readValue(json, SessionEvent.class); var data = assertInstanceOf(SessionAutoTierSwitchFailedEvent.class, event).getData(); assertEquals(AutoTier.BALANCE, data.effectiveAutoTier()); - assertEquals(AutoTier.INTELLIGENCE, data.requestedAutoTier()); + assertEquals(AutoTier.FAST, data.requestedAutoTier()); assertEquals(reason, data.reason()); } diff --git a/java/sdk/src/test/java/com/github/copilot/SessionAutoTierSwitchTest.java b/java/sdk/src/test/java/com/github/copilot/SessionAutoTierSwitchTest.java index adeeca2c6b..74e02c66f9 100644 --- a/java/sdk/src/test/java/com/github/copilot/SessionAutoTierSwitchTest.java +++ b/java/sdk/src/test/java/com/github/copilot/SessionAutoTierSwitchTest.java @@ -52,13 +52,13 @@ void setModel_sends_requested_autoTier() throws Exception { var session = new CopilotSession("sess-2", sockets.client()); var stub = sockets.stubServer(); - session.setModel(new SetModelOptions().setModel("auto").setAutoTier(AutoTier.INTELLIGENCE) - .setReasoningEffort("high")); + session.setModel( + new SetModelOptions().setModel("auto").setAutoTier(AutoTier.FAST).setReasoningEffort("high")); var sent = stub.readOneMessage(); assertEquals("session.model.switchTo", sent.get("method").asText()); var params = sent.get("params"); - assertEquals("intelligence", params.get("autoTier").asText()); + assertEquals("fast", params.get("autoTier").asText()); assertEquals("high", params.get("reasoningEffort").asText()); assertEquals("sess-2", params.get("sessionId").asText()); } @@ -108,12 +108,12 @@ void setAutoTier_sends_the_requested_tier() throws Exception { var session = new CopilotSession("sess-6", sockets.client()); var stub = sockets.stubServer(); - session.setAutoTier(AutoTier.EFFICIENCY); + session.setAutoTier(AutoTier.FAST); var sent = stub.readOneMessage(); assertEquals("session.model.switchAutoTier", sent.get("method").asText()); var params = sent.get("params"); - assertEquals("efficiency", params.get("autoTier").asText()); + assertEquals("fast", params.get("autoTier").asText()); assertEquals("sess-6", params.get("sessionId").asText()); } } @@ -141,7 +141,7 @@ void switchAutoTier_result_deserializes_every_field() throws Exception { { "status": "pending", "effectiveAutoTier": "balance", - "pendingAutoTier": "intelligence", + "pendingAutoTier": "fast", "activatingAutoTier": null, "supersededAutoTier": "efficiency" } @@ -151,7 +151,7 @@ void switchAutoTier_result_deserializes_every_field() throws Exception { assertEquals(ModelSwitchAutoTierStatus.PENDING, result.status()); assertEquals(com.github.copilot.generated.rpc.AutoTier.BALANCE, result.effectiveAutoTier()); - assertEquals(com.github.copilot.generated.rpc.AutoTier.INTELLIGENCE, result.pendingAutoTier()); + assertEquals(com.github.copilot.generated.rpc.AutoTier.FAST, result.pendingAutoTier()); assertNull(result.activatingAutoTier()); assertEquals(com.github.copilot.generated.rpc.AutoTier.EFFICIENCY, result.supersededAutoTier()); } diff --git a/nodejs/README.md b/nodejs/README.md index 2fb98ce972..a6e577be93 100644 --- a/nodejs/README.md +++ b/nodejs/README.md @@ -151,7 +151,7 @@ Create a new conversation session. - `sessionId?: string` - Custom session ID. - `model?: string` - Model to use ("gpt-5", "claude-sonnet-4.5", etc.). **Required when using custom provider.** -- `capi?: CapiSessionOptions` - Copilot API options. With `model: "auto"`, set `autoTier` to `"efficiency"`, `"balance"`, or `"intelligence"` to choose a routing preference. Requires a runtime with Auto tier support and V2 Auto routing. Omission preserves default behavior. See [Auto tier persistence](../docs/features/session-persistence.md#auto-tier-persistence) for resume semantics. +- `capi?: CapiSessionOptions` - Copilot API options. With `model: "auto"`, set `autoTier` to `"efficiency"`, `"balance"`, `"intelligence"`, or `"fast"` to choose a routing preference. `"fast"` is an integrator-only latency preset, not a first-party GitHub Copilot product preference. Requires a runtime with Auto tier support and V2 Auto routing. Omission preserves default behavior. See [Auto tier persistence](../docs/features/session-persistence.md#auto-tier-persistence) for resume semantics. - `reasoningEffort?: "low" | "medium" | "high" | "xhigh" | "max"` - Reasoning effort level for models that support it. Use `listModels()` to check which models support this option. - `tools?: Tool[]` - Custom tools exposed to the CLI. Tools without `handler` are declaration-only and must be resolved via pending tool-call RPCs. - `systemMessage?: SystemMessageConfig` - System message customization (see below) @@ -354,7 +354,7 @@ Change the Auto routing preference without changing the selected model. Pass `nu The runtime does not apply the preference immediately. It records the request and commits it only when a later user turn using the `auto` model successfully obtains a usable model from the provider, so a `pending` status confirms acceptance rather than effect. Only the most recent request survives. -Watch for the outcome through the `session.model_change` event on success or the ephemeral `session.auto_tier_switch_failed` event on failure, and read the authoritative state at any time with `session.rpc.model.getCurrent()`. +Watch for the outcome through the `session.model_change` event on success or the ephemeral `session.auto_tier_switch_failed` event on failure. A failed activation leaves the incumbent effective tier unchanged. Read the authoritative state at any time with `session.rpc.model.getCurrent()`. ```typescript const result = await session.setAutoTier("intelligence"); diff --git a/nodejs/test/client.test.ts b/nodejs/test/client.test.ts index 24adb5cb6e..f8f3848b11 100644 --- a/nodejs/test/client.test.ts +++ b/nodejs/test/client.test.ts @@ -1432,6 +1432,7 @@ describe("CopilotClient", () => { { autoTier: "efficiency" }, { autoTier: "balance" }, { autoTier: "intelligence" }, + { autoTier: "fast" }, { autoTier: "balance", enableWebSocketResponses: false }, ] satisfies (CapiSessionOptions | undefined)[])( "forwards capi options %j in session.create and session.resume", @@ -2676,12 +2677,12 @@ describe("CopilotClient", () => { throw new Error(`Unexpected method: ${method}`); }); - await session.setModel("auto", { autoTier: "intelligence" }); + await session.setModel("auto", { autoTier: "fast" }); expect(spy).toHaveBeenCalledWith("session.model.switchTo", { sessionId: session.sessionId, modelId: "auto", - autoTier: "intelligence", + autoTier: "fast", }); spy.mockRestore(); @@ -2726,7 +2727,7 @@ describe("CopilotClient", () => { return { status: "pending", effectiveAutoTier: "balance", - pendingAutoTier: "intelligence", + pendingAutoTier: "fast", activatingAutoTier: null, supersededAutoTier: null, }; @@ -2734,15 +2735,15 @@ describe("CopilotClient", () => { throw new Error(`Unexpected method: ${method}`); }); - const result = await session.setAutoTier("intelligence"); + const result = await session.setAutoTier("fast"); expect(spy).toHaveBeenCalledWith("session.model.switchAutoTier", { sessionId: session.sessionId, - autoTier: "intelligence", + autoTier: "fast", }); expect(result.status).toBe("pending"); expect(result.effectiveAutoTier).toBe("balance"); - expect(result.pendingAutoTier).toBe("intelligence"); + expect(result.pendingAutoTier).toBe("fast"); expect(result.activatingAutoTier).toBeNull(); spy.mockRestore(); diff --git a/nodejs/test/e2e/auto_tier.e2e.test.ts b/nodejs/test/e2e/auto_tier.e2e.test.ts index 0cb2a1a266..f2b5717e60 100644 --- a/nodejs/test/e2e/auto_tier.e2e.test.ts +++ b/nodejs/test/e2e/auto_tier.e2e.test.ts @@ -5,6 +5,7 @@ import { describe, expect, it } from "vitest"; import { approveAll } from "../../src/index.js"; import { createSdkTestContext } from "./harness/sdkTestContext.js"; +import { getNextEventOfType } from "./harness/sdkTestHelper.js"; /** * The runtime stages an Auto routing preference instead of applying it immediately: a @@ -13,7 +14,7 @@ import { createSdkTestContext } from "./harness/sdkTestContext.js"; * they assert what the runtime actually recorded rather than what the SDK serialized. */ describe("Auto tier switching", async () => { - const { copilotClient: client } = await createSdkTestContext(); + const { copilotClient: client, createClient } = await createSdkTestContext(); it("should stage and reset auto tier preference", async () => { const session = await client.createSession({ @@ -29,10 +30,16 @@ describe("Auto tier switching", async () => { expect((await session.rpc.model.getCurrent()).pendingAutoTier).toBe("efficiency"); // A second request replaces the first and reports the one it displaced. - const superseded = await session.setAutoTier("intelligence"); + const superseded = await session.setAutoTier("fast"); expect(superseded.status).toBe("pending"); - expect(superseded.pendingAutoTier).toBe("intelligence"); + expect(superseded.pendingAutoTier).toBe("fast"); expect(superseded.supersededAutoTier).toBe("efficiency"); + expect((await session.rpc.model.getCurrent()).pendingAutoTier).toBe("fast"); + + const replacedFast = await session.setAutoTier("intelligence"); + expect(replacedFast.status).toBe("pending"); + expect(replacedFast.pendingAutoTier).toBe("intelligence"); + expect(replacedFast.supersededAutoTier).toBe("fast"); expect((await session.rpc.model.getCurrent()).pendingAutoTier).toBe("intelligence"); // Passing null returns the session to provider-default routing. The status is @@ -60,8 +67,8 @@ describe("Auto tier switching", async () => { expect((await session.rpc.model.getCurrent()).pendingAutoTier).toBe("balance"); // Supplying a tier replaces it. - await session.setModel("auto", { autoTier: "intelligence" }); - expect((await session.rpc.model.getCurrent()).pendingAutoTier).toBe("intelligence"); + await session.setModel("auto", { autoTier: "fast" }); + expect((await session.rpc.model.getCurrent()).pendingAutoTier).toBe("fast"); // Supplying null clears it. Omission, a value, and null are three distinct // outcomes, which is why the option cannot collapse to a plain optional field. @@ -70,4 +77,150 @@ describe("Auto tier switching", async () => { await session.disconnect(); }); + + it("should restore and override fast auto tier on cold resume", async () => { + const fastSession = await client.createSession({ + onPermissionRequest: approveAll, + model: "auto", + capi: { autoTier: "fast", enableWebSocketResponses: false }, + }); + const tierlessSession = await client.createSession({ + onPermissionRequest: approveAll, + model: "auto", + capi: { enableWebSocketResponses: false }, + }); + const fastSessionId = fastSession.sessionId; + const tierlessSessionId = tierlessSession.sessionId; + + await fastSession.sendAndWait({ + prompt: "Reply with exactly AUTO_TIER_COLD_RESUME_READY.", + }); + await tierlessSession.sendAndWait({ + prompt: "Reply with exactly AUTO_TIER_TIERLESS_READY.", + }); + expect((await fastSession.rpc.model.getCurrent()).autoTier).toBe("fast"); + expect((await tierlessSession.rpc.model.getCurrent()).autoTier).toBeUndefined(); + + await fastSession.disconnect(); + await tierlessSession.disconnect(); + await client.stop(); + + const restoredClient = createClient(); + const restoredFast = await restoredClient.resumeSession(fastSessionId, { + onPermissionRequest: approveAll, + }); + const restoredTierless = await restoredClient.resumeSession(tierlessSessionId, { + onPermissionRequest: approveAll, + }); + expect((await restoredFast.rpc.model.getCurrent()).autoTier).toBe("fast"); + expect((await restoredTierless.rpc.model.getCurrent()).autoTier).toBeUndefined(); + await restoredFast.disconnect(); + await restoredTierless.disconnect(); + await restoredClient.stop(); + + const overrideClient = createClient(); + const overridden = await overrideClient.resumeSession(fastSessionId, { + onPermissionRequest: approveAll, + model: "auto", + capi: { autoTier: "balance", enableWebSocketResponses: false }, + }); + expect((await overridden.rpc.model.getCurrent()).autoTier).toBe("balance"); + await overridden.disconnect(); + await overrideClient.stop(); + }, 120_000); + + it("should commit fast auto tier after successful turn", async () => { + const session = await client.createSession({ + onPermissionRequest: approveAll, + model: "auto", + capi: { autoTier: "efficiency", enableWebSocketResponses: false }, + }); + + const modelChangePromise = getNextEventOfType(session, "session.model_change"); + const staged = await session.setAutoTier("fast"); + expect(staged.status).toBe("pending"); + expect(staged.effectiveAutoTier).toBe("efficiency"); + expect(staged.pendingAutoTier).toBe("fast"); + + const beforeTurn = await session.rpc.model.getCurrent(); + expect(beforeTurn.autoTier).toBe("efficiency"); + expect(beforeTurn.pendingAutoTier).toBe("fast"); + + await session.sendAndWait({ + prompt: "Reply with exactly AUTO_TIER_FAST_COMMITTED.", + }); + + const modelChange = await modelChangePromise; + expect(modelChange.data.previousModel).toBe("auto"); + expect(modelChange.data.newModel).toBe("auto"); + expect(modelChange.data.previousAutoTier).toBe("efficiency"); + expect(modelChange.data.autoTier).toBe("fast"); + + const committed = await session.rpc.model.getCurrent(); + expect(committed.autoTier).toBe("fast"); + expect(committed.pendingAutoTier).toBeUndefined(); + expect(committed.activatingAutoTier).toBeUndefined(); + + await session.disconnect(); + }, 120_000); + + it("should preserve effective tier when fast activation fails", async () => { + const session = await client.createSession({ + onPermissionRequest: approveAll, + model: "auto", + capi: { autoTier: "efficiency", enableWebSocketResponses: false }, + }); + const sessionId = session.sessionId; + + await session.sendAndWait({ + prompt: "Reply with exactly AUTO_TIER_INITIAL_READY.", + }); + + let fastCommitted = false; + const unsubscribeModelChange = session.on("session.model_change", (event) => { + fastCommitted ||= event.data.autoTier === "fast"; + }); + const failurePromise = getNextEventOfType(session, "session.auto_tier_switch_failed"); + + const staged = await session.setAutoTier("fast"); + expect(staged.status).toBe("pending"); + expect(staged.effectiveAutoTier).toBe("efficiency"); + expect(staged.pendingAutoTier).toBe("fast"); + + await session.sendAndWait({ + prompt: "Reply with exactly AUTO_TIER_FAILURE_RECOVERED.", + }); + + const failure = await failurePromise; + expect(failure.ephemeral).toBe(true); + expect(failure.data.effectiveAutoTier).toBe("efficiency"); + expect(failure.data.requestedAutoTier).toBe("fast"); + expect(failure.data.reason).toBe("request_failed"); + expect(fastCommitted).toBe(false); + + const current = await session.rpc.model.getCurrent(); + expect(current.autoTier).toBe("efficiency"); + expect(current.pendingAutoTier).toBeUndefined(); + expect(current.activatingAutoTier).toBeUndefined(); + + unsubscribeModelChange(); + await session.disconnect(); + await client.stop(); + + const resumedClient = createClient(); + const resumed = await resumedClient.resumeSession(sessionId, { + onPermissionRequest: approveAll, + }); + const resumedCurrent = await resumed.rpc.model.getCurrent(); + expect(resumedCurrent.autoTier).toBe("efficiency"); + expect(resumedCurrent.pendingAutoTier).toBeUndefined(); + expect(resumedCurrent.activatingAutoTier).toBeUndefined(); + + const persisted = await resumed.rpc.eventLog.read({ max: 100, waitMs: 0 }); + expect( + persisted.events.some((event) => event.type === "session.auto_tier_switch_failed") + ).toBe(false); + await resumed.disconnect(); + await resumedClient.stop(); + }, 120_000); }); diff --git a/nodejs/test/session-event-types.test.ts b/nodejs/test/session-event-types.test.ts index d20f3caaf6..cc5cb12bf1 100644 --- a/nodejs/test/session-event-types.test.ts +++ b/nodejs/test/session-event-types.test.ts @@ -133,7 +133,7 @@ type _PermissionRequestedEventStaysAlignedWithSessionEventUnion = _AssertEqual< const _permissionRequestedEventAlignmentCheck: _PermissionRequestedEventStaysAlignedWithSessionEventUnion = true; describe("Session event type exports (#1156)", () => { - it.each(["efficiency", "balance", "intelligence", undefined] satisfies ( + it.each(["efficiency", "balance", "intelligence", "fast", undefined] satisfies ( | AutoTier | undefined )[])("exposes Auto tier %s on start and resume data", (autoTier) => { @@ -169,7 +169,7 @@ describe("Session event type exports (#1156)", () => { (reason) => { const data: AutoTierSwitchFailedData = { reason, - requestedAutoTier: "intelligence", + requestedAutoTier: "fast", effectiveAutoTier: "balance", }; const event: AutoTierSwitchFailedEvent = { @@ -186,7 +186,7 @@ describe("Session event type exports (#1156)", () => { const asSessionEvent: SessionEvent = event; expect(asSessionEvent.type).toBe("session.auto_tier_switch_failed"); expect(data.reason).toBe(reason); - expect(data.requestedAutoTier).toBe("intelligence"); + expect(data.requestedAutoTier).toBe("fast"); } ); diff --git a/python/README.md b/python/README.md index 3b7d937c44..71a6e8ae78 100644 --- a/python/README.md +++ b/python/README.md @@ -306,7 +306,7 @@ finally: These are passed as keyword arguments to `create_session()`: - `model` (str): Model to use ("gpt-5", "claude-sonnet-4.5", etc.). **Required when using custom provider.** -- `capi` (CapiSessionOptions): Copilot API options. With `model="auto"`, set `auto_tier` to `"efficiency"`, `"balance"`, or `"intelligence"` to choose a routing preference. Requires a runtime with Auto tier support and V2 Auto routing. Omission preserves default behavior. See [Auto tier persistence](../docs/features/session-persistence.md#auto-tier-persistence) for resume semantics. +- `capi` (CapiSessionOptions): Copilot API options. With `model="auto"`, set `auto_tier` to `"efficiency"`, `"balance"`, `"intelligence"`, or `"fast"` to choose a routing preference. `"fast"` is an integrator-only latency preset, not a first-party GitHub Copilot product preference. Requires a runtime with Auto tier support and V2 Auto routing. Omission preserves default behavior. See [Auto tier persistence](../docs/features/session-persistence.md#auto-tier-persistence) for resume semantics. - `reasoning_effort` (str): Reasoning effort level for models that support it ("low", "medium", "high", "xhigh", "max"). Use `list_models()` to check which models support this option. - `session_id` (str): Custom session ID - `tools` (list): Custom tools exposed to the CLI. Tools with `handler=None` are declaration-only and must be resolved via pending tool-call RPCs. @@ -492,7 +492,7 @@ async def lookup_issue(params: LookupParams) -> str: Change the Auto routing preference without changing the selected model. The runtime does not apply the preference immediately: it records the request and commits it only when a later user turn using the `auto` model successfully obtains a usable model from the provider, so a `pending` status confirms acceptance rather than effect. Only the most recent request survives. -Watch for the outcome through the `session.model_change` event on success or the ephemeral `session.auto_tier_switch_failed` event on failure. Read the authoritative committed, pending, and activating preferences at any time through the session's `model.getCurrent` RPC method. +Watch for the outcome through the `session.model_change` event on success or the ephemeral `session.auto_tier_switch_failed` event on failure. A failed activation leaves the incumbent effective tier unchanged. Read the authoritative committed, pending, and activating preferences at any time through the session's `model.getCurrent` RPC method. ```python result = await session.set_auto_tier("intelligence") diff --git a/python/copilot/client.py b/python/copilot/client.py index d8d2f7e3f4..4d9d4dfb76 100644 --- a/python/copilot/client.py +++ b/python/copilot/client.py @@ -2385,8 +2385,10 @@ async def create_session( working_directory: Working directory for the session. provider: Provider configuration for Azure or custom endpoints. capi: CAPI provider-scoped options. Set ``auto_tier`` to ``efficiency``, - ``balance``, or ``intelligence`` to select an Auto routing preference - on a runtime with Auto tier support. WebSocket transport is the + ``balance``, ``intelligence``, or ``fast`` to select an Auto routing + preference on a runtime with Auto tier support. ``fast`` is an + integrator-only latency preset, not a first-party GitHub Copilot + product preference. WebSocket transport is the default for the CAPI Responses API whenever the model advertises the ``ws:/responses`` endpoint. Set ``enable_web_socket_responses=False`` to force the HTTP diff --git a/python/copilot/session.py b/python/copilot/session.py index f298ce7802..4149aa5893 100644 --- a/python/copilot/session.py +++ b/python/copilot/session.py @@ -180,7 +180,7 @@ def _capabilities_to_dict(caps: ModelCapabilitiesOverride) -> dict: ReasoningEffort = Literal["low", "medium", "high", "xhigh", "max"] ReasoningSummary = Literal["none", "concise", "detailed"] ContextTier = Literal["default", "long_context"] -AutoTier = Literal["efficiency", "balance", "intelligence"] +AutoTier = Literal["efficiency", "balance", "intelligence", "fast"] SessionFsConventions = Literal["posix", "windows"] diff --git a/python/e2e/test_auto_tier_e2e.py b/python/e2e/test_auto_tier_e2e.py index 5a878c4655..d5fb12cc5a 100644 --- a/python/e2e/test_auto_tier_e2e.py +++ b/python/e2e/test_auto_tier_e2e.py @@ -12,11 +12,16 @@ import pytest -from copilot.rpc import ModelSwitchAutoTierStatus +from copilot import CopilotClient, RuntimeConnection +from copilot.rpc import EventLogReadRequest, ModelSwitchAutoTierStatus from copilot.session import PermissionHandler -from copilot.session_events import AutoTier +from copilot.session_events import ( + AutoTier, + SessionAutoTierSwitchFailedData, + SessionModelChangeData, +) -from .testharness import E2ETestContext +from .testharness import DEFAULT_GITHUB_TOKEN, E2ETestContext, get_next_event_of_type pytestmark = pytest.mark.asyncio(loop_scope="module") @@ -25,6 +30,15 @@ async def pending_auto_tier(session) -> AutoTier | None: return (await session.rpc.model.get_current()).pending_auto_tier +def create_auto_client(ctx: E2ETestContext) -> CopilotClient: + return CopilotClient( + connection=RuntimeConnection.for_stdio(path=ctx.cli_path), + working_directory=ctx.work_dir, + env=ctx.get_env(), + github_token=DEFAULT_GITHUB_TOKEN, + ) + + class TestAutoTier: async def test_should_stage_and_reset_auto_tier_preference(self, ctx: E2ETestContext): session = await ctx.client.create_session( @@ -40,10 +54,16 @@ async def test_should_stage_and_reset_auto_tier_preference(self, ctx: E2ETestCon assert await pending_auto_tier(session) == AutoTier.EFFICIENCY # A second request replaces the first and reports the one it displaced. - superseded = await session.set_auto_tier("intelligence") + superseded = await session.set_auto_tier("fast") assert superseded.status == ModelSwitchAutoTierStatus.PENDING - assert superseded.pending_auto_tier == AutoTier.INTELLIGENCE + assert superseded.pending_auto_tier == AutoTier.FAST assert superseded.superseded_auto_tier == AutoTier.EFFICIENCY + assert await pending_auto_tier(session) == AutoTier.FAST + + replaced_fast = await session.set_auto_tier("intelligence") + assert replaced_fast.status == ModelSwitchAutoTierStatus.PENDING + assert replaced_fast.pending_auto_tier == AutoTier.INTELLIGENCE + assert replaced_fast.superseded_auto_tier == AutoTier.FAST assert await pending_auto_tier(session) == AutoTier.INTELLIGENCE # Passing None returns the session to provider-default routing. The status is @@ -70,8 +90,8 @@ async def test_should_preserve_auto_tier_when_set_model_omits_it(self, ctx: E2ET assert await pending_auto_tier(session) == AutoTier.BALANCE # Supplying a tier replaces it. - await session.set_model("auto", auto_tier="intelligence") - assert await pending_auto_tier(session) == AutoTier.INTELLIGENCE + await session.set_model("auto", auto_tier="fast") + assert await pending_auto_tier(session) == AutoTier.FAST # Supplying None clears it. Omission, a value, and None are three distinct # outcomes, which is why the argument cannot collapse to a plain optional. @@ -79,3 +99,156 @@ async def test_should_preserve_auto_tier_when_set_model_omits_it(self, ctx: E2ET assert await pending_auto_tier(session) is None finally: await session.disconnect() + + async def test_should_restore_and_override_fast_auto_tier_on_cold_resume( + self, ctx: E2ETestContext + ): + client = create_auto_client(ctx) + fast_session = await client.create_session( + on_permission_request=PermissionHandler.approve_all, + model="auto", + capi={"auto_tier": "fast", "enable_web_socket_responses": False}, + ) + tierless_session = await client.create_session( + on_permission_request=PermissionHandler.approve_all, + model="auto", + capi={"enable_web_socket_responses": False}, + ) + fast_session_id = fast_session.session_id + tierless_session_id = tierless_session.session_id + + await fast_session.send_and_wait("Reply with exactly AUTO_TIER_COLD_RESUME_READY.") + await tierless_session.send_and_wait("Reply with exactly AUTO_TIER_TIERLESS_READY.") + assert (await fast_session.rpc.model.get_current()).auto_tier == AutoTier.FAST + assert (await tierless_session.rpc.model.get_current()).auto_tier is None + + await fast_session.disconnect() + await tierless_session.disconnect() + await client.stop() + + restored_client = create_auto_client(ctx) + restored_fast = await restored_client.resume_session( + fast_session_id, + on_permission_request=PermissionHandler.approve_all, + ) + restored_tierless = await restored_client.resume_session( + tierless_session_id, + on_permission_request=PermissionHandler.approve_all, + ) + assert (await restored_fast.rpc.model.get_current()).auto_tier == AutoTier.FAST + assert (await restored_tierless.rpc.model.get_current()).auto_tier is None + await restored_fast.disconnect() + await restored_tierless.disconnect() + await restored_client.stop() + + override_client = create_auto_client(ctx) + overridden = await override_client.resume_session( + fast_session_id, + on_permission_request=PermissionHandler.approve_all, + model="auto", + capi={"auto_tier": "balance", "enable_web_socket_responses": False}, + ) + assert (await overridden.rpc.model.get_current()).auto_tier == AutoTier.BALANCE + await overridden.disconnect() + await override_client.stop() + + async def test_should_commit_fast_auto_tier_after_successful_turn(self, ctx: E2ETestContext): + client = create_auto_client(ctx) + session = await client.create_session( + on_permission_request=PermissionHandler.approve_all, + model="auto", + capi={"auto_tier": "efficiency", "enable_web_socket_responses": False}, + ) + try: + model_change_task = get_next_event_of_type(session, "session.model_change") + staged = await session.set_auto_tier("fast") + assert staged.status == ModelSwitchAutoTierStatus.PENDING + assert staged.effective_auto_tier == AutoTier.EFFICIENCY + assert staged.pending_auto_tier == AutoTier.FAST + + before_turn = await session.rpc.model.get_current() + assert before_turn.auto_tier == AutoTier.EFFICIENCY + assert before_turn.pending_auto_tier == AutoTier.FAST + + await session.send_and_wait("Reply with exactly AUTO_TIER_FAST_COMMITTED.") + + model_change = await model_change_task + assert isinstance(model_change.data, SessionModelChangeData) + assert model_change.data.previous_model == "auto" + assert model_change.data.new_model == "auto" + assert model_change.data.previous_auto_tier == AutoTier.EFFICIENCY + assert model_change.data.auto_tier == AutoTier.FAST + + committed = await session.rpc.model.get_current() + assert committed.auto_tier == AutoTier.FAST + assert committed.pending_auto_tier is None + assert committed.activating_auto_tier is None + finally: + await session.disconnect() + await client.stop() + + async def test_should_preserve_effective_tier_when_fast_activation_fails( + self, ctx: E2ETestContext + ): + client = create_auto_client(ctx) + session = await client.create_session( + on_permission_request=PermissionHandler.approve_all, + model="auto", + capi={"auto_tier": "efficiency", "enable_web_socket_responses": False}, + ) + session_id = session.session_id + + await session.send_and_wait("Reply with exactly AUTO_TIER_INITIAL_READY.") + + fast_committed = False + + def on_event(event): + nonlocal fast_committed + if isinstance(event.data, SessionModelChangeData): + fast_committed |= event.data.auto_tier == AutoTier.FAST + + unsubscribe = session.on(on_event) + failure_task = get_next_event_of_type(session, "session.auto_tier_switch_failed") + + staged = await session.set_auto_tier("fast") + assert staged.status == ModelSwitchAutoTierStatus.PENDING + assert staged.effective_auto_tier == AutoTier.EFFICIENCY + assert staged.pending_auto_tier == AutoTier.FAST + + await session.send_and_wait("Reply with exactly AUTO_TIER_FAILURE_RECOVERED.") + + failure = await failure_task + assert failure.ephemeral is True + assert isinstance(failure.data, SessionAutoTierSwitchFailedData) + assert failure.data.effective_auto_tier == AutoTier.EFFICIENCY + assert failure.data.requested_auto_tier == AutoTier.FAST + assert failure.data.reason.value == "request_failed" + assert fast_committed is False + + current = await session.rpc.model.get_current() + assert current.auto_tier == AutoTier.EFFICIENCY + assert current.pending_auto_tier is None + assert current.activating_auto_tier is None + + unsubscribe() + await session.disconnect() + await client.stop() + + resumed_client = create_auto_client(ctx) + try: + resumed = await resumed_client.resume_session( + session_id, + on_permission_request=PermissionHandler.approve_all, + ) + resumed_current = await resumed.rpc.model.get_current() + assert resumed_current.auto_tier == AutoTier.EFFICIENCY + assert resumed_current.pending_auto_tier is None + assert resumed_current.activating_auto_tier is None + + persisted = await resumed.rpc.event_log.read(EventLogReadRequest(max=100, wait_ms=0)) + assert not any( + event.type.value == "session.auto_tier_switch_failed" for event in persisted.events + ) + await resumed.disconnect() + finally: + await resumed_client.stop() diff --git a/python/test_client.py b/python/test_client.py index 2e3868ef1c..775d54a5db 100644 --- a/python/test_client.py +++ b/python/test_client.py @@ -1246,6 +1246,12 @@ async def mock_request(method, params, **kwargs): {"autoTier": "intelligence"}, {"autoTier": "intelligence"}, ), + ( + {"auto_tier": "fast"}, + {"auto_tier": "fast"}, + {"autoTier": "fast"}, + {"autoTier": "fast"}, + ), ( {"auto_tier": "balance", "enable_web_socket_responses": False}, {"auto_tier": "balance", "enable_web_socket_responses": True}, @@ -2673,10 +2679,10 @@ async def mock_request(method, params, **kwargs): return await original_request(method, params, **kwargs) client._client.request = mock_request - await session.set_model("auto", auto_tier="intelligence") + await session.set_model("auto", auto_tier="fast") assert captured["session.model.switchTo"]["sessionId"] == session.session_id assert captured["session.model.switchTo"]["modelId"] == "auto" - assert captured["session.model.switchTo"]["autoTier"] == "intelligence" + assert captured["session.model.switchTo"]["autoTier"] == "fast" finally: await client.force_stop() @@ -2711,7 +2717,7 @@ async def mock_request(method, params, **kwargs): class TestSetAutoTier: @pytest.mark.asyncio - @pytest.mark.parametrize("auto_tier", ["efficiency", "balance", "intelligence", None]) + @pytest.mark.parametrize("auto_tier", ["efficiency", "balance", "intelligence", "fast", None]) async def test_set_auto_tier_sends_correct_rpc(self, auto_tier): client = CopilotClient(connection=RuntimeConnection.for_stdio(path=CLI_PATH)) await client.start() @@ -2766,16 +2772,16 @@ async def test_set_auto_tier_accepts_the_enum_it_returns(self): async def mock_request(method, params, **kwargs): captured[method] = params if method == "session.model.switchAutoTier": - return {"status": "pending", "effectiveAutoTier": "intelligence"} + return {"status": "pending", "effectiveAutoTier": "fast"} return await original_request(method, params, **kwargs) client._client.request = mock_request - await session.set_auto_tier(AutoTierEnum.INTELLIGENCE) + await session.set_auto_tier(AutoTierEnum.FAST) params = captured["session.model.switchAutoTier"] # The value must be a plain string; the JSON-RPC encoder cannot # serialize an enum. - assert params["autoTier"] == "intelligence" + assert params["autoTier"] == "fast" assert isinstance(params["autoTier"], str) json.dumps(params) finally: diff --git a/python/test_event_forward_compatibility.py b/python/test_event_forward_compatibility.py index 65e39a80ba..4d60d63b87 100644 --- a/python/test_event_forward_compatibility.py +++ b/python/test_event_forward_compatibility.py @@ -40,7 +40,7 @@ class TestEventForwardCompatibility: """Test forward compatibility for unknown event types.""" @pytest.mark.parametrize("event_type", ["session.start", "session.resume"]) - @pytest.mark.parametrize("tier", ["efficiency", "balance", "intelligence", None]) + @pytest.mark.parametrize("tier", ["efficiency", "balance", "intelligence", "fast", None]) def test_auto_tier_lifecycle_events_round_trip(self, event_type, tier): timestamp = "2026-08-28T00:00:00Z" data = ( @@ -87,7 +87,7 @@ def test_auto_tier_switch_failed_event_decodes_every_reason(self, reason): "type": "session.auto_tier_switch_failed", "data": { "effectiveAutoTier": "balance", - "requestedAutoTier": "intelligence", + "requestedAutoTier": "fast", "reason": reason, }, } @@ -95,7 +95,7 @@ def test_auto_tier_switch_failed_event_decodes_every_reason(self, reason): assert isinstance(event.data, SessionAutoTierSwitchFailedData) assert event.data.reason == AutoTierSwitchFailureReason(reason) assert event.data.effective_auto_tier == AutoTier.BALANCE - assert event.data.requested_auto_tier == AutoTier.INTELLIGENCE + assert event.data.requested_auto_tier == AutoTier.FAST def test_auto_tier_switch_failed_event_allows_null_requested_tier(self): # A null requested tier means the attempt to return to provider-default diff --git a/rust/README.md b/rust/README.md index d50dc96ec8..11d9637b22 100644 --- a/rust/README.md +++ b/rust/README.md @@ -365,9 +365,12 @@ next credential-consuming operation; there is no background refresh timer. ### Auto routing tiers Use `CapiSessionOptions::with_auto_tier` to select `AutoTier::Efficiency`, -`AutoTier::Balance`, or `AutoTier::Intelligence`. This option is meaningful only -with model `auto` (Auto mode V2). +`AutoTier::Balance`, `AutoTier::Intelligence`, or `AutoTier::Fast`. This option +is meaningful only with model `auto` (Auto mode V2). It requires a runtime version that supports `capi.autoTier`. +`AutoTier::Fast` is an integrator-only latency preset, not a first-party +GitHub Copilot product preference — the SDK does not decide Fast eligibility +or apply it implicitly. ```rust use github_copilot_sdk::{AutoTier, CapiSessionOptions, SessionConfig}; @@ -388,7 +391,7 @@ resume succeeds; it cannot change a turn that is already in flight. The SDK does Change the Auto routing preference without changing the selected model. The runtime does not apply the preference immediately: it records the request and commits it only when a later user turn using the `auto` model successfully obtains a usable model from the provider, so a `pending` status confirms acceptance rather than effect. Only the most recent request survives. -Watch for the outcome through the `session.model_change` event on success or the ephemeral `session.auto_tier_switch_failed` event on failure. Read the authoritative committed, pending, and activating preferences at any time through the session's `model.getCurrent` RPC method. +Watch for the outcome through the `session.model_change` event on success or the ephemeral `session.auto_tier_switch_failed` event on failure. A failed activation leaves the incumbent effective tier unchanged. Read the authoritative committed, pending, and activating preferences at any time through the session's `model.getCurrent` RPC method. ```rust,ignore use github_copilot_sdk::{AutoTier, ModelSwitchAutoTierStatus}; diff --git a/rust/src/types.rs b/rust/src/types.rs index 8e4051d287..6e3af9273f 100644 --- a/rust/src/types.rs +++ b/rust/src/types.rs @@ -24,6 +24,9 @@ use crate::generated::api_types::{CurrentToolMetadata, OpenCanvasInstance}; /// Acknowledgement and Auto preference snapshot returned by an Auto tier switch. pub use crate::generated::api_types::{ModelSwitchAutoTierResult, ModelSwitchAutoTierStatus}; /// Routing tier for the `auto` model with Auto mode V2. +/// +/// [`AutoTier::Fast`] is an integrator-only latency preset, not a first-party +/// GitHub Copilot product preference. pub use crate::generated::session_events::AutoTier; use crate::generated::session_events::ReasoningSummary; /// Context window tier for models that support tiered context windows. @@ -7644,6 +7647,7 @@ mod tests { (AutoTier::Efficiency, "efficiency"), (AutoTier::Balance, "balance"), (AutoTier::Intelligence, "intelligence"), + (AutoTier::Fast, "fast"), ] { let exported: crate::AutoTier = tier.clone(); let capi = CapiSessionOptions::new().with_auto_tier(exported); diff --git a/rust/tests/api_types_test.rs b/rust/tests/api_types_test.rs index 4d51340925..94f233f19a 100644 --- a/rust/tests/api_types_test.rs +++ b/rust/tests/api_types_test.rs @@ -22,6 +22,7 @@ fn session_events_deserialize_auto_tier() { (Some(AutoTier::Efficiency), Some("efficiency")), (Some(AutoTier::Balance), Some("balance")), (Some(AutoTier::Intelligence), Some("intelligence")), + (Some(AutoTier::Fast), Some("fast")), (None, None), ] { let mut wire = serde_json::json!({ @@ -279,6 +280,7 @@ fn switch_auto_tier_request_serializes_each_tier() { (AutoTier::Efficiency, "efficiency"), (AutoTier::Balance, "balance"), (AutoTier::Intelligence, "intelligence"), + (AutoTier::Fast, "fast"), ] { let request = ModelSwitchAutoTierRequest { auto_tier: Some(tier), @@ -294,16 +296,17 @@ fn switch_auto_tier_result_deserializes_full_snapshot() { let result: ModelSwitchAutoTierResult = serde_json::from_value(serde_json::json!({ "status": "pending", "effectiveAutoTier": "balance", - "pendingAutoTier": "intelligence", + "pendingAutoTier": "fast", "activatingAutoTier": null, - "supersededAutoTier": null + "supersededAutoTier": "efficiency" })) .unwrap(); assert_eq!(result.status, ModelSwitchAutoTierStatus::Pending); assert_eq!(result.effective_auto_tier, Some(AutoTier::Balance)); - assert_eq!(result.pending_auto_tier, Some(AutoTier::Intelligence)); + assert_eq!(result.pending_auto_tier, Some(AutoTier::Fast)); assert_eq!(result.activating_auto_tier, None); + assert_eq!(result.superseded_auto_tier, Some(AutoTier::Efficiency)); } #[test] diff --git a/rust/tests/e2e/auto_tier.rs b/rust/tests/e2e/auto_tier.rs index 85c70dd460..baaa1698c3 100644 --- a/rust/tests/e2e/auto_tier.rs +++ b/rust/tests/e2e/auto_tier.rs @@ -1,9 +1,20 @@ -use github_copilot_sdk::SetModelOptions; -use github_copilot_sdk::rpc::ModelSwitchAutoTierStatus; +use std::sync::Arc; +use std::time::Duration; + +use github_copilot_sdk::handler::ApproveAllHandler; +use github_copilot_sdk::rpc::{EventLogReadRequest, ModelSwitchAutoTierStatus}; use github_copilot_sdk::session::Session; -use github_copilot_sdk::session_events::AutoTier; +use github_copilot_sdk::session_events::{ + AutoTier, AutoTierSwitchFailureReason, SessionAutoTierSwitchFailedData, SessionEventType, + SessionModelChangeData, +}; +use github_copilot_sdk::{ + CapiSessionOptions, MessageOptions, ResumeSessionConfig, SessionConfig, SessionId, + SetModelOptions, +}; +use serde_json::json; -use super::support::with_dedicated_e2e_context; +use super::support::{DEFAULT_TEST_TOKEN, wait_for_event, with_dedicated_e2e_context}; const MODEL_ID: &str = "auto"; @@ -24,6 +35,32 @@ async fn pending_auto_tier(session: &Session) -> Option { .pending_auto_tier } +fn auto_session_config(tier: Option) -> SessionConfig { + let capi = CapiSessionOptions::new().with_enable_web_socket_responses(false); + let capi = match tier { + Some(tier) => capi.with_auto_tier(tier), + None => capi, + }; + SessionConfig::default() + .with_model(MODEL_ID) + .with_capi(capi) + .with_permission_handler(Arc::new(ApproveAllHandler)) + .with_github_token(DEFAULT_TEST_TOKEN) +} + +fn auto_resume_config(session_id: SessionId) -> ResumeSessionConfig { + ResumeSessionConfig::new(session_id) + .with_permission_handler(Arc::new(ApproveAllHandler)) + .with_github_token(DEFAULT_TEST_TOKEN) +} + +async fn send_prompt(session: &Session, prompt: &str) { + session + .send_and_wait(MessageOptions::new(prompt).with_wait_timeout(Duration::from_secs(120))) + .await + .expect("send prompt"); +} + #[tokio::test] async fn should_stage_and_reset_auto_tier_preference() { with_dedicated_e2e_context( @@ -53,12 +90,24 @@ async fn should_stage_and_reset_auto_tier_preference() { // A second request replaces the first and reports the one it displaced. let superseded = session - .set_auto_tier(Some(AutoTier::Intelligence)) + .set_auto_tier(Some(AutoTier::Fast)) .await - .expect("stage intelligence"); + .expect("stage fast"); assert_eq!(superseded.status, ModelSwitchAutoTierStatus::Pending); - assert_eq!(superseded.pending_auto_tier, Some(AutoTier::Intelligence)); + assert_eq!(superseded.pending_auto_tier, Some(AutoTier::Fast)); assert_eq!(superseded.superseded_auto_tier, Some(AutoTier::Efficiency)); + assert_eq!(pending_auto_tier(&session).await, Some(AutoTier::Fast)); + + let replaced_fast = session + .set_auto_tier(Some(AutoTier::Intelligence)) + .await + .expect("stage intelligence"); + assert_eq!(replaced_fast.status, ModelSwitchAutoTierStatus::Pending); + assert_eq!( + replaced_fast.pending_auto_tier, + Some(AutoTier::Intelligence) + ); + assert_eq!(replaced_fast.superseded_auto_tier, Some(AutoTier::Fast)); assert_eq!( pending_auto_tier(&session).await, Some(AutoTier::Intelligence) @@ -111,14 +160,11 @@ async fn should_preserve_auto_tier_when_set_model_omits_it() { session .set_model( MODEL_ID, - Some(SetModelOptions::default().with_auto_tier(AutoTier::Intelligence)), + Some(SetModelOptions::default().with_auto_tier(AutoTier::Fast)), ) .await .expect("set model with a tier"); - assert_eq!( - pending_auto_tier(&session).await, - Some(AutoTier::Intelligence) - ); + assert_eq!(pending_auto_tier(&session).await, Some(AutoTier::Fast)); // Requesting a reset clears it. Omission, a tier, and a reset are three // distinct outcomes, which `AutoTierPreference` makes explicit. @@ -138,3 +184,311 @@ async fn should_preserve_auto_tier_when_set_model_omits_it() { ) .await; } + +#[tokio::test] +async fn should_restore_and_override_fast_auto_tier_on_cold_resume() { + with_dedicated_e2e_context( + "auto_tier", + "should_restore_and_override_fast_auto_tier_on_cold_resume", + |ctx| { + Box::pin(async move { + ctx.set_default_copilot_user(); + let client = ctx.start_client().await; + let fast_session = client + .create_session(auto_session_config(Some(AutoTier::Fast))) + .await + .expect("create fast session"); + let tierless_session = client + .create_session(auto_session_config(None)) + .await + .expect("create tierless session"); + let fast_session_id = fast_session.id().clone(); + let tierless_session_id = tierless_session.id().clone(); + + send_prompt( + &fast_session, + "Reply with exactly AUTO_TIER_COLD_RESUME_READY.", + ) + .await; + send_prompt( + &tierless_session, + "Reply with exactly AUTO_TIER_TIERLESS_READY.", + ) + .await; + + let fast_current = fast_session + .rpc() + .model() + .get_current() + .await + .expect("get fast current model"); + assert_eq!(fast_current.auto_tier, Some(AutoTier::Fast)); + let tierless_current = tierless_session + .rpc() + .model() + .get_current() + .await + .expect("get tierless current model"); + assert_eq!(tierless_current.auto_tier, None); + + fast_session + .disconnect() + .await + .expect("disconnect fast session"); + tierless_session + .disconnect() + .await + .expect("disconnect tierless session"); + client.stop().await.expect("stop initial client"); + + let restored_client = ctx.start_client().await; + let restored_fast = restored_client + .resume_session(auto_resume_config(fast_session_id.clone())) + .await + .expect("resume fast session"); + let restored_tierless = restored_client + .resume_session(auto_resume_config(tierless_session_id)) + .await + .expect("resume tierless session"); + let restored_fast_current = restored_fast + .rpc() + .model() + .get_current() + .await + .expect("get restored fast current model"); + assert_eq!(restored_fast_current.auto_tier, Some(AutoTier::Fast)); + let restored_tierless_current = restored_tierless + .rpc() + .model() + .get_current() + .await + .expect("get restored tierless current model"); + assert_eq!(restored_tierless_current.auto_tier, None); + + restored_fast + .disconnect() + .await + .expect("disconnect restored fast session"); + restored_tierless + .disconnect() + .await + .expect("disconnect restored tierless session"); + restored_client.stop().await.expect("stop restored client"); + + let override_client = ctx.start_client().await; + let overridden = override_client + .resume_session( + auto_resume_config(fast_session_id) + .with_model(MODEL_ID) + .with_capi( + CapiSessionOptions::new() + .with_auto_tier(AutoTier::Balance) + .with_enable_web_socket_responses(false), + ), + ) + .await + .expect("resume session with balance override"); + let overridden_current = overridden + .rpc() + .model() + .get_current() + .await + .expect("get overridden current model"); + assert_eq!(overridden_current.auto_tier, Some(AutoTier::Balance)); + + overridden + .disconnect() + .await + .expect("disconnect overridden session"); + override_client.stop().await.expect("stop override client"); + }) + }, + ) + .await; +} + +#[tokio::test] +async fn should_commit_fast_auto_tier_after_successful_turn() { + with_dedicated_e2e_context( + "auto_tier", + "should_commit_fast_auto_tier_after_successful_turn", + |ctx| { + Box::pin(async move { + ctx.set_default_copilot_user(); + let client = ctx.start_client().await; + let session = client + .create_session(auto_session_config(Some(AutoTier::Efficiency))) + .await + .expect("create session"); + + let model_change = wait_for_event( + session.subscribe(), + "Fast Auto tier model change", + |event| { + event.parsed_type() == SessionEventType::SessionModelChange + && event + .typed_data::() + .is_some_and(|data| data.auto_tier == Some(AutoTier::Fast)) + }, + ); + + let staged = session + .set_auto_tier(Some(AutoTier::Fast)) + .await + .expect("stage fast"); + assert_eq!(staged.status, ModelSwitchAutoTierStatus::Pending); + assert_eq!(staged.effective_auto_tier, Some(AutoTier::Efficiency)); + assert_eq!(staged.pending_auto_tier, Some(AutoTier::Fast)); + + let before_turn = session + .rpc() + .model() + .get_current() + .await + .expect("get current model before turn"); + assert_eq!(before_turn.auto_tier, Some(AutoTier::Efficiency)); + assert_eq!(before_turn.pending_auto_tier, Some(AutoTier::Fast)); + + send_prompt(&session, "Reply with exactly AUTO_TIER_FAST_COMMITTED.").await; + + let model_change = model_change.await; + let data = model_change + .typed_data::() + .expect("typed model change data"); + assert_eq!(data.previous_model.as_deref(), Some(MODEL_ID)); + assert_eq!(data.new_model, MODEL_ID); + assert_eq!(data.previous_auto_tier, Some(AutoTier::Efficiency)); + assert_eq!(data.auto_tier, Some(AutoTier::Fast)); + + let committed = session + .rpc() + .model() + .get_current() + .await + .expect("get current model after turn"); + assert_eq!(committed.auto_tier, Some(AutoTier::Fast)); + assert_eq!(committed.pending_auto_tier, None); + assert_eq!(committed.activating_auto_tier, None); + + session.disconnect().await.expect("disconnect session"); + client.stop().await.expect("stop client"); + }) + }, + ) + .await; +} + +#[tokio::test] +async fn should_preserve_effective_tier_when_fast_activation_fails() { + with_dedicated_e2e_context( + "auto_tier", + "should_preserve_effective_tier_when_fast_activation_fails", + |ctx| { + Box::pin(async move { + ctx.set_default_copilot_user(); + let client = ctx.start_client().await; + let session = client + .create_session(auto_session_config(Some(AutoTier::Efficiency))) + .await + .expect("create session"); + let session_id = session.id().clone(); + send_prompt(&session, "Reply with exactly AUTO_TIER_INITIAL_READY.").await; + + let failure = wait_for_event( + session.subscribe(), + "Fast Auto tier activation failure", + |event| event.parsed_type() == SessionEventType::SessionAutoTierSwitchFailed, + ); + let mut model_changes = session.subscribe(); + + let staged = session + .set_auto_tier(Some(AutoTier::Fast)) + .await + .expect("stage fast"); + assert_eq!(staged.status, ModelSwitchAutoTierStatus::Pending); + assert_eq!(staged.effective_auto_tier, Some(AutoTier::Efficiency)); + assert_eq!(staged.pending_auto_tier, Some(AutoTier::Fast)); + + send_prompt(&session, "Reply with exactly AUTO_TIER_FAILURE_RECOVERED.").await; + + let failure = failure.await; + assert_eq!(failure.ephemeral, Some(true)); + let data = failure + .typed_data::() + .expect("typed Auto tier failure data"); + assert_eq!(data.effective_auto_tier, Some(AutoTier::Efficiency)); + assert_eq!(data.requested_auto_tier, Some(AutoTier::Fast)); + assert_eq!(data.reason, AutoTierSwitchFailureReason::RequestFailed); + + while let Ok(Ok(event)) = + tokio::time::timeout(Duration::from_millis(25), model_changes.recv()).await + { + assert!( + event.parsed_type() != SessionEventType::SessionModelChange + || !event + .typed_data::() + .is_some_and(|data| data.auto_tier == Some(AutoTier::Fast)), + "Fast tier committed after failed activation" + ); + } + + let current = session + .rpc() + .model() + .get_current() + .await + .expect("get current model after failure"); + assert_eq!(current.auto_tier, Some(AutoTier::Efficiency)); + assert_eq!(current.pending_auto_tier, None); + assert_eq!(current.activating_auto_tier, None); + + session.disconnect().await.expect("disconnect session"); + client.stop().await.expect("stop initial client"); + + let resumed_client = ctx.start_client().await; + let resumed = resumed_client + .resume_session(auto_resume_config(session_id)) + .await + .expect("resume session"); + let resumed_current = resumed + .rpc() + .model() + .get_current() + .await + .expect("get current model after resume"); + assert_eq!(resumed_current.auto_tier, Some(AutoTier::Efficiency)); + assert_eq!(resumed_current.pending_auto_tier, None); + assert_eq!(resumed_current.activating_auto_tier, None); + + let persisted = resumed + .rpc() + .event_log() + .read(EventLogReadRequest { + agent_ids: None, + agent_scope: None, + cursor: None, + direction: None, + include_ephemeral: Some(false), + max: Some(100), + types: Some(json!("*")), + wait_ms: Some(0), + }) + .await + .expect("read event log"); + assert!( + persisted.events.iter().all(|event| { + event.parsed_type() != SessionEventType::SessionAutoTierSwitchFailed + }), + "ephemeral failure event was replayed after cold resume" + ); + + resumed + .disconnect() + .await + .expect("disconnect resumed session"); + resumed_client.stop().await.expect("stop resumed client"); + }) + }, + ) + .await; +} diff --git a/test/harness/replayingCapiProxy.test.ts b/test/harness/replayingCapiProxy.test.ts index 062fe89ae9..80bcd6ed24 100644 --- a/test/harness/replayingCapiProxy.test.ts +++ b/test/harness/replayingCapiProxy.test.ts @@ -1698,5 +1698,55 @@ Always include PINEAPPLE_COCONUT_42. await proxy.stop(); } }); + + test("returns cached Auto responses in order", async () => { + const cachePath = path.join(tempDir, "cache.yaml"); + const autoResponses = [ + { + body: { + session_token: "first-token", + selected_model: { id: "test-model" }, + }, + }, + { + statusCode: 500, + body: { + session_token: "unused-token", + selected_model: { id: "unused-model" }, + }, + }, + ]; + await writeFile( + cachePath, + yaml.stringify({ + models: ["test-model"], + autoResponses, + conversations: [], + } satisfies NormalizedData), + ); + + const proxy = new ReplayingCapiProxy( + "http://localhost:9999", + cachePath, + workDir, + ); + const proxyUrl = await proxy.start(); + + try { + const success = await makeRequest(proxyUrl, "/auto", { + body: { prompt: "first" }, + }); + expect(success.status).toBe(200); + expect(JSON.parse(success.body)).toEqual(autoResponses[0].body); + + const failure = await makeRequest(proxyUrl, "/auto", { + body: { prompt: "second" }, + }); + expect(failure.status).toBe(500); + expect(JSON.parse(failure.body)).toEqual(autoResponses[1].body); + } finally { + await proxy.stop(); + } + }); }); }); diff --git a/test/harness/replayingCapiProxy.ts b/test/harness/replayingCapiProxy.ts index 9bcafcdad4..4ecbcdc52d 100644 --- a/test/harness/replayingCapiProxy.ts +++ b/test/harness/replayingCapiProxy.ts @@ -165,6 +165,7 @@ export class ReplayingCapiProxy extends CapturingHttpProxy { workDir, testInfo, backend: "capi", + autoResponseIndex: 0, toolResultNormalizers: [...this.defaultToolResultNormalizers], }; } @@ -199,6 +200,7 @@ export class ReplayingCapiProxy extends CapturingHttpProxy { workDir: config.workDir, testInfo: config.testInfo, backend: parseReplayBackend(config.backend), + autoResponseIndex: 0, toolResultNormalizers: [...this.defaultToolResultNormalizers], }; @@ -431,6 +433,29 @@ export class ReplayingCapiProxy extends CapturingHttpProxy { return; } + // Deterministic Auto-tier fixture responses for CAPI's `/auto` + // routing endpoint. Consumed one per call in fixture order so + // Auto-tier lifecycle scenarios (cold resume, successful + // activation, failed activation) can control what model `/auto` + // selects and, via `statusCode`, simulate an `/auto` failure. + if ( + options.requestOptions.path === "/auto" && + options.requestOptions.method === "POST" + ) { + const response = + state.storedData?.autoResponses?.[state.autoResponseIndex]; + if (response) { + state.autoResponseIndex++; + options.onResponseStart(response.statusCode ?? 200, { + "content-type": "application/json", + ...commonResponseHeaders, + }); + options.onData(Buffer.from(JSON.stringify(response.body))); + options.onResponseEnd(); + return; + } + } + // Keep GitHub MCP tests hermetic while still capturing the request at // the CAPI proxy. The tests only need a successful transport handshake; // no fake tools are exposed. @@ -698,6 +723,9 @@ async function writeCapturesToDisk( ]), ]; } + if (state.storedData?.autoResponses?.length) { + data.autoResponses = state.storedData.autoResponses; + } if (data.conversations.length > 0) { let yamlText = yaml.stringify(data, { lineWidth: 120 }); @@ -2093,6 +2121,7 @@ type ReplayingCapiProxyState = { testInfo?: { file: string; line?: number }; backend: ReplayBackend; storedData?: NormalizedData | undefined; + autoResponseIndex: number; toolResultNormalizers: ToolResultNormalizer[]; }; @@ -2130,6 +2159,35 @@ export interface NormalizedData { models: string[]; errors?: NormalizedErrorResponse[]; conversations: NormalizedConversation[]; + /** + * Ordered fixture responses for CAPI's `POST /auto` routing endpoint, + * consumed one per call in array order. Used by deterministic Auto-tier + * lifecycle scenarios (cold resume, successful activation, failed + * activation) that need to control what model `/auto` selects without + * depending on live, non-deterministic Auto routing. + */ + autoResponses?: AutoResponseStub[]; +} + +/** + * A single deterministic `/auto` fixture response. `statusCode` defaults to + * 200; a non-2xx value (e.g. 500) simulates an `/auto` request failure so + * tests can exercise failed Auto-tier activation without a real routing + * error. + */ +export interface AutoResponseStub { + statusCode?: number; + body: { + session_token: string; + selected_model: { + id: string; + name?: string; + capabilities?: { + supports: Record; + limits: Record; + }; + }; + }; } function sortJsonKeys(obj: unknown): unknown { diff --git a/test/snapshots/auto_tier/should_commit_fast_auto_tier_after_successful_turn.yaml b/test/snapshots/auto_tier/should_commit_fast_auto_tier_after_successful_turn.yaml new file mode 100644 index 0000000000..c4ea26d401 --- /dev/null +++ b/test/snapshots/auto_tier/should_commit_fast_auto_tier_after_successful_turn.yaml @@ -0,0 +1,21 @@ +models: + - claude-sonnet-5 +conversations: + - messages: + - role: system + content: ${system} + - role: user + content: Reply with exactly AUTO_TIER_FAST_COMMITTED. + - role: assistant + content: AUTO_TIER_FAST_COMMITTED +autoResponses: + - body: + session_token: auto-tier-replay-token-1 + selected_model: + id: claude-sonnet-5 + name: Claude Sonnet 5 + capabilities: + supports: + vision: false + limits: + max_context_window_tokens: 128000 diff --git a/test/snapshots/auto_tier/should_preserve_effective_tier_when_fast_activation_fails.yaml b/test/snapshots/auto_tier/should_preserve_effective_tier_when_fast_activation_fails.yaml new file mode 100644 index 0000000000..44adec410d --- /dev/null +++ b/test/snapshots/auto_tier/should_preserve_effective_tier_when_fast_activation_fails.yaml @@ -0,0 +1,30 @@ +models: + - claude-sonnet-5 +conversations: + - messages: + - role: system + content: ${system} + - role: user + content: Reply with exactly AUTO_TIER_INITIAL_READY. + - role: assistant + content: AUTO_TIER_INITIAL_READY + - role: user + content: Reply with exactly AUTO_TIER_FAILURE_RECOVERED. + - role: assistant + content: AUTO_TIER_FAILURE_RECOVERED +autoResponses: + - body: + session_token: auto-tier-replay-token-1 + selected_model: + id: claude-sonnet-5 + name: Claude Sonnet 5 + capabilities: + supports: + vision: false + limits: + max_context_window_tokens: 128000 + - statusCode: 500 + body: + session_token: unused-auto-tier-token + selected_model: + id: unused-model diff --git a/test/snapshots/auto_tier/should_restore_and_override_fast_auto_tier_on_cold_resume.yaml b/test/snapshots/auto_tier/should_restore_and_override_fast_auto_tier_on_cold_resume.yaml new file mode 100644 index 0000000000..ca831aaa51 --- /dev/null +++ b/test/snapshots/auto_tier/should_restore_and_override_fast_auto_tier_on_cold_resume.yaml @@ -0,0 +1,38 @@ +models: + - claude-sonnet-5 +conversations: + - messages: + - role: system + content: ${system} + - role: user + content: Reply with exactly AUTO_TIER_COLD_RESUME_READY. + - role: assistant + content: AUTO_TIER_COLD_RESUME_READY + - messages: + - role: system + content: ${system} + - role: user + content: Reply with exactly AUTO_TIER_TIERLESS_READY. + - role: assistant + content: AUTO_TIER_TIERLESS_READY +autoResponses: + - body: + session_token: auto-tier-replay-token-1 + selected_model: + id: claude-sonnet-5 + name: Claude Sonnet 5 + capabilities: + supports: + vision: false + limits: + max_context_window_tokens: 128000 + - body: + session_token: auto-tier-replay-token-2 + selected_model: + id: claude-sonnet-5 + name: Claude Sonnet 5 + capabilities: + supports: + vision: false + limits: + max_context_window_tokens: 128000