diff --git a/docs/setup/in-process-runtime.md b/docs/setup/in-process-runtime.md new file mode 100644 index 0000000000..800daef116 --- /dev/null +++ b/docs/setup/in-process-runtime.md @@ -0,0 +1,217 @@ +# Run the Copilot runtime in process + +In-process hosting loads the native Copilot runtime into your application process instead of starting a separate Copilot CLI process. Use it to remove child-process management while keeping the same Copilot SDK sessions, events, tools, hooks, and JSON-RPC behavior. + +> [!WARNING] +> In-process hosting is experimental in every SDK. Test startup, model turns, and shutdown behavior on every operating system and architecture that you deploy. + +## When to use in-process hosting + +In-process hosting is a good fit when: + +* Your application must run without a separate runtime process. +* You want the SDK to own the runtime lifecycle. +* You can ship a native library for each deployment platform. +* Process-wide environment and working-directory settings are acceptable. + +Use the [default bundled CLI setup](./bundled-cli.md) when process isolation and the most established deployment path are more important. Use a [backend service](./backend-services.md) when multiple application instances must connect to a shared runtime over TCP. + +## How it works + +The SDK loads the Copilot runtime native library and binds its fixed C ABI. All SDK methods continue to use the existing `Content-Length`-framed JSON-RPC protocol over an in-memory connection. + +```mermaid +flowchart LR + App["Your application"] + SDK["Copilot SDK client"] + FFI["C ABI connection"] + Runtime["Native Copilot runtime"] + + App --> SDK + SDK -- "JSON-RPC frames" --> FFI + FFI --> Runtime + Runtime -- "Events and callbacks" --> FFI + FFI --> SDK +``` + +The runtime: + +* Runs in the application process without Node.js, a child process, a TCP port, or a connection token. +* Supports the same sessions, streaming events, tools, hooks, permissions, and server-to-client requests as other transports. +* Can invoke SDK callbacks from native worker threads. The SDK handles thread marshalling and callback lifetime. +* Keeps the loaded native library and its worker pool available for the lifetime of the application process. + +## SDK requirements + +All SDKs expose an explicit in-process connection option. Some languages require additional build or package configuration. + +| SDK | Connection option | Additional requirement | +|-----|-------------------|------------------------| +| TypeScript | `RuntimeConnection.forInProcess()` | None when the package includes a compatible runtime bundle | +| Python | `RuntimeConnection.for_inprocess()` | Pre-download with `python -m copilot download-runtime --in-process` when runtime download is unavailable during startup | +| Go | `copilot.InProcessConnection{}` | Build with `-tags copilot_inprocess` | +| .NET | `RuntimeConnection.ForInProcess()` | Allow the `GHCP001` experimental API diagnostic | +| Rust | `Transport::InProcess` | Enable the `bundled-in-process` Cargo feature | +| Java | `RuntimeConnection.forInProcess()` | Add JNA, a platform runtime classifier, and experimental API opt-in | + +The native runtime bundle must match the host operating system, CPU architecture, and, on Linux, C library. Unsupported hosts fail during runtime resolution or startup instead of falling back to a child process. + +## Configure an in-process connection + +Pass the language-specific connection option when you create the client. + +
+TypeScript + + + +```typescript +import { CopilotClient, RuntimeConnection } from "@github/copilot-sdk"; + +const client = new CopilotClient({ + connection: RuntimeConnection.forInProcess(), +}); + +await client.start(); +``` + +
+
+Python + + + +```python +from copilot import CopilotClient, RuntimeConnection + +client = CopilotClient( + connection=RuntimeConnection.for_inprocess(), +) + +await client.start() +``` + +
+
+Go + + + +```go +client := copilot.NewClient(&copilot.ClientOptions{ + Connection: copilot.InProcessConnection{}, +}) + +if err := client.Start(context.Background()); err != nil { + log.Fatal(err) +} +defer client.Stop() +``` + +
+
+.NET + + + +```csharp +#pragma warning disable GHCP001 + +var client = new CopilotClient(new CopilotClientOptions +{ + Connection = RuntimeConnection.ForInProcess(), +}); + +await client.StartAsync(); +``` + +
+
+Rust + + + +```rust +let options = ClientOptions::default() + .with_transport(Transport::InProcess); + +let client = Client::start(options).await?; +``` + +
+
+Java + + + +```java +CopilotClientOptions options = new CopilotClientOptions() + .setConnection(RuntimeConnection.forInProcess()); + +CopilotClient client = new CopilotClient(options); +client.start().join(); +``` + +
+ +You can also set `COPILOT_SDK_DEFAULT_CONNECTION=inprocess` before starting the application. The SDK uses this value only when the client does not specify a connection explicitly. An invalid value causes startup to fail. + +Prefer explicit client configuration in application code. Use the environment variable when deployment configuration must select the transport without changing the application. + +## Configure the runtime + +The SDK converts supported typed client options into native runtime arguments and host-scoped environment values. Depending on the SDK, these options include: + +* Authentication token and logged-in-user fallback. +* Copilot base directory. +* Log level. +* Session idle timeout. +* Remote session mode. + +The in-process runtime receives a snapshot of the host environment plus supported SDK-managed overrides. It does not mutate the host environment. + +Set process-wide values before creating the first in-process client. This includes environment variables that are not represented by typed client options and the application's current working directory. + +## Runtime library resolution + +Each SDK first looks for a compatible bundled or cached runtime library. You can set `COPILOT_CLI_PATH` to point into a compatible Copilot runtime package when you need to provide the runtime separately. + +Only one native runtime library path and version can normally be loaded in a process. Starting another client with the same loaded library is supported, but attempting to load a different runtime library fails. + +For production deployments: + +1. Build and test the application for each target platform. +1. Ensure that the matching native runtime artifact is included in the deployed package or available through the SDK's runtime download mechanism. +1. Start at least one session and complete a model turn in a deployment smoke test. +1. Stop clients gracefully before the application exits. + +## Lifecycle behavior + +Starting an in-process client loads the native library, creates a runtime host, opens an in-memory connection, and performs the normal SDK protocol-version handshake. + +During graceful shutdown, the SDK: + +1. Closes active sessions. +1. Requests normal runtime shutdown over JSON-RPC. +1. Closes the JSON-RPC and native connections. +1. Releases the runtime host. + +The native library can remain loaded until the application process exits. Do not depend on unloading and replacing the runtime library after first use. + +## Limitations + +In-process hosting has these current constraints: + +* **Experimental API**: behavior and packaging requirements can change between releases. +* **Shared process state**: all clients share the host process environment, current working directory, native library, and runtime worker pool. +* **Restricted process options**: SDK options for an arbitrary environment, working directory, telemetry configuration, executable path, or CLI arguments are rejected where applicable. Configure process-global values on the host process and use supported typed options for runtime settings. +* **No per-client working directory**: the runtime uses the hosting process working directory. +* **One runtime version per process**: loading another native library path or version is not supported. +* **Platform maturity varies**: some SDK and platform combinations have reduced model-turn or shutdown coverage. Validate the exact combination that you deploy. + +## Further reading + +* [Choosing a setup path](./choosing-a-setup-path.md): compare in-process hosting with other deployment models +* [Default setup](./bundled-cli.md): run the bundled runtime in a managed child process +* [Backend services](./backend-services.md): connect applications to a shared runtime over TCP +* [Session lifecycle](../hooks/session-lifecycle.md): handle session start and end events