From 30dde8f43b240a626c027aab3b0e84d06713ef7d Mon Sep 17 00:00:00 2001 From: "Darwin D. Wu" Date: Thu, 27 Aug 2026 00:08:53 -0700 Subject: [PATCH 01/29] docs(rfd): define user-owned LLM integrations --- rfd/0005/IMPLEMENTATION.org | 124 +++++++++ rfd/0005/README.adoc | 497 ++++++++++++++++++++++++++++++++++++ rfd/README.md | 1 + 3 files changed, 622 insertions(+) create mode 100644 rfd/0005/IMPLEMENTATION.org create mode 100644 rfd/0005/README.adoc diff --git a/rfd/0005/IMPLEMENTATION.org b/rfd/0005/IMPLEMENTATION.org new file mode 100644 index 0000000..783c1f7 --- /dev/null +++ b/rfd/0005/IMPLEMENTATION.org @@ -0,0 +1,124 @@ +#+TITLE: RFD 0005 implementation checklist + +Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations]]. + +* Phase 0: Verify the OpenAI Codex contract + +- [ ] Trace the current OpenAI Codex device authorization and refresh flow from + OpenAI's published Codex source and record endpoints, polling behavior, + expiry, token rotation, and account identification used by the spike. +- [ ] Verify ReqLLM's ~openai_codex~ provider with an honest Kodo originator for + a text response, Kodo tool calls, structured output, and every reasoning + effort required by current role mappings. +- [ ] Verify refresh-token rotation, control-plane restart recovery, quota + errors, revocation, and required reauthorization without using a shared OAuth + file. +- [ ] Document compatibility criteria, the Beta support boundary, and the + provider response that disables ChatGPT subscription integration rather than + impersonating another client. + +* Phase 1: Build encrypted integration storage and OpenAI API keys + +- [ ] Generate migrations for user-owned provider integrations, unique active + user/provider identity, non-secret status metadata, credential generation, + and encryption-key version. +- [ ] Add authenticated encryption with unique nonces, versioned keys, safe + redaction, and production configuration that distinguishes the Kodo + credential-encryption key from all LLM provider credentials. +- [ ] Implement scope-first integration context operations for listing, + connecting, replacing, validating, and disconnecting credentials without + accepting browser-supplied ownership. +- [ ] Add ~Kodo.LLM~ credential values and translate them into request-local + ReqLLM options without mutating process environment, application config, or + shared credential files. +- [ ] Resolve credentials from the session owner and each role model's provider + before recording an invocation, with actionable missing, invalid, and + disconnected integration errors. +- [ ] Build a shared responsive settings shell with Account and Integrations + navigation, a framed desktop/tablet workspace, compact persistent rail, + deep-linkable detail pane, clear return action, accessible narrow-screen + section navigation, and consistent page headings and action placement that + preserve the RFD interface sketches' hierarchy. +- [ ] Show only implemented user-owned settings; do not add a Personal/Workspace + switch or placeholder categories before Kodo defines workspace-owned + configuration and authorization. +- [ ] Restyle the existing account email and password forms as distinct, + balanced settings surfaces without changing generated authentication, + confirmation, or sudo-mode behavior. +- [ ] Add ~/integrations~ to the existing authenticated LiveView session and + browser pipeline, require sudo mode for credential changes, and implement the + OpenAI Platform API-key provider card, contextual form, connection status, + loading state, and accessible feedback. +- [ ] Add LiveView selector tests for the settings shell, section navigation, + provider cards, contextual forms, and device-flow states, plus representative + desktop, tablet, and narrow-mobile browser checks for the sketched layouts. +- [ ] Preserve the OpenAI environment key only as an explicitly enabled, + deployment-managed migration fallback after user-owned resolution. +- [ ] Test encryption round trips and redaction, ownership isolation, concurrent + users with different OpenAI keys, key replacement during a session, missing + integration behavior, and migration fallback precedence. + +* Phase 2: Add Anthropic and OpenRouter API keys + +- [ ] Add Anthropic API-key connection, fixed endpoint configuration, + provider-specific non-destructive validation, and safe error classification. +- [ ] Add OpenRouter API-key connection, fixed endpoint configuration, + provider-specific non-destructive validation, and explicit OpenRouter billing + identity for routed models. +- [ ] Extend integration settings and model-selection feedback to distinguish + OpenAI API, Anthropic, and OpenRouter providers without displaying saved + credentials. +- [ ] Test a role mapping that uses different connected providers for primary, + search, and review, including absence or revocation of one required + integration during a turn. +- [ ] Add opt-in live-provider checks for OpenAI API, Anthropic, and OpenRouter + while keeping ordinary tests independent of external provider access. + +* Phase 3: Add ChatGPT subscription device authorization beta + +- [ ] Generate persistence for expiring, user-scoped, single-use device + authorization attempts with encrypted sensitive fields, refresh lease state, + and bounded cleanup. +- [ ] Implement device-code creation, interval-aware asynchronous polling, + cancellation, expiry, terminal errors, and authorization-code exchange using + Req rather than embedded browser scripts. +- [ ] Store access tokens, refresh tokens, account identifiers, expiries, and + credential generations in the encrypted integration payload without logging + device or token material. +- [ ] Implement database-backed refresh leases, bounded waiter behavior, + compare-and-update protection, atomic refresh-token rotation, lease takeover, + and terminal ~invalid_grant~ reauthorization. +- [ ] Add the "ChatGPT Subscription (Beta)" connection flow, billing + explanation, progress state, cancellation, quota errors, disconnect, and + reauthorization to the authenticated integrations page. +- [ ] Route ~openai_codex~ models only through the user's ChatGPT integration + and prohibit silent fallback to OpenAI API billing. +- [ ] Test authorization ownership and expiry, polling backoff, two-node refresh + contention, delayed refresh responses, rotated refresh tokens, restart + recovery, revocation, and secret redaction. +- [ ] Run the Phase 0 live compatibility suite against the production path and + keep the feature disabled if any required Codex capability or identity check + fails. + +* Phase 4: Remove boot-time LLM credentials and harden operations + +- [ ] Add onboarding and session/model-settings feedback that links users to + every missing provider integration required by their selected role mapping. +- [ ] Remove provider API keys from required runtime configuration and remove + user request fallback to ~OPENAI_API_KEY~, ~ANTHROPIC_API_KEY~, and + ~OPENROUTER_API_KEY~. +- [ ] Update live-provider CI and development setup to create explicit test-user + integrations without exposing credentials in fixtures, logs, or command + output. +- [ ] Add bounded-cardinality telemetry and credential-free security audit logs + for connection changes, validation, refresh outcomes, and reauthorization. +- [ ] Implement and test bounded-batch encryption-key rotation with a documented + previous-key migration window and rollback procedure. +- [ ] Verify provider credentials never reach runner registration, runner + WebSockets, tool subprocesses, session events, browser responses, exception + inspection, or telemetry metadata. +- [ ] Document encryption-key provisioning, backup handling, credential and + provider revocation, key rotation, device-flow troubleshooting, refresh + failure alerts, and rollback from each rollout phase. +- [ ] Run ~mix precommit~ and the opt-in provider compatibility checks before + declaring the RFD implemented. diff --git a/rfd/0005/README.adoc b/rfd/0005/README.adoc new file mode 100644 index 0000000..ce839d6 --- /dev/null +++ b/rfd/0005/README.adoc @@ -0,0 +1,497 @@ +:authors: Darwin D Wu +:state: discussion +:discussion: https://github.com/chaba-dev/kodo/pull/25 +:labels: software, integrations, llm, oauth, security +:toc: + += RFD 5 Store user-owned LLM provider integrations + +== Goal + +Let each Kodo user connect the LLM providers used by their agent sessions +without requiring an operator to supply an OpenAI API key when the control +plane boots. The first provider set includes OpenAI Platform API keys, +Anthropic API keys, OpenRouter API keys, and OpenAI Codex access through a +linked ChatGPT subscription. + +Provider credentials remain in the control plane. Runners continue to receive +only Kodo credentials and tool requests; they never receive LLM API keys, +access tokens, or refresh tokens. + +== Current state + +Kodo delegates provider authentication to ReqLLM. With no request-local +credential, ReqLLM reads `OPENAI_API_KEY` from the control-plane environment. +This creates one deployment-wide provider identity, makes credential changes an +operational restart concern, and prevents users from choosing separately billed +provider accounts. + +The existing execution boundary is otherwise suitable. Agent inference runs in +the Phoenix control plane, sessions have a durable owning user, and model role +mappings identify the provider used for each request. The Rust runner executes +workspace tools and does not call LLM providers directly. + +== Decision + +Kodo will persist user-owned provider integrations and resolve one for every +model request. The model specification selects the provider; the authenticated +session owner selects the credential. ReqLLM receives that credential through +request-local options so concurrent users never share mutable process or +application configuration. + +The initial provider identities are distinct even when they share a vendor: + +* `openai` uses an OpenAI Platform API key and Platform billing. +* `openai_codex` uses a ChatGPT subscription OAuth credential and the Codex + responses service. +* `anthropic` uses an Anthropic Console API key. +* `openrouter` uses an OpenRouter API key. + +One active integration is allowed per user and provider identity in the first +release. Treating `openai` and `openai_codex` separately lets a user connect +both and makes billing and model routing explicit. Supporting several accounts +for one provider may be added later with an explicit integration selection in +model settings; the first release does not add that complexity. + +Model role mappings continue to own model selection. They do not contain +credentials. A mapping may use different providers for primary, search, and +review roles, in which case the session owner must have every required +integration. Kodo checks this when starting a turn and resolves the current +credential again immediately before each provider request. Key replacement, +token refresh, disconnection, or revocation therefore takes effect without +rewriting session history. + +== Provider support + +=== API-key providers + +OpenAI, Anthropic, and OpenRouter use the same integration lifecycle: connect, +validate, replace, test, and disconnect. Provider endpoints are fixed by Kodo; +the initial UI does not accept arbitrary base URLs or HTTP headers. This avoids +turning provider configuration into a server-side request forgery or +credential-exfiltration path. + +Validation is non-destructive and provider-specific. A transient validation +failure does not expose the credential or silently fall back to another user's +or deployment's key. Kodo records a bounded status and safe error code, while +the detailed provider error is redacted before logging or display. A saved +credential is never rendered back to the browser; the UI shows only provider, +status, validation time, and a non-secret fingerprint where useful. + +OpenRouter is a provider in its own right rather than a transparent fallback +for direct provider integrations. Models routed through OpenRouter use the +OpenRouter integration and billing even when the underlying model was produced +by OpenAI or Anthropic. + +=== ChatGPT subscription + +ChatGPT subscription support uses OpenAI Codex device authorization. The user +starts the flow in Kodo, receives an OpenAI verification URL and one-time code, +and approves the request in a browser. Kodo polls at OpenAI's instructed +interval, exchanges the resulting authorization code and PKCE verifier, and +stores the returned access token, refresh token, account identifier, and expiry +as encrypted credentials. + +Authorization attempts are user-scoped, single-use, and expire after the +provider deadline. Polling continues only while an attempt is active, respects +provider backoff, and stops on completion, cancellation, expiry, or a terminal +provider error. Attempt identifiers and one-time codes are treated as secrets +and are not written to logs or telemetry. + +The OpenAI Codex source publishes the device flow, and Amp and other hosted +agents demonstrate subscription-backed use. The downstream +`chatgpt.com/backend-api/codex` inference contract is nevertheless less formal +than the OpenAI Platform API. Kodo labels this integration "ChatGPT +Subscription (Beta)", identifies itself honestly in supported client headers, +and does not silently replace it with an OpenAI API key if authorization or +subscription quota fails. + +Before general availability, a live compatibility check must confirm device +authorization, a tool-calling turn, structured output used by review and search +roles, refresh-token rotation, restart recovery, and Kodo's client identity. +If OpenAI rejects Kodo's identity or restricts this use, the integration remains +disabled rather than impersonating Codex, Amp, Pi, or another client. + +Anthropic subscription OAuth is outside this RFD. Anthropic permits API keys +and supported cloud-provider credentials for third-party applications but +currently prohibits third parties from collecting or intermediating Claude +subscription credentials. Kodo will not offer Claude subscription login unless +Anthropic publishes a contract that permits this use. + +== Persistence and encryption + +Provider integrations store non-secret indexing and operational metadata +separately from one versioned encrypted credential payload. The durable record +contains at least: + +* a generated identifier, owning user, provider identity, and authentication + type; +* encrypted credentials and the encryption-key version; +* connection status, provider account identifier when available, and token + expiry; +* validation and refresh timestamps plus a bounded safe error code; and +* normal insertion and update timestamps. + +The database enforces ownership references and uniqueness of the active user +and provider pair. Context functions take the authenticated scope as their +first argument and apply the owning user in the query. Changesets never cast a +`user_id` supplied by a browser. + +Credentials use authenticated encryption with a unique nonce per write. Kodo +requires a deployment-level credential-encryption key, supplied independently +of all LLM providers, and records a key version so ciphertext can be rotated. +Production refuses to persist or decrypt integrations when this key is absent +or malformed. Database encryption does not remove the need to protect the +control-plane process; it protects database copies, backups, and read-only +database access from directly revealing provider credentials. + +The credential payload is opaque to Ecto query code and never appears in +session events, model mappings, changeset errors, exception inspection, +telemetry metadata, or client responses. Deleting an integration removes its +credential payload. Backup retention and provider-side revocation remain +operational concerns and are documented with the deployment procedure. + +== Request-time resolution + +The provider-independent LLM boundary accepts a Kodo credential value in its +generation options. The ReqLLM adapter translates an API-key credential to +`:api_key` and a Codex credential to request-local OAuth options. No request +uses `System.put_env/2`, `Application.put_env/3`, ReqLLM's global key store, or a +shared OAuth file. + +Credential resolution occurs before Kodo persists a model invocation as +started. A missing, disconnected, or reauthorization-required integration +returns a domain error naming the provider and required user action without +including credential material. The session can accept a later turn after the +user reconnects the provider. Provider errors must not trigger fallback to a +differently billed integration unless model settings explicitly define such a +fallback in a future proposal. + +Model validation remains credential-independent. Provider capability checks +may resolve model metadata without making a billed request, while connection +validation and inference use the session owner's integration. + +== Refresh and multi-node coordination + +API keys do not refresh. Codex access tokens refresh shortly before expiry, +using the stored refresh token and OpenAI's token endpoint. A successful +refresh atomically replaces the access token, rotated refresh token, account +identifier, and expiry in one encrypted payload. + +Only one control-plane replica may refresh an integration at a time. Kodo uses +a short database-backed refresh lease so it does not hold a database +transaction or row lock across an external HTTP request. Other callers wait for +a bounded interval and reload the integration; an expired lease can be taken +over. The lease owner updates the integration only if its lease and credential +generation still match, preventing a delayed response from overwriting a newer +authorization or refresh. + +An `invalid_grant` or equivalent terminal response marks the integration as +requiring authorization and discards unusable access for request purposes. +Transient provider or network failures preserve the credential, release or +expire the lease, and return a retryable provider-unavailable error. Refresh +tokens are never retried concurrently because providers may rotate and +invalidate them on first use. + +== User interface and authorization + +Integration management lives at `/integrations` inside the existing +`:require_authenticated_user` LiveView session and browser pipeline. The route +requires login because every integration is user-owned and its status controls +that user's model requests. Connecting, replacing, and disconnecting a +credential require sudo mode because they change access to a separately billed +external account. + +The page provides distinct actions for OpenAI API, ChatGPT Subscription, +Anthropic, and OpenRouter. It explains the billing identity before connection, +shows progress and expiry for device authorization, and provides actionable +states for missing credentials, invalid keys, quota exhaustion, and required +reauthorization. It never embeds raw scripts; device-flow progress uses normal +LiveView events and asynchronous server work. + +Session creation and model settings identify integrations by provider and +billing path. If a selected role lacks a connection, the UI links to the +integration page rather than instructing the operator to restart Kodo with an +environment variable. + +== Settings information architecture and visual direction + +The integration work establishes a shared settings experience rather than +adding another isolated generated form. Account and integration pages keep +their existing `/users/settings` and `/integrations` paths, but render inside +one settings shell with consistent navigation, page headings, content width, +and action placement. + +The visual direction takes cues from Amp's tablet interface: a quiet neutral +canvas, clear sectional navigation, subtle separators, rounded surfaces, +generous spacing, restrained accent color, and controls sized for touch. Kodo +uses those principles for settings rather than copying Amp's conversation, +project, or composer chrome. + +At desktop and tablet widths, settings appear as one framed workspace within +the authenticated application: a compact persistent rail lists Account and +Integrations, a subtle divider separates navigation from content, and the +selected page occupies a balanced detail pane. The frame may use a rounded +outer surface at larger widths but remains a normal deep-linkable page, not a +transient modal whose state disappears on refresh. A clear back or close action +returns to the user's prior application context. + +Kodo does not add Personal/Workspace switching, Billing, Usage, Secrets, or +other placeholder destinations merely because they appear in the reference. +Only implemented settings are visible. If Kodo later gains workspace-owned +configuration, its ownership, authorization, and navigation require a separate +decision rather than overloading user-owned integrations. + +On narrow phones the frame becomes edge-to-edge and the rail becomes horizontal +section navigation above the page; it must not create a second off-canvas menu +inside the existing application navigation. The content pane uses one column +without shrinking desktop cards into unreadable two-column forms. Focus order, +selected-state contrast, keyboard navigation, scroll containment, and touch +target size remain equivalent across layouts. + +The Account page groups email and password changes into separately titled +surfaces with concise security context. Its generated authentication behavior, +sudo-mode checks, form semantics, and confirmation flow do not change. The +Integrations page uses one provider card per billing identity. Each card gives +provider purpose, authentication method, billing path, connection status, and +one clear primary action before exposing secondary actions such as test, +replace, reconnect, or disconnect. + +Connection forms open within the provider's page context rather than placing +four permanent secret inputs on the screen. Device authorization emphasizes +the OpenAI-owned verification destination, the one-time code, remaining time, +and a cancellable progress state. Success and failure feedback stays attached +to the affected provider card and is also announced accessibly; transient +loading must not shift unrelated cards or discard entered credentials. + +== Interface sketches + +These sketches communicate hierarchy and responsive behavior, not exact copy, +dimensions, colors, or component styling. + +=== Desktop and tablet + +[source,text] +---- ++--------------------------------------------------------------------------+ +| <- Sessions Settings darwin67 | ++------------------+-------------------------------------------------------+ +| | | +| SETTINGS | Provider integrations | +| | Connect each billing identity used by your agents. | +| Account | | +| > Integrations | +-------------------------------------------------+ | +| | | ChatGPT Subscription (Beta) CONNECTED | | +| | | OpenAI Codex - ChatGPT plan billing | | +| | | Last checked 2 min ago [Test] [Reconnect] | | +| | +-------------------------------------------------+ | +| | | +| | +-------------------------------------------------+ | +| | | OpenAI API NOT CONNECTED | | +| | | Platform API usage and billing | | +| | | [Connect] | | +| | +-------------------------------------------------+ | +| | | +| | +----------------------+ +----------------------+ | +| | | Anthropic API | | OpenRouter | | +| | | NOT CONNECTED | | CONNECTED | | +| | | [Connect] | | [Test] [Replace] | | +| | +----------------------+ +----------------------+ | +| | | ++------------------+-------------------------------------------------------+ +---- + +The detail pane may use a two-column card row only when both cards remain +readable and actions do not wrap ambiguously. ChatGPT and OpenAI API remain +separate cards because their authentication, endpoint, model namespace, and +billing identity differ. + +=== Narrow mobile + +[source,text] +---- ++--------------------------------------+ +| <- Settings | ++--------------------------------------+ +| [ Account ] [ Integrations ] | ++--------------------------------------+ +| | +| Provider integrations | +| Connect providers used by agents. | +| | +| +--------------------------------+ | +| | ChatGPT Subscription (Beta) | | +| | CONNECTED | | +| | ChatGPT plan billing | | +| | | | +| | [Test] [Reconnect] | | +| +--------------------------------+ | +| | +| +--------------------------------+ | +| | OpenAI API - NOT CONNECTED | | +| | Platform API billing | | +| | [Connect] | | +| +--------------------------------+ | +| | +| +--------------------------------+ | +| | Anthropic API | | +| | NOT CONNECTED [Connect] | | +| +--------------------------------+ | +| | +| +--------------------------------+ | +| | OpenRouter | | +| | CONNECTED [Manage] | | +| +--------------------------------+ | +| | ++--------------------------------------+ +---- + +Cards become a single sequence on mobile. Section navigation remains visible +without horizontally scrolling provider content, and secondary actions may +collapse behind one labelled Manage action when width is constrained. + +=== ChatGPT device authorization + +[source,text] +---- ++-----------------------------------------------------------+ +| ChatGPT Subscription (Beta) | +| | +| 1. Open the OpenAI verification page: | +| | +| https://auth.openai.com/codex/device | +| | +| 2. Enter this one-time code: | +| | +| ABCD-EFGH [Copy] | +| | +| Waiting for OpenAI authorization... 12:41 left | +| [==============================>................] | +| | +| [Cancel] | ++-----------------------------------------------------------+ +---- + +The code is rendered as selectable text with a copy action in the real +interface. Status updates use a live region, the countdown is not the only +expiry cue, and cancelling returns the provider card to a stable disconnected +state. + +== Migration and compatibility + +The rollout temporarily preserves existing environment-key behavior as a +deployment fallback so current installations and live-provider CI do not stop +working when the schema first ships. Resolution during this interval is: + +. the session owner's active provider integration; then +. the matching legacy deployment environment key, if fallback is enabled. + +Kodo never copies a deployment key into a user's integration because there is +no safe implicit owner. The fallback is visibly deployment-managed and cannot +be viewed or edited in user settings. It is removed after integration +onboarding and provider-specific live tests are established. + +At the end of this RFD, no LLM API key or OAuth token is required at boot. A +credential-encryption key remains required because encrypted database secrets +need a root of trust outside the database. Installations with no connected +provider can boot and serve account and integration pages, but cannot start a +model turn until the user connects every provider required by the selected +role mapping. + +== Security and operations + +Kodo applies the following boundaries: + +* Provider secrets remain in the Phoenix control plane and are never sent to + runners, browsers after submission, or Amp orbs used as Kodo runners. +* Provider endpoints and authorization endpoints are allowlisted in code. +* Logs and exceptions redact API keys, bearer tokens, refresh tokens, device + codes, authorization codes, and encrypted payloads. +* Metrics use bounded provider, authentication-type, operation, and outcome + labels; user, account, integration, and token identifiers stay out of metric + labels. +* Connection changes and terminal refresh failures produce security-relevant + audit logs without credential material. +* Encryption-key rotation re-encrypts credentials in bounded batches and keeps + the previous decryption key only for the documented migration window. +* Disconnecting instructs the user how to revoke the credential at the + provider when Kodo cannot revoke it directly. + +Operational documentation covers initial encryption-key provisioning, backup +handling, rotation, provider revocation, device-flow failures, refresh failure +rates, and safe rollback while legacy environment fallback exists. + +== Alternatives considered + +=== Continue deployment environment keys + +Environment keys are simple but preserve one billing identity for every user, +require operator involvement, and cannot support user-scoped OAuth refresh. +They remain only as a migration bridge. + +=== Write credentials to runner environments + +This would let provider clients run near workspaces but expands the secret +boundary to every runner and subprocess. Kodo already performs inference in the +control plane, so this adds risk without a corresponding capability. + +=== Use ReqLLM global keys or OAuth files + +ReqLLM supports process-wide keys and file-backed OAuth refresh for local +applications. Those stores cannot safely distinguish concurrent Kodo users or +coordinate refresh across control-plane replicas. Kodo therefore owns the +database lifecycle and passes only request-local credentials to ReqLLM. + +=== Store provider-specific secret columns + +Separate API-key, access-token, and refresh-token columns make accidental +selection and logging easier and require schema changes for every credential +shape. A versioned encrypted payload with queryable non-secret metadata keeps +provider details inside the integration boundary. + +== Phased delivery + +Each phase is independently reviewable and deployable. Detailed progress is +tracked in link:IMPLEMENTATION.org[RFD 5 implementation checklist]. + +=== Phase 0: Verify the Codex contract + +Run a bounded live spike against the current ReqLLM `openai_codex` provider. +Confirm device authorization mechanics from OpenAI's Codex source, Kodo's +honest client identity, required model capabilities, token refresh, and restart +recovery. Record the supported request shape and a disable criterion before +building the production OAuth UI. + +=== Phase 1: Build encrypted integration storage + +Add the user-owned integration schema, authenticated encryption and key +versioning, scoped context operations, safe inspection, and migration-time +environment fallback. Add the shared responsive settings shell, improve the +existing Account presentation without changing its behavior, and ship the +authenticated integration page with OpenAI Platform API-key connection first. +This establishes the full path from UI through request-local ReqLLM options. + +=== Phase 2: Add Anthropic and OpenRouter + +Reuse the API-key lifecycle for Anthropic and OpenRouter, add provider-specific +validation and errors, and verify model-role mappings can use several providers +in one session without sharing credentials or billing paths. + +=== Phase 3: Add ChatGPT device authorization beta + +Add durable user-scoped authorization attempts, asynchronous polling, encrypted +OAuth persistence, refresh leases, rotated-token updates, reauthorization, and +the ChatGPT Subscription billing identity. Gate availability on Phase 0's +compatibility criteria and retain the Beta label. + +=== Phase 4: Remove boot-time LLM credentials and harden operations + +Complete onboarding and missing-integration UX, remove runtime dependence on +provider environment keys, update CI to create explicit test integrations, +exercise multi-node refresh and key rotation, and publish operator guidance. +The encryption key remains the only integration-related boot secret. + +== Implementation + +Progress against this proposal is tracked in the +link:IMPLEMENTATION.org[RFD 5 implementation checklist]. diff --git a/rfd/README.md b/rfd/README.md index 151a759..a2bd35f 100644 --- a/rfd/README.md +++ b/rfd/README.md @@ -11,6 +11,7 @@ should be read. | [2: Extend Kodo to mobile and tablet clients](0002/README.adoc) | Start with a responsive PWA, handle disconnection gracefully, and preserve a path to native clients | | [3: Define behavioral evaluation and release benchmarking](0003/README.adoc) | Separate role quality from multi-turn behavior, calibrate observable scoring, and benchmark frozen release candidates | | [4: Introduce a full-screen terminal interface](0004/README.adoc) | Make Ratatui the default client, retain non-interactive runner hosting, and preserve reusable application semantics | +| [5: Store user-owned LLM provider integrations](0005/README.adoc) | Encrypt provider credentials, resolve them per model request, and support OpenAI, Anthropic, OpenRouter, and ChatGPT subscriptions without boot-time LLM credentials | ## Source format From 28f9aba1d051e44260311dfe6fee9f59270f7de2 Mon Sep 17 00:00:00 2001 From: "Darwin D. Wu" Date: Thu, 27 Aug 2026 00:18:48 -0700 Subject: [PATCH 02/29] docs(rfd): clarify provider integration questions --- rfd/0005/IMPLEMENTATION.org | 9 ++++++ rfd/0005/README.adoc | 60 +++++++++++++++++++++++++++++-------- 2 files changed, 56 insertions(+), 13 deletions(-) diff --git a/rfd/0005/IMPLEMENTATION.org b/rfd/0005/IMPLEMENTATION.org index 783c1f7..8893510 100644 --- a/rfd/0005/IMPLEMENTATION.org +++ b/rfd/0005/IMPLEMENTATION.org @@ -16,6 +16,9 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations - [ ] Document compatibility criteria, the Beta support boundary, and the provider response that disables ChatGPT subscription integration rather than impersonating another client. +- [ ] Record the support basis for third-party Codex use and the project + decision that either enables the private-API Beta or keeps it disabled by + default; technical compatibility alone is insufficient. * Phase 1: Build encrypted integration storage and OpenAI API keys @@ -54,6 +57,9 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations desktop, tablet, and narrow-mobile browser checks for the sketched layouts. - [ ] Preserve the OpenAI environment key only as an explicitly enabled, deployment-managed migration fallback after user-owned resolution. +- [ ] Decide and test first-run model routing after a provider is connected, + including confirmation of provider-compatible role presets, distinct + Platform and subscription billing, and preservation of existing sessions. - [ ] Test encryption round trips and redaction, ownership isolation, concurrent users with different OpenAI keys, key replacement during a session, missing integration behavior, and migration fallback precedence. @@ -65,6 +71,9 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations - [ ] Add OpenRouter API-key connection, fixed endpoint configuration, provider-specific non-destructive validation, and explicit OpenRouter billing identity for routed models. +- [ ] Document whether validation for each API-key provider is unbilled or an + explicit potentially billed Test action, and represent saved-but-unverified + credentials separately from valid and invalid credentials. - [ ] Extend integration settings and model-selection feedback to distinguish OpenAI API, Anthropic, and OpenRouter providers without displaying saved credentials. diff --git a/rfd/0005/README.adoc b/rfd/0005/README.adoc index ce839d6..c195cf1 100644 --- a/rfd/0005/README.adoc +++ b/rfd/0005/README.adoc @@ -98,8 +98,7 @@ provider backoff, and stops on completion, cancellation, expiry, or a terminal provider error. Attempt identifiers and one-time codes are treated as secrets and are not written to logs or telemetry. -The OpenAI Codex source publishes the device flow, and Amp and other hosted -agents demonstrate subscription-backed use. The downstream +The OpenAI Codex source publishes the device flow. The downstream `chatgpt.com/backend-api/codex` inference contract is nevertheless less formal than the OpenAI Platform API. Kodo labels this integration "ChatGPT Subscription (Beta)", identifies itself honestly in supported client headers, @@ -110,7 +109,7 @@ Before general availability, a live compatibility check must confirm device authorization, a tool-calling turn, structured output used by review and search roles, refresh-token rotation, restart recovery, and Kodo's client identity. If OpenAI rejects Kodo's identity or restricts this use, the integration remains -disabled rather than impersonating Codex, Amp, Pi, or another client. +disabled rather than impersonating another client. Anthropic subscription OAuth is outside this RFD. Anthropic permits API keys and supported cloud-provider credentials for third-party applications but @@ -222,11 +221,10 @@ their existing `/users/settings` and `/integrations` paths, but render inside one settings shell with consistent navigation, page headings, content width, and action placement. -The visual direction takes cues from Amp's tablet interface: a quiet neutral -canvas, clear sectional navigation, subtle separators, rounded surfaces, -generous spacing, restrained accent color, and controls sized for touch. Kodo -uses those principles for settings rather than copying Amp's conversation, -project, or composer chrome. +The visual direction uses a quiet neutral canvas, clear sectional navigation, +subtle separators, rounded surfaces, generous spacing, restrained accent +color, and controls sized for touch. These principles support settings without +introducing conversation, project, or composer chrome into the page. At desktop and tablet widths, settings appear as one framed workspace within the authenticated application: a compact persistent rail lists Account and @@ -237,10 +235,10 @@ transient modal whose state disappears on refresh. A clear back or close action returns to the user's prior application context. Kodo does not add Personal/Workspace switching, Billing, Usage, Secrets, or -other placeholder destinations merely because they appear in the reference. -Only implemented settings are visible. If Kodo later gains workspace-owned -configuration, its ownership, authorization, and navigation require a separate -decision rather than overloading user-owned integrations. +other placeholder destinations. Only implemented settings are visible. If Kodo +later gains workspace-owned configuration, its ownership, authorization, and +navigation require a separate decision rather than overloading user-owned +integrations. On narrow phones the frame becomes edge-to-edge and the rail becomes horizontal section navigation above the page; it must not create a second off-canvas menu @@ -403,7 +401,7 @@ role mapping. Kodo applies the following boundaries: * Provider secrets remain in the Phoenix control plane and are never sent to - runners, browsers after submission, or Amp orbs used as Kodo runners. + runners, browsers after submission, or hosted runner environments. * Provider endpoints and authorization endpoints are allowlisted in code. * Logs and exceptions redact API keys, bearer tokens, refresh tokens, device codes, authorization codes, and encrypted payloads. @@ -421,6 +419,42 @@ Operational documentation covers initial encryption-key provisioning, backup handling, rotation, provider revocation, device-flow failures, refresh failure rates, and safe rollback while legacy environment fallback exists. +== Questions to resolve + +The following questions do not reopen the settled ownership, encryption, +request-local credential, or no-runner-secret boundaries. Each has a named +decision point before the affected phase can ship. + +=== Codex support basis and client identity + +A successful device flow and inference request prove technical compatibility, +not that OpenAI offers a supported third-party contract for Kodo's use. Phase 0 +must establish the basis on which Kodo is willing to expose the integration: +published provider documentation or terms, direct provider confirmation, or an +explicit project decision to accept a private-API Beta with a documented +disable policy. The decision must also confirm that Kodo's own originator and +client identity are accepted. Without that decision, Phase 3 remains disabled +by default even if the live compatibility checks pass. + +=== First-run model routing + +Connecting a provider does not by itself decide which models should serve the +primary, search, and review roles. Before Phase 1 completes, Kodo must decide +whether onboarding asks the user to confirm a provider-compatible preset or +automatically selects one for new sessions. Existing session mappings must +never be rewritten. The chosen flow must distinguish `openai` Platform models +from `openai_codex` subscription models and explain their different billing +paths. + +=== Credential validation and possible cost + +Providers do not all expose an equivalent unbilled credential-introspection +endpoint. Before each API-key provider ships, Kodo must identify a reliable +non-billed validation request or decide that validation is an explicit model +request that may incur a small charge. Saving a credential must remain possible +during a provider outage; the UI must distinguish saved, unverified, valid, and +invalid states and disclose any billed Test action before it runs. + == Alternatives considered === Continue deployment environment keys From 70a79760cfb73b431d4ca0f6639d1a3644d9ce5c Mon Sep 17 00:00:00 2001 From: "Darwin D. Wu" Date: Thu, 27 Aug 2026 00:35:15 -0700 Subject: [PATCH 03/29] docs(rfd): decide Codex beta support boundary --- rfd/0005/IMPLEMENTATION.org | 7 ++++--- rfd/0005/README.adoc | 29 +++++++++++++---------------- 2 files changed, 17 insertions(+), 19 deletions(-) diff --git a/rfd/0005/IMPLEMENTATION.org b/rfd/0005/IMPLEMENTATION.org index 8893510..03103e4 100644 --- a/rfd/0005/IMPLEMENTATION.org +++ b/rfd/0005/IMPLEMENTATION.org @@ -16,9 +16,10 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations - [ ] Document compatibility criteria, the Beta support boundary, and the provider response that disables ChatGPT subscription integration rather than impersonating another client. -- [ ] Record the support basis for third-party Codex use and the project - decision that either enables the private-API Beta or keeps it disabled by - default; technical compatibility alone is insufficient. +- [ ] Gate ChatGPT Subscription UI and requests behind a deployment-level + feature flag that defaults to disabled, document operator opt-in and the + private-interface support boundary, and require a later decision before + changing the default. * Phase 1: Build encrypted integration storage and OpenAI API keys diff --git a/rfd/0005/README.adoc b/rfd/0005/README.adoc index c195cf1..90a6f11 100644 --- a/rfd/0005/README.adoc +++ b/rfd/0005/README.adoc @@ -105,11 +105,19 @@ Subscription (Beta)", identifies itself honestly in supported client headers, and does not silently replace it with an OpenAI API key if authorization or subscription quota fails. -Before general availability, a live compatibility check must confirm device -authorization, a tool-calling turn, structured output used by review and search -roles, refresh-token rotation, restart recovery, and Kodo's client identity. -If OpenAI rejects Kodo's identity or restricts this use, the integration remains -disabled rather than impersonating another client. +ChatGPT Subscription is disabled by default behind a deployment-level feature +flag. An operator may explicitly enable the Beta only after accepting its +private-interface support boundary and running the compatibility check for +device authorization, a tool-calling turn, structured output used by review and +search roles, refresh-token rotation, restart recovery, and Kodo's client +identity. Kodo never impersonates another client. An explicit provider response +that rejects Kodo's identity or third-party use marks the integration +unsupported rather than retrying or falling back to Platform billing. + +Enabling ChatGPT Subscription by default requires a later decision backed by +published provider support or direct provider confirmation. Ordinary transient, +quota, and authorization errors do not change the deployment feature flag and +remain scoped to the affected integration. Anthropic subscription OAuth is outside this RFD. Anthropic permits API keys and supported cloud-provider credentials for third-party applications but @@ -425,17 +433,6 @@ The following questions do not reopen the settled ownership, encryption, request-local credential, or no-runner-secret boundaries. Each has a named decision point before the affected phase can ship. -=== Codex support basis and client identity - -A successful device flow and inference request prove technical compatibility, -not that OpenAI offers a supported third-party contract for Kodo's use. Phase 0 -must establish the basis on which Kodo is willing to expose the integration: -published provider documentation or terms, direct provider confirmation, or an -explicit project decision to accept a private-API Beta with a documented -disable policy. The decision must also confirm that Kodo's own originator and -client identity are accepted. Without that decision, Phase 3 remains disabled -by default even if the live compatibility checks pass. - === First-run model routing Connecting a provider does not by itself decide which models should serve the From bb8571c9522e1661fdfb084ea5632dd24cd6052e Mon Sep 17 00:00:00 2001 From: "Darwin D. Wu" Date: Thu, 27 Aug 2026 00:37:58 -0700 Subject: [PATCH 04/29] docs(rfd): decide integration model routing --- rfd/0005/IMPLEMENTATION.org | 9 ++++++--- rfd/0005/README.adoc | 31 +++++++++++++++++-------------- 2 files changed, 23 insertions(+), 17 deletions(-) diff --git a/rfd/0005/IMPLEMENTATION.org b/rfd/0005/IMPLEMENTATION.org index 03103e4..7d29ab2 100644 --- a/rfd/0005/IMPLEMENTATION.org +++ b/rfd/0005/IMPLEMENTATION.org @@ -58,9 +58,12 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations desktop, tablet, and narrow-mobile browser checks for the sketched layouts. - [ ] Preserve the OpenAI environment key only as an explicitly enabled, deployment-managed migration fallback after user-owned resolution. -- [ ] Decide and test first-run model routing after a provider is connected, - including confirmation of provider-compatible role presets, distinct - Platform and subscription billing, and preservation of existing sessions. +- [ ] Keep provider connection separate from routing changes; when no usable + mapping exists, present one provider-compatible role preset with explicit + billing paths and apply it only after "Use recommended models" confirmation. +- [ ] Test that connecting another provider leaves a usable mapping unchanged, + confirmed presets affect only defaults and new sessions, and disconnecting a + required integration never silently reroutes existing sessions. - [ ] Test encryption round trips and redaction, ownership isolation, concurrent users with different OpenAI keys, key replacement during a session, missing integration behavior, and migration fallback precedence. diff --git a/rfd/0005/README.adoc b/rfd/0005/README.adoc index 90a6f11..513a5f3 100644 --- a/rfd/0005/README.adoc +++ b/rfd/0005/README.adoc @@ -61,6 +61,19 @@ credential again immediately before each provider request. Key replacement, token refresh, disconnection, or revocation therefore takes effect without rewriting session history. +Connecting a provider and changing model routing are separate actions. If the +current role mapping remains usable, connecting another provider does not +prompt for or apply a routing change. If a user has no usable mapping, Kodo +presents one recommended provider-compatible preset and shows its primary, +search, and review models with their Platform, subscription, or aggregator +billing paths. The preset is applied only after the user chooses "Use +recommended models". + +Confirmed presets affect new sessions and user defaults; existing session +mappings are never rewritten. Disconnecting an integration marks dependent +mappings unavailable and links to model settings rather than silently routing +requests through a differently billed provider. + == Provider support === API-key providers @@ -427,21 +440,11 @@ Operational documentation covers initial encryption-key provisioning, backup handling, rotation, provider revocation, device-flow failures, refresh failure rates, and safe rollback while legacy environment fallback exists. -== Questions to resolve - -The following questions do not reopen the settled ownership, encryption, -request-local credential, or no-runner-secret boundaries. Each has a named -decision point before the affected phase can ship. - -=== First-run model routing +== Question to resolve -Connecting a provider does not by itself decide which models should serve the -primary, search, and review roles. Before Phase 1 completes, Kodo must decide -whether onboarding asks the user to confirm a provider-compatible preset or -automatically selects one for new sessions. Existing session mappings must -never be rewritten. The chosen flow must distinguish `openai` Platform models -from `openai_codex` subscription models and explain their different billing -paths. +The remaining question does not reopen the settled ownership, encryption, +request-local credential, no-runner-secret, or model-routing boundaries. It +must be answered before the affected providers ship. === Credential validation and possible cost From f291ddb28a28682617c997a27c881a44c60694c6 Mon Sep 17 00:00:00 2001 From: "Darwin D. Wu" Date: Thu, 27 Aug 2026 01:10:16 -0700 Subject: [PATCH 05/29] docs(rfd): define token refresh process lifecycle --- rfd/0005/IMPLEMENTATION.org | 3 +++ rfd/0005/README.adoc | 14 ++++++++++++++ 2 files changed, 17 insertions(+) diff --git a/rfd/0005/IMPLEMENTATION.org b/rfd/0005/IMPLEMENTATION.org index 7d29ab2..e22f00f 100644 --- a/rfd/0005/IMPLEMENTATION.org +++ b/rfd/0005/IMPLEMENTATION.org @@ -98,6 +98,9 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations - [ ] Store access tokens, refresh tokens, account identifiers, expiries, and credential generations in the encrypted integration payload without logging device or token material. +- [ ] Run just-in-time refreshes as bounded tasks under + ~Kodo.ControlPlaneTaskSupervisor~; do not create a long-lived supervised + process or in-memory refresh timer per integration. - [ ] Implement database-backed refresh leases, bounded waiter behavior, compare-and-update protection, atomic refresh-token rotation, lease takeover, and terminal ~invalid_grant~ reauthorization. diff --git a/rfd/0005/README.adoc b/rfd/0005/README.adoc index 513a5f3..b675eae 100644 --- a/rfd/0005/README.adoc +++ b/rfd/0005/README.adoc @@ -198,6 +198,20 @@ using the stored refresh token and OpenAI's token endpoint. A successful refresh atomically replaces the access token, rotated refresh token, account identifier, and expiry in one encrypted payload. +Refresh is just-in-time, bounded work rather than a long-lived process per +integration. A request that encounters a near-expiry token starts the external +HTTP operation under Kodo's existing task supervisor after acquiring the +database lease. The task holds decrypted credentials only for the operation's +duration. A task or node failure leaves durable credential state unchanged and +the expired lease available for takeover. + +Kodo does not start a `DynamicSupervisor` child or keep an in-memory timer for +each integration. Such processes would duplicate across replicas, lose their +schedule during restarts, and still require database coordination. If proactive +refresh becomes necessary, a single supervised sweeper or durable job system +may discover due integrations, but it must invoke the same lease-protected +refresh operation and keep the database as the source of truth. + Only one control-plane replica may refresh an integration at a time. Kodo uses a short database-backed refresh lease so it does not hold a database transaction or row lock across an external HTTP request. Other callers wait for From ebade385827c3b5bd6dfaca7760cd317d1ebcd85 Mon Sep 17 00:00:00 2001 From: "Darwin D. Wu" Date: Thu, 27 Aug 2026 10:29:11 -0700 Subject: [PATCH 06/29] docs(rfd): remove legacy provider fallback --- rfd/0005/IMPLEMENTATION.org | 16 +++++------- rfd/0005/README.adoc | 51 ++++++++++++++++++++----------------- 2 files changed, 34 insertions(+), 33 deletions(-) diff --git a/rfd/0005/IMPLEMENTATION.org b/rfd/0005/IMPLEMENTATION.org index e22f00f..a5848a4 100644 --- a/rfd/0005/IMPLEMENTATION.org +++ b/rfd/0005/IMPLEMENTATION.org @@ -38,6 +38,10 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations - [ ] Resolve credentials from the session owner and each role model's provider before recording an invocation, with actionable missing, invalid, and disconnected integration errors. +- [ ] Remove provider-key environment lookup from the production adapter and + make missing user integrations fail closed; give development and opt-in + live-provider checks explicit test-user integrations or typed test + credentials through the same Kodo-owned boundary. - [ ] Build a shared responsive settings shell with Account and Integrations navigation, a framed desktop/tablet workspace, compact persistent rail, deep-linkable detail pane, clear return action, accessible narrow-screen @@ -56,8 +60,6 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations - [ ] Add LiveView selector tests for the settings shell, section navigation, provider cards, contextual forms, and device-flow states, plus representative desktop, tablet, and narrow-mobile browser checks for the sketched layouts. -- [ ] Preserve the OpenAI environment key only as an explicitly enabled, - deployment-managed migration fallback after user-owned resolution. - [ ] Keep provider connection separate from routing changes; when no usable mapping exists, present one provider-compatible role preset with explicit billing paths and apply it only after "Use recommended models" confirmation. @@ -66,7 +68,7 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations required integration never silently reroutes existing sessions. - [ ] Test encryption round trips and redaction, ownership isolation, concurrent users with different OpenAI keys, key replacement during a session, missing - integration behavior, and migration fallback precedence. + integration behavior, and the absence of environment-key fallback. * Phase 2: Add Anthropic and OpenRouter API keys @@ -116,16 +118,10 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations keep the feature disabled if any required Codex capability or identity check fails. -* Phase 4: Remove boot-time LLM credentials and harden operations +* Phase 4: Harden operations - [ ] Add onboarding and session/model-settings feedback that links users to every missing provider integration required by their selected role mapping. -- [ ] Remove provider API keys from required runtime configuration and remove - user request fallback to ~OPENAI_API_KEY~, ~ANTHROPIC_API_KEY~, and - ~OPENROUTER_API_KEY~. -- [ ] Update live-provider CI and development setup to create explicit test-user - integrations without exposing credentials in fixtures, logs, or command - output. - [ ] Add bounded-cardinality telemetry and credential-free security audit logs for connection changes, validation, refresh outcomes, and reauthorization. - [ ] Implement and test bounded-batch encryption-key rotation with a documented diff --git a/rfd/0005/README.adoc b/rfd/0005/README.adoc index b675eae..fbde9bf 100644 --- a/rfd/0005/README.adoc +++ b/rfd/0005/README.adoc @@ -39,6 +39,12 @@ session owner selects the credential. ReqLLM receives that credential through request-local options so concurrent users never share mutable process or application configuration. +Kodo is not yet live, so this proposal does not preserve a runtime migration +fallback to deployment environment keys. Once user-owned integrations land, a +missing user credential fails closed and links to integration settings. The +production adapter never looks up `OPENAI_API_KEY` or another provider key from +the process environment. + The initial provider identities are distinct even when they share a vendor: * `openai` uses an OpenAI Platform API key and Platform billing. @@ -410,19 +416,17 @@ interface. Status updates use a live region, the countdown is not the only expiry cue, and cancelling returns the provider card to a stable disconnected state. -== Migration and compatibility - -The rollout temporarily preserves existing environment-key behavior as a -deployment fallback so current installations and live-provider CI do not stop -working when the schema first ships. Resolution during this interval is: +== Pre-release cutover -. the session owner's active provider integration; then -. the matching legacy deployment environment key, if fallback is enabled. +Kodo has no production users or stored provider integrations to migrate. The +integration schema, request-local credential boundary, missing-integration UX, +and removal of ReqLLM's environment-key path therefore land in the same phase. +There is no deployment fallback, automatic import, or implicit owner for an +existing environment key. -Kodo never copies a deployment key into a user's integration because there is -no safe implicit owner. The fallback is visibly deployment-managed and cannot -be viewed or edited in user settings. It is removed after integration -onboarding and provider-specific live tests are established. +Development and opt-in live-provider tests create an explicit test-user +integration or pass a typed test credential through the same Kodo-owned +boundary. They do not add environment lookup to the production adapter. At the end of this RFD, no LLM API key or OAuth token is required at boot. A credential-encryption key remains required because encrypted database secrets @@ -452,7 +456,7 @@ Kodo applies the following boundaries: Operational documentation covers initial encryption-key provisioning, backup handling, rotation, provider revocation, device-flow failures, refresh failure -rates, and safe rollback while legacy environment fallback exists. +rates, and safe rollback across integration schema and encryption-key changes. == Question to resolve @@ -475,7 +479,8 @@ invalid states and disclose any billed Test action before it runs. Environment keys are simple but preserve one billing identity for every user, require operator involvement, and cannot support user-scoped OAuth refresh. -They remain only as a migration bridge. +Because Kodo is not yet live, retaining them as a migration bridge adds billing +ambiguity without preserving production compatibility. === Write credentials to runner environments @@ -513,11 +518,12 @@ building the production OAuth UI. === Phase 1: Build encrypted integration storage Add the user-owned integration schema, authenticated encryption and key -versioning, scoped context operations, safe inspection, and migration-time -environment fallback. Add the shared responsive settings shell, improve the -existing Account presentation without changing its behavior, and ship the -authenticated integration page with OpenAI Platform API-key connection first. -This establishes the full path from UI through request-local ReqLLM options. +versioning, scoped context operations, safe inspection, explicit live-test +credentials, and fail-closed request-time resolution. Add the shared responsive +settings shell, improve the existing Account presentation without changing its +behavior, and ship the authenticated integration page with OpenAI Platform +API-key connection first. This establishes the full path from UI through +request-local ReqLLM options without a provider-key environment fallback. === Phase 2: Add Anthropic and OpenRouter @@ -532,12 +538,11 @@ OAuth persistence, refresh leases, rotated-token updates, reauthorization, and the ChatGPT Subscription billing identity. Gate availability on Phase 0's compatibility criteria and retain the Beta label. -=== Phase 4: Remove boot-time LLM credentials and harden operations +=== Phase 4: Harden operations -Complete onboarding and missing-integration UX, remove runtime dependence on -provider environment keys, update CI to create explicit test integrations, -exercise multi-node refresh and key rotation, and publish operator guidance. -The encryption key remains the only integration-related boot secret. +Complete operational hardening, exercise multi-node refresh and key rotation, +and publish operator guidance. The encryption key remains the only +integration-related boot secret. == Implementation From 1cd03efb8ac0439dd8dc95b310ad91e71502f025 Mon Sep 17 00:00:00 2001 From: "Darwin D. Wu" Date: Thu, 27 Aug 2026 10:37:18 -0700 Subject: [PATCH 07/29] docs(rfd): define safe token refresh takeover --- rfd/0005/IMPLEMENTATION.org | 21 +++++++++----- rfd/0005/README.adoc | 55 ++++++++++++++++++++++++------------- 2 files changed, 50 insertions(+), 26 deletions(-) diff --git a/rfd/0005/IMPLEMENTATION.org b/rfd/0005/IMPLEMENTATION.org index a5848a4..0a96c94 100644 --- a/rfd/0005/IMPLEMENTATION.org +++ b/rfd/0005/IMPLEMENTATION.org @@ -101,19 +101,26 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations credential generations in the encrypted integration payload without logging device or token material. - [ ] Run just-in-time refreshes as bounded tasks under - ~Kodo.ControlPlaneTaskSupervisor~; do not create a long-lived supervised - process or in-memory refresh timer per integration. -- [ ] Implement database-backed refresh leases, bounded waiter behavior, - compare-and-update protection, atomic refresh-token rotation, lease takeover, - and terminal ~invalid_grant~ reauthorization. + ~Kodo.ControlPlaneTaskSupervisor~; coordinate callers directly through the + database rather than adding a refresh coordinator ~GenServer~, long-lived + supervised process, or in-memory refresh timer per integration. +- [ ] Implement database-time refresh leases whose duration exceeds the hard + HTTP timeout plus a safety margin, bounded waiter behavior, lease takeover, + and credential-generation compare-and-update protection for every success + and failure-derived state change. +- [ ] Let the first generation-matching successful response atomically persist + rotated tokens even after lease expiry, and make a stale or takeover + ~invalid_grant~ reload and wait for a possible generation advance before + requiring reauthorization. - [ ] Add the "ChatGPT Subscription (Beta)" connection flow, billing explanation, progress state, cancellation, quota errors, disconnect, and reauthorization to the authenticated integrations page. - [ ] Route ~openai_codex~ models only through the user's ChatGPT integration and prohibit silent fallback to OpenAI API billing. - [ ] Test authorization ownership and expiry, polling backoff, two-node refresh - contention, delayed refresh responses, rotated refresh tokens, restart - recovery, revocation, and secret redaction. + contention, late success after lease expiry, takeover ~invalid_grant~, node + pause, replacement during refresh, permanently lost rotated-token responses, + restart recovery, revocation, and secret redaction. - [ ] Run the Phase 0 live compatibility suite against the production path and keep the feature disabled if any required Codex capability or identity check fails. diff --git a/rfd/0005/README.adoc b/rfd/0005/README.adoc index fbde9bf..9509bdd 100644 --- a/rfd/0005/README.adoc +++ b/rfd/0005/README.adoc @@ -211,27 +211,44 @@ database lease. The task holds decrypted credentials only for the operation's duration. A task or node failure leaves durable credential state unchanged and the expired lease available for takeover. -Kodo does not start a `DynamicSupervisor` child or keep an in-memory timer for -each integration. Such processes would duplicate across replicas, lose their -schedule during restarts, and still require database coordination. If proactive -refresh becomes necessary, a single supervised sweeper or durable job system -may discover due integrations, but it must invoke the same lease-protected -refresh operation and keep the database as the source of truth. - -Only one control-plane replica may refresh an integration at a time. Kodo uses -a short database-backed refresh lease so it does not hold a database -transaction or row lock across an external HTTP request. Other callers wait for -a bounded interval and reload the integration; an expired lease can be taken -over. The lease owner updates the integration only if its lease and credential -generation still match, preventing a delayed response from overwriting a newer -authorization or refresh. +Callers coordinate directly through the database; Kodo does not add a refresh +coordinator `GenServer`, a `DynamicSupervisor` child, or an in-memory timer per +integration. A process-local coordinator would not prevent another replica +from refreshing, while a cluster-wide process would require leader election +and still need durable fencing during failover. If proactive refresh becomes +necessary, a single supervised sweeper or durable job system may discover due +integrations, but it must invoke the same lease-protected refresh operation and +keep the database as the source of truth. + +Kodo admits one lease holder for an integration at a time. It uses a short +database-backed refresh lease, calculated from database time, so it does not +hold a database transaction or row lock across an external HTTP request. The +lease duration exceeds the hard refresh-request timeout plus a safety margin. +Other callers wait for a bounded interval and reload the integration; an +expired lease can be taken over, although a paused former holder can briefly +overlap the replacement after resuming. + +Every attempt captures the credential generation it read. A successful +response may atomically replace the encrypted payload and advance that +generation even if its lease expired while the task or node was paused, but +only if no refresh, replacement, reconnection, or disconnection has already +advanced it. The first successful response wins; later responses reload the +stored result. Failure-derived state changes use the same generation fencing. An `invalid_grant` or equivalent terminal response marks the integration as -requiring authorization and discards unusable access for request purposes. -Transient provider or network failures preserve the credential, release or -expire the lease, and return a retryable provider-unavailable error. Refresh -tokens are never retried concurrently because providers may rotate and -invalidate them on first use. +requiring authorization and discards unusable access for request purposes only +after its generation check succeeds. A stale-lease or takeover attempt that +receives `invalid_grant` first reloads and waits for a bounded interval so a +late successful response can advance the generation; it must not overwrite +that success with a terminal state. Transient provider or network failures +preserve the credential, release or expire the lease, and return a retryable +provider-unavailable error. + +This protocol minimizes concurrent refresh but cannot make an external token +endpoint exactly once. If the provider rotates a refresh token and Kodo +permanently loses the successful response, the only safe recovery may be user +reauthorization. Kodo documents and tests this boundary rather than silently +retrying an ambiguously consumed token. == User interface and authorization From d8d33c34b5d8e3b5398d09264758c83dd7464398 Mon Sep 17 00:00:00 2001 From: "Darwin D. Wu" Date: Thu, 27 Aug 2026 10:41:34 -0700 Subject: [PATCH 08/29] docs(rfd): bind encrypted credentials to ownership --- rfd/0005/IMPLEMENTATION.org | 15 +++++++++++---- rfd/0005/README.adoc | 30 +++++++++++++++++++++--------- 2 files changed, 32 insertions(+), 13 deletions(-) diff --git a/rfd/0005/IMPLEMENTATION.org b/rfd/0005/IMPLEMENTATION.org index 0a96c94..5fffaa9 100644 --- a/rfd/0005/IMPLEMENTATION.org +++ b/rfd/0005/IMPLEMENTATION.org @@ -26,9 +26,14 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations - [ ] Generate migrations for user-owned provider integrations, unique active user/provider identity, non-secret status metadata, credential generation, and encryption-key version. -- [ ] Add authenticated encryption with unique nonces, versioned keys, safe - redaction, and production configuration that distinguishes the Kodo - credential-encryption key from all LLM provider credentials. +- [ ] Add authenticated encryption with unique nonces, versioned keys, and AEAD + associated data binding integration ID, user ID, provider, authentication + type, and payload format version; keep provider account and workspace + identifiers inside the encrypted payload. +- [ ] Add safe redaction and production configuration that distinguishes the + Kodo credential-encryption key ring from all LLM provider credentials, + validates every current and previous key before replica readiness, and fails + closed on missing keys or payload authentication failures. - [ ] Implement scope-first integration context operations for listing, connecting, replacing, validating, and disconnecting credentials without accepting browser-supplied ownership. @@ -68,7 +73,9 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations required integration never silently reroutes existing sessions. - [ ] Test encryption round trips and redaction, ownership isolation, concurrent users with different OpenAI keys, key replacement during a session, missing - integration behavior, and the absence of environment-key fallback. + integration behavior, ciphertext swaps between users and providers, + tampering, wrong and missing keys, unknown payload versions, nonce + uniqueness, and the absence of environment-key fallback. * Phase 2: Add Anthropic and OpenRouter API keys diff --git a/rfd/0005/README.adoc b/rfd/0005/README.adoc index 9509bdd..15374f3 100644 --- a/rfd/0005/README.adoc +++ b/rfd/0005/README.adoc @@ -153,8 +153,7 @@ contains at least: * a generated identifier, owning user, provider identity, and authentication type; * encrypted credentials and the encryption-key version; -* connection status, provider account identifier when available, and token - expiry; +* connection status and token expiry; * validation and refresh timestamps plus a bounded safe error code; and * normal insertion and update timestamps. @@ -163,13 +162,26 @@ and provider pair. Context functions take the authenticated scope as their first argument and apply the owning user in the query. Changesets never cast a `user_id` supplied by a browser. -Credentials use authenticated encryption with a unique nonce per write. Kodo -requires a deployment-level credential-encryption key, supplied independently -of all LLM providers, and records a key version so ciphertext can be rotated. -Production refuses to persist or decrypt integrations when this key is absent -or malformed. Database encryption does not remove the need to protect the -control-plane process; it protects database copies, backups, and read-only -database access from directly revealing provider credentials. +Credentials use authenticated encryption with a unique nonce per write. API +keys, access and refresh tokens, and provider account or workspace identifiers +remain inside the encrypted payload. Each write binds its ciphertext through +AEAD associated data containing the integration identifier, owning user, +provider identity, authentication type, and credential-payload format version. +A valid payload copied to another integration, user, provider, or +authentication type therefore fails authentication rather than becoming a +cross-user credential or billing path. + +Kodo requires a deployment-level credential-encryption key ring, supplied +independently of all LLM providers, and records a key version so ciphertext can +be rotated. Every production replica validates the complete configured current +and previous key ring before becoming ready to serve integration operations. +Production refuses to persist or decrypt integrations when a required key is +absent or malformed. An AEAD association or authentication failure fails closed +with a bounded corruption error and never falls back or sends the credential to +a provider; recovery requires credential replacement or an operator procedure. +Database encryption does not remove the need to protect the control-plane +process; it protects database copies, backups, and read-only database access +from directly revealing provider credentials. The credential payload is opaque to Ecto query code and never appears in session events, model mappings, changeset errors, exception inspection, From 294fecbed0bb5b890c51a7d9df5f99afcb5ff44d Mon Sep 17 00:00:00 2001 From: "Darwin D. Wu" Date: Thu, 27 Aug 2026 10:46:34 -0700 Subject: [PATCH 09/29] docs(rfd): require Codex compatibility implementation --- rfd/0005/IMPLEMENTATION.org | 29 +++++++++++++----------- rfd/0005/README.adoc | 44 ++++++++++++++++++++++--------------- rfd/README.md | 2 +- 3 files changed, 43 insertions(+), 32 deletions(-) diff --git a/rfd/0005/IMPLEMENTATION.org b/rfd/0005/IMPLEMENTATION.org index 5fffaa9..489376d 100644 --- a/rfd/0005/IMPLEMENTATION.org +++ b/rfd/0005/IMPLEMENTATION.org @@ -2,24 +2,26 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations]]. -* Phase 0: Verify the OpenAI Codex contract +* Phase 0: Implement and verify the OpenAI Codex contract - [ ] Trace the current OpenAI Codex device authorization and refresh flow from - OpenAI's published Codex source and record endpoints, polling behavior, - expiry, token rotation, and account identification used by the spike. -- [ ] Verify ReqLLM's ~openai_codex~ provider with an honest Kodo originator for - a text response, Kodo tool calls, structured output, and every reasoning - effort required by current role mappings. + OpenAI's published Codex source and record the public OAuth client identity, + scopes, device and token endpoints, PKCE behavior, polling rules, expiry, + token rotation, account identification, and client headers. +- [ ] Reproduce the complete device flow in a bounded Kodo live spike and + verify ReqLLM's ~openai_codex~ provider with a Kodo originator for a text + response, Kodo tool calls, structured output, and every reasoning effort + required by current role mappings. - [ ] Verify refresh-token rotation, control-plane restart recovery, quota errors, revocation, and required reauthorization without using a shared OAuth file. -- [ ] Document compatibility criteria, the Beta support boundary, and the - provider response that disables ChatGPT subscription integration rather than - impersonating another client. +- [ ] Document the exact working contract, regression diagnostics, upstream + public-client dependency, and Beta support boundary without claiming formal + OpenAI support unless separately confirmed. - [ ] Gate ChatGPT Subscription UI and requests behind a deployment-level feature flag that defaults to disabled, document operator opt-in and the - private-interface support boundary, and require a later decision before - changing the default. + private-interface support boundary, retain it as an emergency disable + mechanism, and require a later decision before changing the default. * Phase 1: Build encrypted integration storage and OpenAI API keys @@ -129,8 +131,9 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations pause, replacement during refresh, permanently lost rotated-token responses, restart recovery, revocation, and secret redaction. - [ ] Run the Phase 0 live compatibility suite against the production path and - keep the feature disabled if any required Codex capability or identity check - fails. + do not declare ChatGPT/Codex implementation complete while a required + capability fails; classify later contract failures as regressions and never + fall back to Platform billing. * Phase 4: Harden operations diff --git a/rfd/0005/README.adoc b/rfd/0005/README.adoc index 15374f3..6c7470a 100644 --- a/rfd/0005/README.adoc +++ b/rfd/0005/README.adoc @@ -120,23 +120,27 @@ and are not written to logs or telemetry. The OpenAI Codex source publishes the device flow. The downstream `chatgpt.com/backend-api/codex` inference contract is nevertheless less formal than the OpenAI Platform API. Kodo labels this integration "ChatGPT -Subscription (Beta)", identifies itself honestly in supported client headers, -and does not silently replace it with an OpenAI API key if authorization or -subscription quota fails. +Subscription (Beta)", identifies its request origin as Kodo in supported +headers, and does not silently replace it with an OpenAI API key if +authorization or subscription quota fails. Kodo records the exact upstream +public OAuth client identity, scopes, endpoints, and headers on which the +working flow depends and does not describe that compatibility as formal OpenAI +support without separate confirmation. ChatGPT Subscription is disabled by default behind a deployment-level feature flag. An operator may explicitly enable the Beta only after accepting its private-interface support boundary and running the compatibility check for device authorization, a tool-calling turn, structured output used by review and -search roles, refresh-token rotation, restart recovery, and Kodo's client -identity. Kodo never impersonates another client. An explicit provider response -that rejects Kodo's identity or third-party use marks the integration -unsupported rather than retrying or falling back to Platform billing. +search roles, refresh-token rotation, restart recovery, and the documented +client contract. A provider rejection or contract change is a compatibility +regression to fix; the deployment flag provides an emergency disable mechanism +while affected integrations report an actionable unavailable state. Kodo does +not retry through another client identity or fall back to Platform billing. Enabling ChatGPT Subscription by default requires a later decision backed by -published provider support or direct provider confirmation. Ordinary transient, -quota, and authorization errors do not change the deployment feature flag and -remain scoped to the affected integration. +operational experience with the compatibility and support burden. Ordinary +transient, quota, and authorization errors do not change the deployment feature +flag and remain scoped to the affected integration. Anthropic subscription OAuth is outside this RFD. Anthropic permits API keys and supported cloud-provider credentials for third-party applications but @@ -536,13 +540,15 @@ provider details inside the integration boundary. Each phase is independently reviewable and deployable. Detailed progress is tracked in link:IMPLEMENTATION.org[RFD 5 implementation checklist]. -=== Phase 0: Verify the Codex contract +=== Phase 0: Implement and verify the Codex contract -Run a bounded live spike against the current ReqLLM `openai_codex` provider. -Confirm device authorization mechanics from OpenAI's Codex source, Kodo's -honest client identity, required model capabilities, token refresh, and restart -recovery. Record the supported request shape and a disable criterion before -building the production OAuth UI. +Trace and reproduce the working Codex device flow in a bounded Kodo live spike +against the current ReqLLM `openai_codex` provider. Record the upstream public +OAuth client identity, scopes, device and token endpoints, PKCE and polling +rules, account identification, Kodo originator headers, required model +capabilities, token refresh, and restart recovery. Phase 0 defines the +production contract and regression diagnostics; it is not a gate for omitting +the required ChatGPT/Codex integration. === Phase 1: Build encrypted integration storage @@ -564,8 +570,10 @@ in one session without sharing credentials or billing paths. Add durable user-scoped authorization attempts, asynchronous polling, encrypted OAuth persistence, refresh leases, rotated-token updates, reauthorization, and -the ChatGPT Subscription billing identity. Gate availability on Phase 0's -compatibility criteria and retain the Beta label. +the ChatGPT Subscription billing identity. Require Phase 0's compatibility +suite to pass, retain the Beta label, and treat later failures as regressions +that leave the emergency deployment flag available while Kodo fixes the +contract. === Phase 4: Harden operations diff --git a/rfd/README.md b/rfd/README.md index a2bd35f..bdd695b 100644 --- a/rfd/README.md +++ b/rfd/README.md @@ -11,7 +11,7 @@ should be read. | [2: Extend Kodo to mobile and tablet clients](0002/README.adoc) | Start with a responsive PWA, handle disconnection gracefully, and preserve a path to native clients | | [3: Define behavioral evaluation and release benchmarking](0003/README.adoc) | Separate role quality from multi-turn behavior, calibrate observable scoring, and benchmark frozen release candidates | | [4: Introduce a full-screen terminal interface](0004/README.adoc) | Make Ratatui the default client, retain non-interactive runner hosting, and preserve reusable application semantics | -| [5: Store user-owned LLM provider integrations](0005/README.adoc) | Encrypt provider credentials, resolve them per model request, and support OpenAI, Anthropic, OpenRouter, and ChatGPT subscriptions without boot-time LLM credentials | +| [5: Store user-owned LLM provider integrations](0005/README.adoc) | Encrypt provider credentials, resolve them per model request, and support OpenAI, Anthropic, OpenRouter, and an opt-in ChatGPT/Codex Beta without boot-time LLM credentials | ## Source format From 287dc1981e9e267080bda37089f0e880a8bb4f1e Mon Sep 17 00:00:00 2001 From: "Darwin D. Wu" Date: Thu, 27 Aug 2026 11:53:43 -0700 Subject: [PATCH 10/29] docs(rfd): ship Codex beta without feature flag --- rfd/0005/IMPLEMENTATION.org | 7 +++---- rfd/0005/README.adoc | 26 ++++++++++---------------- rfd/README.md | 2 +- 3 files changed, 14 insertions(+), 21 deletions(-) diff --git a/rfd/0005/IMPLEMENTATION.org b/rfd/0005/IMPLEMENTATION.org index 489376d..659db57 100644 --- a/rfd/0005/IMPLEMENTATION.org +++ b/rfd/0005/IMPLEMENTATION.org @@ -18,10 +18,9 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations - [ ] Document the exact working contract, regression diagnostics, upstream public-client dependency, and Beta support boundary without claiming formal OpenAI support unless separately confirmed. -- [ ] Gate ChatGPT Subscription UI and requests behind a deployment-level - feature flag that defaults to disabled, document operator opt-in and the - private-interface support boundary, retain it as an emergency disable - mechanism, and require a later decision before changing the default. +- [ ] Ship ChatGPT Subscription as a Beta without a deployment feature flag + once its implementation and required compatibility suite pass; surface later + contract regressions as actionable integration-unavailable errors. * Phase 1: Build encrypted integration storage and OpenAI API keys diff --git a/rfd/0005/README.adoc b/rfd/0005/README.adoc index 6c7470a..d553ed4 100644 --- a/rfd/0005/README.adoc +++ b/rfd/0005/README.adoc @@ -127,20 +127,15 @@ public OAuth client identity, scopes, endpoints, and headers on which the working flow depends and does not describe that compatibility as formal OpenAI support without separate confirmation. -ChatGPT Subscription is disabled by default behind a deployment-level feature -flag. An operator may explicitly enable the Beta only after accepting its -private-interface support boundary and running the compatibility check for -device authorization, a tool-calling turn, structured output used by review and -search roles, refresh-token rotation, restart recovery, and the documented -client contract. A provider rejection or contract change is a compatibility -regression to fix; the deployment flag provides an emergency disable mechanism -while affected integrations report an actionable unavailable state. Kodo does -not retry through another client identity or fall back to Platform billing. - -Enabling ChatGPT Subscription by default requires a later decision backed by -operational experience with the compatibility and support burden. Ordinary -transient, quota, and authorization errors do not change the deployment feature -flag and remain scoped to the affected integration. +ChatGPT Subscription ships as a Beta when its implementation and compatibility +suite pass for device authorization, a tool-calling turn, structured output +used by review and search roles, refresh-token rotation, restart recovery, and +the documented client contract. It does not require a deployment feature flag. +A provider rejection or contract change is a compatibility regression to fix; +affected integrations report an actionable unavailable state. Kodo does not +retry through another client identity or fall back to Platform billing. +Ordinary transient, quota, and authorization errors remain scoped to the +affected integration. Anthropic subscription OAuth is outside this RFD. Anthropic permits API keys and supported cloud-provider credentials for third-party applications but @@ -572,8 +567,7 @@ Add durable user-scoped authorization attempts, asynchronous polling, encrypted OAuth persistence, refresh leases, rotated-token updates, reauthorization, and the ChatGPT Subscription billing identity. Require Phase 0's compatibility suite to pass, retain the Beta label, and treat later failures as regressions -that leave the emergency deployment flag available while Kodo fixes the -contract. +that leave affected integrations unavailable while Kodo fixes the contract. === Phase 4: Harden operations diff --git a/rfd/README.md b/rfd/README.md index bdd695b..9b5dec2 100644 --- a/rfd/README.md +++ b/rfd/README.md @@ -11,7 +11,7 @@ should be read. | [2: Extend Kodo to mobile and tablet clients](0002/README.adoc) | Start with a responsive PWA, handle disconnection gracefully, and preserve a path to native clients | | [3: Define behavioral evaluation and release benchmarking](0003/README.adoc) | Separate role quality from multi-turn behavior, calibrate observable scoring, and benchmark frozen release candidates | | [4: Introduce a full-screen terminal interface](0004/README.adoc) | Make Ratatui the default client, retain non-interactive runner hosting, and preserve reusable application semantics | -| [5: Store user-owned LLM provider integrations](0005/README.adoc) | Encrypt provider credentials, resolve them per model request, and support OpenAI, Anthropic, OpenRouter, and an opt-in ChatGPT/Codex Beta without boot-time LLM credentials | +| [5: Store user-owned LLM provider integrations](0005/README.adoc) | Encrypt provider credentials, resolve them per model request, and support OpenAI, Anthropic, OpenRouter, and a ChatGPT/Codex Beta without boot-time LLM credentials | ## Source format From 7e07e9796a75063171451bd1d65f2bdb90b9526d Mon Sep 17 00:00:00 2001 From: "Darwin D. Wu" Date: Thu, 27 Aug 2026 11:57:34 -0700 Subject: [PATCH 11/29] docs(rfd): define integration lifecycle states --- rfd/0005/IMPLEMENTATION.org | 21 +++++++----- rfd/0005/README.adoc | 66 ++++++++++++++++++++++++++----------- 2 files changed, 59 insertions(+), 28 deletions(-) diff --git a/rfd/0005/IMPLEMENTATION.org b/rfd/0005/IMPLEMENTATION.org index 659db57..d31a7de 100644 --- a/rfd/0005/IMPLEMENTATION.org +++ b/rfd/0005/IMPLEMENTATION.org @@ -24,9 +24,9 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations * Phase 1: Build encrypted integration storage and OpenAI API keys -- [ ] Generate migrations for user-owned provider integrations, unique active - user/provider identity, non-secret status metadata, credential generation, - and encryption-key version. +- [ ] Generate migrations for one durable integration per user/provider pair, + separate connection and validation states, non-secret status metadata, + credential generation, and encryption-key version. - [ ] Add authenticated encryption with unique nonces, versioned keys, and AEAD associated data binding integration ID, user ID, provider, authentication type, and payload format version; keep provider account and workspace @@ -37,13 +37,15 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations closed on missing keys or payload authentication failures. - [ ] Implement scope-first integration context operations for listing, connecting, replacing, validating, and disconnecting credentials without - accepting browser-supplied ownership. + accepting browser-supplied ownership; disconnect clears the encrypted payload + but preserves the row, and every credential change advances its generation. - [ ] Add ~Kodo.LLM~ credential values and translate them into request-local ReqLLM options without mutating process environment, application config, or shared credential files. - [ ] Resolve credentials from the session owner and each role model's provider - before recording an invocation, with actionable missing, invalid, and - disconnected integration errors. + before recording an invocation, with actionable missing, invalid, + disconnected, and reauthorization-required integration errors; allow + ~unverified~ and validation-~unavailable~ credentials. - [ ] Remove provider-key environment lookup from the production adapter and make missing user integrations fail closed; give development and opt-in live-provider checks explicit test-user integrations or typed test @@ -76,7 +78,8 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations users with different OpenAI keys, key replacement during a session, missing integration behavior, ciphertext swaps between users and providers, tampering, wrong and missing keys, unknown payload versions, nonce - uniqueness, and the absence of environment-key fallback. + uniqueness, lifecycle transitions, generation-fenced asynchronous updates, + and the absence of environment-key fallback. * Phase 2: Add Anthropic and OpenRouter API keys @@ -87,7 +90,9 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations identity for routed models. - [ ] Document whether validation for each API-key provider is unbilled or an explicit potentially billed Test action, and represent saved-but-unverified - credentials separately from valid and invalid credentials. + credentials separately from valid, invalid, and validation-unavailable + credentials; do not classify quota, entitlement, or provider outages as + credential invalidity. - [ ] Extend integration settings and model-selection feedback to distinguish OpenAI API, Anthropic, and OpenRouter providers without displaying saved credentials. diff --git a/rfd/0005/README.adoc b/rfd/0005/README.adoc index d553ed4..7b719db 100644 --- a/rfd/0005/README.adoc +++ b/rfd/0005/README.adoc @@ -53,11 +53,11 @@ The initial provider identities are distinct even when they share a vendor: * `anthropic` uses an Anthropic Console API key. * `openrouter` uses an OpenRouter API key. -One active integration is allowed per user and provider identity in the first -release. Treating `openai` and `openai_codex` separately lets a user connect -both and makes billing and model routing explicit. Supporting several accounts -for one provider may be added later with an explicit integration selection in -model settings; the first release does not add that complexity. +One durable integration row is allowed per user and provider identity in the +first release. Treating `openai` and `openai_codex` separately lets a user +connect both and makes billing and model routing explicit. Supporting several +accounts for one provider may be added later with an explicit integration +selection in model settings; the first release does not add that complexity. Model role mappings continue to own model selection. They do not contain credentials. A mapping may use different providers for primary, search, and @@ -75,6 +75,14 @@ search, and review models with their Platform, subscription, or aggregator billing paths. The preset is applied only after the user chooses "Use recommended models". +A mapping is usable when every role resolves to an allowed provider whose +integration contains a credential, has connection state `connected`, and does +not have validation state `invalid`. Saved-but-`unverified` credentials and +credentials whose latest validation is `unavailable` remain usable. Provider +outages, quota exhaustion, missing model entitlement, and Codex contract +regressions are request outcomes; they do not mutate credential validity or +make another billing path eligible. + Confirmed presets affect new sessions and user defaults; existing session mappings are never rewritten. Disconnecting an integration marks dependent mappings unavailable and links to model settings rather than silently routing @@ -91,11 +99,11 @@ turning provider configuration into a server-side request forgery or credential-exfiltration path. Validation is non-destructive and provider-specific. A transient validation -failure does not expose the credential or silently fall back to another user's -or deployment's key. Kodo records a bounded status and safe error code, while -the detailed provider error is redacted before logging or display. A saved -credential is never rendered back to the browser; the UI shows only provider, -status, validation time, and a non-secret fingerprint where useful. +failure does not expose or invalidate the credential. Kodo records a bounded +status and safe error code, while the detailed provider error is redacted +before logging or display. A saved credential is never rendered back to the +browser; the UI shows only provider, status, validation time, and a non-secret +fingerprint where useful. OpenRouter is a provider in its own right rather than a transparent fallback for direct provider integrations. Models routed through OpenRouter use the @@ -156,11 +164,27 @@ contains at least: * validation and refresh timestamps plus a bounded safe error code; and * normal insertion and update timestamps. -The database enforces ownership references and uniqueness of the active user -and provider pair. Context functions take the authenticated scope as their -first argument and apply the owning user in the query. Changesets never cast a +The database enforces ownership references and uniqueness of the user and +provider pair. Context functions take the authenticated scope as their first +argument and apply the owning user in the query. Changesets never cast a `user_id` supplied by a browser. +Connection and validation are separate state axes. Connection is one of +`disconnected`, `connected`, or `reauthorization_required`. Validation is one +of `unverified`, `valid`, `invalid`, or `unavailable`. Connecting, replacing, +or reauthorizing stores a credential, sets connection to `connected`, resets +validation to `unverified`, and advances the credential generation. +Disconnecting clears the encrypted payload, sets connection to `disconnected`, +and advances the generation while preserving the integration row. + +An authentication probe sets validation to `valid` only on success or to +`invalid` only on a confirmed authentication failure. Network errors, rate +limits, and provider outages produce `unavailable` without clearing the +credential or blocking inference. Quota and model-entitlement errors remain +request outcomes. Every asynchronous validation or provider-derived status +update compares the credential generation it read so it cannot overwrite a +replacement, reconnection, or disconnection. + Credentials use authenticated encryption with a unique nonce per write. API keys, access and refresh tokens, and provider account or workspace identifiers remain inside the encrypted payload. Each write binds its ciphertext through @@ -184,9 +208,10 @@ from directly revealing provider credentials. The credential payload is opaque to Ecto query code and never appears in session events, model mappings, changeset errors, exception inspection, -telemetry metadata, or client responses. Deleting an integration removes its -credential payload. Backup retention and provider-side revocation remain -operational concerns and are documented with the deployment procedure. +telemetry metadata, or client responses. Disconnecting an integration removes +its credential payload from the durable database row. Backup retention and +provider-side revocation remain operational concerns and are documented with +the deployment procedure. == Request-time resolution @@ -197,10 +222,11 @@ uses `System.put_env/2`, `Application.put_env/3`, ReqLLM's global key store, or shared OAuth file. Credential resolution occurs before Kodo persists a model invocation as -started. A missing, disconnected, or reauthorization-required integration -returns a domain error naming the provider and required user action without -including credential material. The session can accept a later turn after the -user reconnects the provider. Provider errors must not trigger fallback to a +started. A missing integration, a `disconnected` or +`reauthorization_required` connection, or an `invalid` validation returns a +domain error naming the provider and required user action without including +credential material. The session can accept a later turn after the user +reconnects the provider. Provider errors must not trigger fallback to a differently billed integration unless model settings explicitly define such a fallback in a future proposal. From 157402c8de4623191012f10f40e549219450a706 Mon Sep 17 00:00:00 2001 From: "Darwin D. Wu" Date: Thu, 27 Aug 2026 12:00:57 -0700 Subject: [PATCH 12/29] docs(rfd): enforce canonical provider identity --- rfd/0005/IMPLEMENTATION.org | 26 +++++++++++++++++++------- rfd/0005/README.adoc | 37 +++++++++++++++++++++++++++++-------- 2 files changed, 48 insertions(+), 15 deletions(-) diff --git a/rfd/0005/IMPLEMENTATION.org b/rfd/0005/IMPLEMENTATION.org index d31a7de..57b3fa2 100644 --- a/rfd/0005/IMPLEMENTATION.org +++ b/rfd/0005/IMPLEMENTATION.org @@ -41,11 +41,20 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations but preserves the row, and every credential change advances its generation. - [ ] Add ~Kodo.LLM~ credential values and translate them into request-local ReqLLM options without mutating process environment, application config, or - shared credential files. -- [ ] Resolve credentials from the session owner and each role model's provider - before recording an invocation, with actionable missing, invalid, - disconnected, and reauthorization-required integration errors; allow - ~unverified~ and validation-~unavailable~ credentials. + shared credential files; include the canonical provider identity in every + typed credential. +- [ ] Resolve each model through ~ReqLLM.model/1~, allow only ~openai~, + ~openai_codex~, ~anthropic~, and ~openrouter~, and fetch the session owner's + integration using the resulting ~LLMDB.Model.provider~; replace model-string + splitting with that resolved provider when storing derived display data. +- [ ] Require exact equality between resolved model and credential providers + before recording an invocation or making an HTTP request, with actionable + missing, invalid, disconnected, reauthorization-required, malformed-model, + unsupported-provider, and mismatch errors; allow ~unverified~ and + validation-~unavailable~ credentials. +- [ ] Persist credential-free invocation provenance containing canonical + provider, authentication type, and Platform, subscription, or aggregator + billing path. - [ ] Remove provider-key environment lookup from the production adapter and make missing user integrations fail closed; give development and opt-in live-provider checks explicit test-user integrations or typed test @@ -73,13 +82,16 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations billing paths and apply it only after "Use recommended models" confirmation. - [ ] Test that connecting another provider leaves a usable mapping unchanged, confirmed presets affect only defaults and new sessions, and disconnecting a - required integration never silently reroutes existing sessions. + required integration derives an unavailable mapping without rewriting or + silently rerouting existing sessions. - [ ] Test encryption round trips and redaction, ownership isolation, concurrent users with different OpenAI keys, key replacement during a session, missing integration behavior, ciphertext swaps between users and providers, tampering, wrong and missing keys, unknown payload versions, nonce uniqueness, lifecycle transitions, generation-fenced asynchronous updates, - and the absence of environment-key fallback. + catalog aliases, inline model specs, OpenRouter underlying vendors, malformed + models, credential/provider mismatches, billing provenance, and the absence + of environment-key fallback. * Phase 2: Add Anthropic and OpenRouter API keys diff --git a/rfd/0005/README.adoc b/rfd/0005/README.adoc index 7b719db..dcb37d5 100644 --- a/rfd/0005/README.adoc +++ b/rfd/0005/README.adoc @@ -84,9 +84,11 @@ regressions are request outcomes; they do not mutate credential validity or make another billing path eligible. Confirmed presets affect new sessions and user defaults; existing session -mappings are never rewritten. Disconnecting an integration marks dependent -mappings unavailable and links to model settings rather than silently routing -requests through a differently billed provider. +mappings are never rewritten. Mapping availability is derived from the current +model resolution and integration state rather than persisted into each mapping. +Disconnecting an integration therefore makes dependent mappings unavailable +and links to model settings without rewriting them or silently routing requests +through a differently billed provider. == Provider support @@ -215,11 +217,24 @@ the deployment procedure. == Request-time resolution -The provider-independent LLM boundary accepts a Kodo credential value in its -generation options. The ReqLLM adapter translates an API-key credential to -`:api_key` and a Codex credential to request-local OAuth options. No request -uses `System.put_env/2`, `Application.put_env/3`, ReqLLM's global key store, or a -shared OAuth file. +Kodo resolves each model specification through `ReqLLM.model/1` before looking +up a credential. The `provider` on the resulting `LLMDB.Model` is canonical; +provider text derived from a stored model string is display data and cannot +select a credential. This RFD allows only `openai`, `openai_codex`, `anthropic`, +and `openrouter`. Unknown, malformed, and unsupported models fail before a +provider request. + +The provider-independent LLM boundary accepts the resolved model and a typed +Kodo credential value containing its provider identity. It fetches the +integration by the canonical provider and requires exact equality between the +resolved model provider and credential provider before recording an invocation +or making an HTTP request. This check applies equally to catalog aliases, +inline model specifications, and OpenRouter models whose underlying model +vendor is OpenAI or Anthropic. + +The ReqLLM adapter translates an API-key credential to `:api_key` and a Codex +credential to request-local OAuth options. No request uses `System.put_env/2`, +`Application.put_env/3`, ReqLLM's global key store, or a shared OAuth file. Credential resolution occurs before Kodo persists a model invocation as started. A missing integration, a `disconnected` or @@ -230,6 +245,12 @@ reconnects the provider. Provider errors must not trigger fallback to a differently billed integration unless model settings explicitly define such a fallback in a future proposal. +Each started invocation records non-secret billing provenance from this +resolved boundary: canonical provider, authentication type, and billing path +(`platform`, `subscription`, or `aggregator`). It never records credential +material. This provenance remains stable even if the integration or mapping is +later replaced. + Model validation remains credential-independent. Provider capability checks may resolve model metadata without making a billed request, while connection validation and inference use the session owner's integration. From d95d90ef309090d463687726f9cad351c0fc24c8 Mon Sep 17 00:00:00 2001 From: "Darwin D. Wu" Date: Thu, 27 Aug 2026 15:40:27 -0700 Subject: [PATCH 13/29] docs(rfd): separate model identity from billing route --- rfd/0005/IMPLEMENTATION.org | 23 ++++++++-- rfd/0005/README.adoc | 89 ++++++++++++++++++++++++------------- 2 files changed, 78 insertions(+), 34 deletions(-) diff --git a/rfd/0005/IMPLEMENTATION.org b/rfd/0005/IMPLEMENTATION.org index 57b3fa2..4dbc170 100644 --- a/rfd/0005/IMPLEMENTATION.org +++ b/rfd/0005/IMPLEMENTATION.org @@ -12,6 +12,9 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations verify ReqLLM's ~openai_codex~ provider with a Kodo originator for a text response, Kodo tool calls, structured output, and every reasoning effort required by current role mappings. +- [ ] Record exact model identities available through both ~openai~ and + ~openai_codex~ and verify each route preserves the same role capability + contract; do not treat merely similar models as interchangeable. - [ ] Verify refresh-token rotation, control-plane restart recovery, quota errors, revocation, and required reauthorization without using a shared OAuth file. @@ -47,6 +50,9 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations ~openai_codex~, ~anthropic~, and ~openrouter~, and fetch the session owner's integration using the resulting ~LLMDB.Model.provider~; replace model-string splitting with that resolved provider when storing derived display data. +- [ ] Persist exact model identity separately from execution provider and + billing route in every session role mapping; user and repository default + changes affect new sessions only. - [ ] Require exact equality between resolved model and credential providers before recording an invocation or making an HTTP request, with actionable missing, invalid, disconnected, reauthorization-required, malformed-model, @@ -140,12 +146,23 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations - [ ] Add the "ChatGPT Subscription (Beta)" connection flow, billing explanation, progress state, cancellation, quota errors, disconnect, and reauthorization to the authenticated integrations page. -- [ ] Route ~openai_codex~ models only through the user's ChatGPT integration - and prohibit silent fallback to OpenAI API billing. +- [ ] Add an explicit per-session execution-route change between OpenAI + Platform and ChatGPT Subscription only when the destination supports every + affected role's exact preserved model and capability contract; show billing + confirmation and append a durable route-change event. +- [ ] Freeze the route for an in-flight turn and its continuations, apply a + confirmed route change at the next turn boundary, and replay normalized + visible assistant/tool history without provider-specific reasoning, + signature, or cache state. +- [ ] Route requests whose current execution route is ~openai_codex~ only + through the user's ChatGPT integration and prohibit silent fallback to OpenAI + API billing. - [ ] Test authorization ownership and expiry, polling backoff, two-node refresh contention, late success after lease expiry, takeover ~invalid_grant~, node pause, replacement during refresh, permanently lost rotated-token responses, - restart recovery, revocation, and secret redaction. + restart recovery, revocation, exact-model route changes, incompatible route + rejection, in-flight route freezing, normalized cross-route replay, no + automatic route fallback, and secret redaction. - [ ] Run the Phase 0 live compatibility suite against the production path and do not declare ChatGPT/Codex implementation complete while a required capability fails; classify later contract failures as regressions and never diff --git a/rfd/0005/README.adoc b/rfd/0005/README.adoc index dcb37d5..5e5ed29 100644 --- a/rfd/0005/README.adoc +++ b/rfd/0005/README.adoc @@ -34,10 +34,11 @@ workspace tools and does not call LLM providers directly. == Decision Kodo will persist user-owned provider integrations and resolve one for every -model request. The model specification selects the provider; the authenticated -session owner selects the credential. ReqLLM receives that credential through -request-local options so concurrent users never share mutable process or -application configuration. +model request. A session role mapping selects an exact model identity and an +execution route selects the provider and billing path; the authenticated +session owner selects the credential for that route. ReqLLM receives that +credential through request-local options so concurrent users never share +mutable process or application configuration. Kodo is not yet live, so this proposal does not preserve a runtime migration fallback to deployment environment keys. Once user-owned integrations land, a @@ -59,21 +60,43 @@ connect both and makes billing and model routing explicit. Supporting several accounts for one provider may be added later with an explicit integration selection in model settings; the first release does not add that complexity. -Model role mappings continue to own model selection. They do not contain -credentials. A mapping may use different providers for primary, search, and -review roles, in which case the session owner must have every required -integration. Kodo checks this when starting a turn and resolves the current -credential again immediately before each provider request. Key replacement, -token refresh, disconnection, or revocation therefore takes effect without -rewriting session history. - -Connecting a provider and changing model routing are separate actions. If the -current role mapping remains usable, connecting another provider does not -prompt for or apply a routing change. If a user has no usable mapping, Kodo -presents one recommended provider-compatible preset and shows its primary, -search, and review models with their Platform, subscription, or aggregator -billing paths. The preset is applied only after the user chooses "Use -recommended models". +Model role mappings continue to own model selection. Each session preserves +the exact primary, search, and review model identities selected at creation; +later user or repository default changes affect new sessions only. Mappings do +not contain credentials. Their execution routes may use different providers, +in which case the session owner must have every required integration. Kodo +checks this when starting a turn and resolves the current credential again +immediately before each provider request. Key replacement, token refresh, +disconnection, or revocation therefore takes effect without rewriting session +history. + +The first release permits an explicit execution-route change between +`openai` Platform billing and `openai_codex` ChatGPT subscription billing while +preserving the role's exact model identity. Kodo accepts the change only when +both routes expose that exact model and satisfy the same role capability +contract. It records a durable route-change event, shows the destination +billing path for confirmation, and applies the new route at the next turn +boundary; an in-flight turn and all its continuations finish on the old route. +Quota, disconnection, or provider failure never triggers this change +automatically. + +After a route change, Kodo replays normalized visible assistant content, tool +calls, and tool results while omitting provider-specific reasoning signatures, +encrypted reasoning state, and cache metadata that cannot safely cross the +boundary. The visible conversation and selected model remain intact, but +provider-private cache or hidden reasoning continuity is not promised. If the +destination does not expose the exact model, Kodo rejects the change and keeps +the current route. Switching to a merely similar model is deferred to a future +decision backed by behavioral benchmarks and scoring evidence that the quality +change is acceptable. + +Connecting a provider, changing user model defaults, and changing an existing +session's execution route are separate actions. If the current role mapping +remains usable, connecting another provider does not prompt for or apply either +change. If a user has no usable mapping, Kodo presents one recommended +provider-compatible preset and shows its primary, search, and review models +with their Platform, subscription, or aggregator billing paths. The preset is +applied only after the user chooses "Use recommended models". A mapping is usable when every role resolves to an allowed provider whose integration contains a credential, has connection state `connected`, and does @@ -83,12 +106,14 @@ outages, quota exhaustion, missing model entitlement, and Codex contract regressions are request outcomes; they do not mutate credential validity or make another billing path eligible. -Confirmed presets affect new sessions and user defaults; existing session -mappings are never rewritten. Mapping availability is derived from the current -model resolution and integration state rather than persisted into each mapping. -Disconnecting an integration therefore makes dependent mappings unavailable -and links to model settings without rewriting them or silently routing requests -through a differently billed provider. +Confirmed presets affect new sessions and user defaults; they never rewrite an +existing session's model identities or execution routes. Only the explicit, +same-model route-change action above can change an existing route. Mapping +availability is derived from the current model resolution and integration state +rather than persisted into each mapping. Disconnecting an integration therefore +makes dependent mappings unavailable and links to model settings without +rewriting them or silently routing requests through a differently billed +provider. == Provider support @@ -217,12 +242,14 @@ the deployment procedure. == Request-time resolution -Kodo resolves each model specification through `ReqLLM.model/1` before looking -up a credential. The `provider` on the resulting `LLMDB.Model` is canonical; -provider text derived from a stored model string is display data and cannot -select a credential. This RFD allows only `openai`, `openai_codex`, `anthropic`, -and `openrouter`. Unknown, malformed, and unsupported models fail before a -provider request. +Kodo combines the session's preserved model identity with its current execution +route and resolves that provider-specific model specification through +`ReqLLM.model/1` before looking up a credential. The `provider` on the resulting +`LLMDB.Model` is canonical and must equal the selected route; provider text +derived from a stored model string is display data and cannot select a +credential. This RFD allows only `openai`, `openai_codex`, `anthropic`, and +`openrouter`. Unknown, malformed, unsupported, and route-incompatible models +fail before a provider request. The provider-independent LLM boundary accepts the resolved model and a typed Kodo credential value containing its provider identity. It fetches the From 465e795db86ce9ed04df2af0c0842866d36c2e11 Mon Sep 17 00:00:00 2001 From: "Darwin D. Wu" Date: Thu, 27 Aug 2026 16:39:37 -0700 Subject: [PATCH 14/29] docs(rfd): require explicit LLM credential ownership --- rfd/0005/IMPLEMENTATION.org | 24 +++++++++++++++++------ rfd/0005/README.adoc | 39 +++++++++++++++++++++++++++---------- 2 files changed, 47 insertions(+), 16 deletions(-) diff --git a/rfd/0005/IMPLEMENTATION.org b/rfd/0005/IMPLEMENTATION.org index 4dbc170..72a2f20 100644 --- a/rfd/0005/IMPLEMENTATION.org +++ b/rfd/0005/IMPLEMENTATION.org @@ -46,6 +46,9 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations ReqLLM options without mutating process environment, application config, or shared credential files; include the canonical provider identity in every typed credential. +- [ ] Require an explicitly owned typed credential in the ~Kodo.LLM~ text/tool + and structured-output generation callbacks and update primary, search, + review, and every direct adapter caller to supply one. - [ ] Resolve each model through ~ReqLLM.model/1~, allow only ~openai~, ~openai_codex~, ~anthropic~, and ~openrouter~, and fetch the session owner's integration using the resulting ~LLMDB.Model.provider~; replace model-string @@ -62,9 +65,16 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations provider, authentication type, and Platform, subscription, or aggregator billing path. - [ ] Remove provider-key environment lookup from the production adapter and - make missing user integrations fail closed; give development and opt-in - live-provider checks explicit test-user integrations or typed test - credentials through the same Kodo-owned boundary. + make missing user integrations fail closed; update the evaluation runner, + development setup, and opt-in live-provider checks to require an explicit + test user with encrypted integrations through the same Kodo-owned boundary. +- [ ] Pass only API keys or current OAuth access tokens and required account + identifiers into ReqLLM; keep refresh tokens, expiry, generation, and rotation + state inside Kodo. +- [ ] Preflight every role integration before accepting a turn, resolve again + immediately before every generation call, and normalize Req/ReqLLM failures + into bounded domain errors before persistence, inspection, logging, + telemetry, or browser display. - [ ] Build a shared responsive settings shell with Account and Integrations navigation, a framed desktop/tablet workspace, compact persistent rail, deep-linkable detail pane, clear return action, accessible narrow-screen @@ -96,8 +106,9 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations tampering, wrong and missing keys, unknown payload versions, nonce uniqueness, lifecycle transitions, generation-fenced asynchronous updates, catalog aliases, inline model specs, OpenRouter underlying vendors, malformed - models, credential/provider mismatches, billing provenance, and the absence - of environment-key fallback. + models, credential/provider mismatches, billing provenance, text and object + generation ownership, evaluation ownership, safe provider errors, secret + redaction, and the absence of environment-key fallback. * Phase 2: Add Anthropic and OpenRouter API keys @@ -118,7 +129,8 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations search, and review, including absence or revocation of one required integration during a turn. - [ ] Add opt-in live-provider checks for OpenAI API, Anthropic, and OpenRouter - while keeping ordinary tests independent of external provider access. + using explicit test-user integrations while keeping ordinary tests + independent of external provider access. * Phase 3: Add ChatGPT subscription device authorization beta diff --git a/rfd/0005/README.adoc b/rfd/0005/README.adoc index 5e5ed29..a6fcbab 100644 --- a/rfd/0005/README.adoc +++ b/rfd/0005/README.adoc @@ -29,7 +29,9 @@ provider accounts. The existing execution boundary is otherwise suitable. Agent inference runs in the Phoenix control plane, sessions have a durable owning user, and model role mappings identify the provider used for each request. The Rust runner executes -workspace tools and does not call LLM providers directly. +workspace tools and does not call LLM providers directly. The evaluation runner +currently calls the LLM adapter without a durable session owner, so this RFD +must bring that path under an explicit test-user integration as well. == Decision @@ -40,6 +42,12 @@ session owner selects the credential for that route. ReqLLM receives that credential through request-local options so concurrent users never share mutable process or application configuration. +Every text, tool-calling, and structured-output generation requires an explicit +typed Kodo credential whose owner is known before the adapter is called. This +includes primary, search, review, evaluation, and live-provider paths. No +production adapter entry point infers an owner or credential from ambient +configuration. + Kodo is not yet live, so this proposal does not preserve a runtime migration fallback to deployment environment keys. Once user-owned integrations land, a missing user credential fails closed and links to integration settings. The @@ -65,10 +73,10 @@ the exact primary, search, and review model identities selected at creation; later user or repository default changes affect new sessions only. Mappings do not contain credentials. Their execution routes may use different providers, in which case the session owner must have every required integration. Kodo -checks this when starting a turn and resolves the current credential again -immediately before each provider request. Key replacement, token refresh, -disconnection, or revocation therefore takes effect without rewriting session -history. +preflights every role integration before accepting a turn and resolves the +current credential again immediately before each provider request. Key +replacement, token refresh, disconnection, or revocation therefore takes effect +without rewriting session history. The first release permits an explicit execution-route change between `openai` Platform billing and `openai_codex` ChatGPT subscription billing while @@ -260,8 +268,11 @@ inline model specifications, and OpenRouter models whose underlying model vendor is OpenAI or Anthropic. The ReqLLM adapter translates an API-key credential to `:api_key` and a Codex -credential to request-local OAuth options. No request uses `System.put_env/2`, -`Application.put_env/3`, ReqLLM's global key store, or a shared OAuth file. +credential to request-local OAuth options. Only the current API key or OAuth +access token and required decrypted account identifier cross into ReqLLM; +refresh tokens, expiry, generation, and rotation state remain in Kodo. No +request uses `System.put_env/2`, `Application.put_env/3`, ReqLLM's global key +store, or a shared OAuth file. Credential resolution occurs before Kodo persists a model invocation as started. A missing integration, a `disconnected` or @@ -278,6 +289,12 @@ resolved boundary: canonical provider, authentication type, and billing path material. This provenance remains stable even if the integration or mapping is later replaced. +Kodo classifies Req and ReqLLM failures into bounded domain errors before they +leave the LLM boundary. Session events, exception inspection, logs, telemetry, +and browser responses may contain only the provider, operation, bounded outcome +class, and actionable user guidance; they never contain raw provider bodies, +headers, request options, URLs with secrets, or inspected credential values. + Model validation remains credential-independent. Provider capability checks may resolve model metadata without making a billed request, while connection validation and inference use the session owner's integration. @@ -526,9 +543,11 @@ and removal of ReqLLM's environment-key path therefore land in the same phase. There is no deployment fallback, automatic import, or implicit owner for an existing environment key. -Development and opt-in live-provider tests create an explicit test-user -integration or pass a typed test credential through the same Kodo-owned -boundary. They do not add environment lookup to the production adapter. +Development, evaluation, and opt-in live-provider runs require an explicit test +user whose encrypted integrations are provisioned through the Kodo context. +Ordinary tests use fake adapters and fake typed credentials without external +provider access. Neither path adds environment lookup or owner inference to the +production adapter. At the end of this RFD, no LLM API key or OAuth token is required at boot. A credential-encryption key remains required because encrypted database secrets From 0b87d16ccf75e997088bf90eae24fac3e7ea9807 Mon Sep 17 00:00:00 2001 From: "Darwin D. Wu" Date: Thu, 27 Aug 2026 16:57:59 -0700 Subject: [PATCH 15/29] docs(rfd): coordinate device authorization claims --- rfd/0005/IMPLEMENTATION.org | 28 ++++++++++++++++++++++------ rfd/0005/README.adoc | 25 +++++++++++++++++++++++++ 2 files changed, 47 insertions(+), 6 deletions(-) diff --git a/rfd/0005/IMPLEMENTATION.org b/rfd/0005/IMPLEMENTATION.org index 72a2f20..9a3adb6 100644 --- a/rfd/0005/IMPLEMENTATION.org +++ b/rfd/0005/IMPLEMENTATION.org @@ -134,12 +134,25 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations * Phase 3: Add ChatGPT subscription device authorization beta -- [ ] Generate persistence for expiring, user-scoped, single-use device - authorization attempts with encrypted sensitive fields, refresh lease state, - and bounded cleanup. +- [ ] Generate persistence for one active, expiring, user-scoped, single-use + device authorization attempt per ~openai_codex~ integration, including + encrypted sensitive fields, attempt and expected integration generations, + provider deadline and polling interval, database-time claim state, and + bounded cleanup. - [ ] Implement device-code creation, interval-aware asynchronous polling, cancellation, expiry, terminal errors, and authorization-code exchange using Req rather than embedded browser scripts. +- [ ] Trigger claims only from attempt creation or LiveView mount/remount, use + one conditional update against the exact attempt row, and run a successful + claim as a finite task under ~Kodo.ControlPlaneTaskSupervisor~ that survives + the originating LiveView closing. +- [ ] Do not run a device-polling sweeper on every replica; expire attempts + lazily and delete terminal rows in bounded batches, using one database-elected + cleanup owner only if periodic cleanup becomes necessary. +- [ ] Make a new attempt atomically supersede the prior attempt and fence + cancellation, expiry, completion, replacement, and disconnection by attempt + generation; install OAuth credentials only when both attempt and expected + integration generations still match. - [ ] Store access tokens, refresh tokens, account identifiers, expiries, and credential generations in the encrypted integration payload without logging device or token material. @@ -170,9 +183,12 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations through the user's ChatGPT integration and prohibit silent fallback to OpenAI API billing. - [ ] Test authorization ownership and expiry, polling backoff, two-node refresh - contention, late success after lease expiry, takeover ~invalid_grant~, node - pause, replacement during refresh, permanently lost rotated-token responses, - restart recovery, revocation, exact-model route changes, incompatible route + contention, multi-node and concurrent-tab device claims, duplicate exchange + and completion, LiveView closure and remount, node restart, superseded and + stale attempts, cancellation/replacement/disconnection races, late refresh + success after lease expiry, takeover ~invalid_grant~, node pause, replacement + during refresh, permanently lost rotated-token responses, refresh restart + recovery, revocation, exact-model route changes, incompatible route rejection, in-flight route freezing, normalized cross-route replay, no automatic route fallback, and secret redaction. - [ ] Run the Phase 0 live compatibility suite against the production path and diff --git a/rfd/0005/README.adoc b/rfd/0005/README.adoc index a6fcbab..dd89778 100644 --- a/rfd/0005/README.adoc +++ b/rfd/0005/README.adoc @@ -160,6 +160,31 @@ provider backoff, and stops on completion, cancellation, expiry, or a terminal provider error. Attempt identifiers and one-time codes are treated as secrets and are not written to logs or telemetry. +The database permits one active attempt per user and `openai_codex` +integration. Each attempt records an attempt generation, expected integration +generation, state, provider deadline and polling interval, encrypted sensitive +fields, and a database-time claim lease. Starting a new attempt atomically +cancels and supersedes the prior attempt. Cancellation, expiry, completion, +replacement, and disconnection transitions compare the attempt generation; +completion installs credentials only while the attempt remains active and both +its attempt and expected integration generations still match. + +Device polling is connection-driven rather than replica-driven. The node that +handles creation or a LiveView mount attempts one conditional claim update +against that attempt's row and, on success, starts one finite polling task under +`Kodo.ControlPlaneTaskSupervisor`. Closing the LiveView does not cancel that +task. After a node failure, a browser reconnect or remount can resume the +persisted attempt when the old claim expires. Concurrent tabs may contend for +the same row, but replicas with no connection for that attempt perform no work. +Kodo does not run a per-replica polling sweeper, so claim traffic is +proportional to active device flows rather than cluster size. + +Duplicate or stale polling and code-exchange results become no-ops after their +generation check and cannot replace newer credentials. Expiry is applied +lazily when an attempt is read, claimed, or superseded. Terminal rows are +deleted in bounded batches; if periodic cleanup later becomes necessary, one +database-elected cleanup owner runs it rather than every replica. + The OpenAI Codex source publishes the device flow. The downstream `chatgpt.com/backend-api/codex` inference contract is nevertheless less formal than the OpenAI Platform API. Kodo labels this integration "ChatGPT From b64f45366ff2ffd507834fc3105a8df6e3f55dfc Mon Sep 17 00:00:00 2001 From: "Darwin D. Wu" Date: Thu, 27 Aug 2026 17:36:40 -0700 Subject: [PATCH 16/29] docs(rfd): define integration authorization controls --- rfd/0005/IMPLEMENTATION.org | 27 ++++++++++++++++++++++----- rfd/0005/README.adoc | 31 ++++++++++++++++++++++++++----- 2 files changed, 48 insertions(+), 10 deletions(-) diff --git a/rfd/0005/IMPLEMENTATION.org b/rfd/0005/IMPLEMENTATION.org index 9a3adb6..9b95698 100644 --- a/rfd/0005/IMPLEMENTATION.org +++ b/rfd/0005/IMPLEMENTATION.org @@ -87,15 +87,26 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations balanced settings surfaces without changing generated authentication, confirmation, or sudo-mode behavior. - [ ] Add ~/integrations~ to the existing authenticated LiveView session and - browser pipeline, require sudo mode for credential changes, and implement the - OpenAI Platform API-key provider card, contextual form, connection status, - loading state, and accessible feedback. + browser pipeline; require a fresh server-side sudo check for API-key + submission or replacement, authorization start, and disconnection; preserve + the intended action across reauthentication; and implement the OpenAI + Platform API-key provider card, contextual form, connection status, loading + state, and accessible feedback. +- [ ] Permit an active generation-fenced authorization attempt to complete + after sudo expiry, require authenticated ownership but not fresh sudo for + cancellation, and keep status viewing, automatic validation, and ordinary + inference outside sudo mode. +- [ ] Add every provider secret field to Phoenix parameter filtering before + shipping its form, and test that filtered request logs, changeset errors, + exception inspection, and LiveView events cannot expose credential material. - [ ] Add LiveView selector tests for the settings shell, section navigation, provider cards, contextual forms, and device-flow states, plus representative desktop, tablet, and narrow-mobile browser checks for the sketched layouts. - [ ] Keep provider connection separate from routing changes; when no usable mapping exists, present one provider-compatible role preset with explicit billing paths and apply it only after "Use recommended models" confirmation. +- [ ] Use an explicit billing confirmation rather than sudo mode for a + same-model execution-route change during an active session. - [ ] Test that connecting another provider leaves a usable mapping unchanged, confirmed presets affect only defaults and new sessions, and disconnecting a required integration derives an unavailable mapping without rewriting or @@ -108,7 +119,10 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations catalog aliases, inline model specs, OpenRouter underlying vendors, malformed models, credential/provider mismatches, billing provenance, text and object generation ownership, evaluation ownership, safe provider errors, secret - redaction, and the absence of environment-key fallback. + redaction, the absence of environment-key fallback, cross-user sensitive + actions, expired sudo mode, stale LiveView sockets, post-sudo authorization + completion, ownership-checked cancellation, and explicit route-change billing + confirmation. * Phase 2: Add Anthropic and OpenRouter API keys @@ -201,7 +215,10 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations - [ ] Add onboarding and session/model-settings feedback that links users to every missing provider integration required by their selected role mapping. - [ ] Add bounded-cardinality telemetry and credential-free security audit logs - for connection changes, validation, refresh outcomes, and reauthorization. + for API-key submission and replacement, authorization start/completion and + cancellation, disconnection, validation outcomes, execution-route changes, + refresh outcomes, and reauthorization; permit internal actor and integration + IDs only in audit records, never metric labels. - [ ] Implement and test bounded-batch encryption-key rotation with a documented previous-key migration window and rollback procedure. - [ ] Verify provider credentials never reach runner registration, runner diff --git a/rfd/0005/README.adoc b/rfd/0005/README.adoc index dd89778..9d27993 100644 --- a/rfd/0005/README.adoc +++ b/rfd/0005/README.adoc @@ -382,9 +382,22 @@ retrying an ambiguously consumed token. Integration management lives at `/integrations` inside the existing `:require_authenticated_user` LiveView session and browser pipeline. The route requires login because every integration is user-owned and its status controls -that user's model requests. Connecting, replacing, and disconnecting a -credential require sudo mode because they change access to a separately billed -external account. +that user's model requests. Submitting or replacing an API key, starting OAuth +or device reauthorization, and disconnecting an integration require sudo mode +because they change access to a separately billed external account. Kodo checks +recent authentication again in the server-side handler for each sensitive +event rather than trusting a LiveView that was mounted while sudo mode was +active. After reauthentication, Kodo returns the user to the intended provider +action without requiring them to rediscover the affected integration. + +An OAuth or device flow created while sudo mode was active may complete after +that mode expires only while the same user-owned attempt remains active and its +generation checks succeed. Cancelling such an attempt requires authenticated +ownership but not renewed sudo mode. Viewing status, automatic credential +validation, and ordinary model requests do not require sudo mode. An explicit +same-model execution-route change uses a clear billing confirmation instead of +sudo mode so it does not interrupt an active session; the existing turn-boundary +and exact-model compatibility rules still apply. The page provides distinct actions for OpenAI API, ChatGPT Subscription, Anthropic, and OpenRouter. It explains the billing identity before connection, @@ -593,8 +606,16 @@ Kodo applies the following boundaries: * Metrics use bounded provider, authentication-type, operation, and outcome labels; user, account, integration, and token identifiers stay out of metric labels. -* Connection changes and terminal refresh failures produce security-relevant - audit logs without credential material. +* API-key submission and replacement, authorization start and completion, + cancellation, disconnection, validation outcomes, execution-route changes, + refresh outcomes, and required reauthorization produce security-relevant + audit logs without credential material. Audit records may contain internal + actor and integration identifiers for investigation, but those identifiers + never become metric labels. +* Phoenix parameter filtering covers every provider secret field before the + first credential form ships. Audit data excludes API keys, access and refresh + tokens, device and authorization codes, encrypted payloads, raw provider + responses, and secret-bearing URLs. * Encryption-key rotation re-encrypts credentials in bounded batches and keeps the previous decryption key only for the documented migration window. * Disconnecting instructs the user how to revoke the credential at the From 71cf5be4e319bab86bff842d84b893c5f73539b4 Mon Sep 17 00:00:00 2001 From: "Darwin D. Wu" Date: Fri, 28 Aug 2026 00:37:50 -0700 Subject: [PATCH 17/29] docs(rfd): reject credential-bearing redirects --- rfd/0005/IMPLEMENTATION.org | 11 ++++++++++- rfd/0005/README.adoc | 28 ++++++++++++++++++++++------ 2 files changed, 32 insertions(+), 7 deletions(-) diff --git a/rfd/0005/IMPLEMENTATION.org b/rfd/0005/IMPLEMENTATION.org index 9b95698..96f57dc 100644 --- a/rfd/0005/IMPLEMENTATION.org +++ b/rfd/0005/IMPLEMENTATION.org @@ -75,6 +75,10 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations immediately before every generation call, and normalize Req/ReqLLM failures into bounded domain errors before persistence, inspection, logging, telemetry, or browser display. +- [ ] Centralize exact HTTPS provider-origin allowlists and disable automatic + redirects for every credential-bearing validation, inference, authorization, + polling, exchange, and refresh request; classify every ~3xx~, including + same-origin redirects, as provider unavailable without a follow-up request. - [ ] Build a shared responsive settings shell with Account and Integrations navigation, a framed desktop/tablet workspace, compact persistent rail, deep-linkable detail pane, clear return action, accessible narrow-screen @@ -156,6 +160,9 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations - [ ] Implement device-code creation, interval-aware asynchronous polling, cancellation, expiry, terminal errors, and authorization-code exchange using Req rather than embedded browser scripts. +- [ ] Render the fixed allowlisted HTTPS verification destination and one-time + code separately, reject a provider-returned verification URL that does not + match the exact allowlist, and never automatically navigate to that URL. - [ ] Trigger claims only from attempt creation or LiveView mount/remount, use one conditional update against the exact attempt row, and run a successful claim as a finite task under ~Kodo.ControlPlaneTaskSupervisor~ that survives @@ -204,7 +211,9 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations during refresh, permanently lost rotated-token responses, refresh restart recovery, revocation, exact-model route changes, incompatible route rejection, in-flight route freezing, normalized cross-route replay, no - automatic route fallback, and secret redaction. + automatic route fallback, same-origin and cross-origin redirect rejection, + malicious verification URLs, absence of a credential-bearing follow-up + request, and secret redaction. - [ ] Run the Phase 0 live compatibility suite against the production path and do not declare ChatGPT/Codex implementation complete while a required capability fails; classify later contract failures as regressions and never diff --git a/rfd/0005/README.adoc b/rfd/0005/README.adoc index 9d27993..dfc8aed 100644 --- a/rfd/0005/README.adoc +++ b/rfd/0005/README.adoc @@ -133,6 +133,15 @@ the initial UI does not accept arbitrary base URLs or HTTP headers. This avoids turning provider configuration into a server-side request forgery or credential-exfiltration path. +Every credential-bearing provider request uses a fixed HTTPS origin and +disables automatic redirect following. This includes validation, inference, +device authorization and polling, authorization-code exchange, and token +refresh. Kodo treats every `3xx` response, including a same-origin redirect, as +provider unavailable without issuing a second request. A legitimate provider +endpoint migration therefore requires an explicit code and allowlist update +rather than risking an API key, bearer token, device code, or authorization +code being forwarded to an unexpected destination. + Validation is non-destructive and provider-specific. A transient validation failure does not expose or invalidate the credential. Kodo records a bounded status and safe error code, while the detailed provider error is redacted @@ -148,11 +157,16 @@ by OpenAI or Anthropic. === ChatGPT subscription ChatGPT subscription support uses OpenAI Codex device authorization. The user -starts the flow in Kodo, receives an OpenAI verification URL and one-time code, -and approves the request in a browser. Kodo polls at OpenAI's instructed -interval, exchanges the resulting authorization code and PKCE verifier, and -stores the returned access token, refresh token, account identifier, and expiry -as encrypted credentials. +starts the flow in Kodo, receives an OpenAI verification destination and +one-time code, and approves the request in a browser. The destination rendered +by Kodo is a fixed HTTPS URL from the reviewed provider contract. Kodo validates +that a provider-returned verification URL matches the exact allowlist, but does +not automatically navigate to it or render an untrusted destination. It shows +the fixed destination and code separately so a returned URL cannot become a +phishing or code-exfiltration path. Kodo polls at OpenAI's instructed interval, +exchanges the resulting authorization code and PKCE verifier, and stores the +returned access token, refresh token, account identifier, and expiry as +encrypted credentials. Authorization attempts are user-scoped, single-use, and expire after the provider deadline. Polling continues only while an attempt is active, respects @@ -600,7 +614,9 @@ Kodo applies the following boundaries: * Provider secrets remain in the Phoenix control plane and are never sent to runners, browsers after submission, or hosted runner environments. -* Provider endpoints and authorization endpoints are allowlisted in code. +* Provider endpoints and authorization endpoints are exact HTTPS allowlists in + code. Credential-bearing requests never follow redirects, and device + verification UI never navigates automatically to a provider-returned URL. * Logs and exceptions redact API keys, bearer tokens, refresh tokens, device codes, authorization codes, and encrypted payloads. * Metrics use bounded provider, authentication-type, operation, and outcome From c9a4f312a7f381cdf83f3a1f3b2a630b6ebdc640 Mon Sep 17 00:00:00 2001 From: "Darwin D. Wu" Date: Fri, 28 Aug 2026 00:40:31 -0700 Subject: [PATCH 18/29] docs(rfd): define encryption key rotation rollout --- rfd/0005/IMPLEMENTATION.org | 21 +++++++++++++++------ rfd/0005/README.adoc | 28 +++++++++++++++++++++++++--- 2 files changed, 40 insertions(+), 9 deletions(-) diff --git a/rfd/0005/IMPLEMENTATION.org b/rfd/0005/IMPLEMENTATION.org index 96f57dc..dbf48d8 100644 --- a/rfd/0005/IMPLEMENTATION.org +++ b/rfd/0005/IMPLEMENTATION.org @@ -29,7 +29,9 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations - [ ] Generate migrations for one durable integration per user/provider pair, separate connection and validation states, non-secret status metadata, - credential generation, and encryption-key version. + credential generation, and encryption-key version. Treat this as a fresh + encrypted schema with no plaintext, environment-key, or legacy integration + import path. - [ ] Add authenticated encryption with unique nonces, versioned keys, and AEAD associated data binding integration ID, user ID, provider, authentication type, and payload format version; keep provider account and workspace @@ -228,13 +230,20 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations cancellation, disconnection, validation outcomes, execution-route changes, refresh outcomes, and reauthorization; permit internal actor and integration IDs only in audit records, never metric labels. -- [ ] Implement and test bounded-batch encryption-key rotation with a documented - previous-key migration window and rollback procedure. +- [ ] Implement and test staged encryption-key rotation: distribute the new key + as non-current, verify every replica accepts both keys, make it current for + writes, and re-encrypt old rows in bounded batches using credential-generation + and old-key-version CAS checks. +- [ ] Retain the old key until no payload uses it and the application rollback + and backup-retention windows close; test concurrent replacement and + disconnection, interrupted-batch resume, mixed-version reads, rollback while + both keys remain configured, and refusal to start with an incomplete ring. - [ ] Verify provider credentials never reach runner registration, runner WebSockets, tool subprocesses, session events, browser responses, exception inspection, or telemetry metadata. -- [ ] Document encryption-key provisioning, backup handling, credential and - provider revocation, key rotation, device-flow troubleshooting, refresh - failure alerts, and rollback from each rollout phase. +- [ ] Document encryption-key provisioning, separately protected key backups, + database-backup retention, credential and provider revocation, staged key + rotation, device-flow troubleshooting, refresh failure alerts, and rollback + from each rollout phase without plaintext or provider-environment fallback. - [ ] Run ~mix precommit~ and the opt-in provider compatibility checks before declaring the RFD implemented. diff --git a/rfd/0005/README.adoc b/rfd/0005/README.adoc index dfc8aed..d075439 100644 --- a/rfd/0005/README.adoc +++ b/rfd/0005/README.adoc @@ -271,7 +271,7 @@ cross-user credential or billing path. Kodo requires a deployment-level credential-encryption key ring, supplied independently of all LLM providers, and records a key version so ciphertext can be rotated. Every production replica validates the complete configured current -and previous key ring before becoming ready to serve integration operations. +and non-current key ring before becoming ready to serve integration operations. Production refuses to persist or decrypt integrations when a required key is absent or malformed. An AEAD association or authentication failure fails closed with a bounded corruption error and never falls back or sends the credential to @@ -280,6 +280,25 @@ Database encryption does not remove the need to protect the control-plane process; it protects database copies, backups, and read-only database access from directly revealing provider credentials. +Kodo is not yet live, so Phase 1 creates a fresh encrypted schema and does not +import plaintext credentials, legacy provider environment variables, or an +earlier integration format. Post-launch key rotation is staged. Operators first +distribute a new non-current key and wait until every replica reports readiness +with both keys. They then make the new key current for writes and re-encrypt old +rows in bounded batches. Each rewrite compares the credential generation and +old key version it read, so replacement or disconnection wins and the batch +retries or skips the stale row without changing its logical credential +generation. + +The old key remains available for decryption through the documented application +rollback and backup-retention window. Rollback restores compatible application +behavior while both keys are still configured; it never removes key material +first, downgrades ciphertext to plaintext, or enables a provider-environment +fallback. Operators remove an old key only after the database has no payloads +at that version, all replicas use the new ring, the rollback window has closed, +and backups that require the old key have expired or remain paired with a +separately protected copy of that key. + The credential payload is opaque to Ecto query code and never appears in session events, model mappings, changeset errors, exception inspection, telemetry metadata, or client responses. Disconnecting an integration removes @@ -632,14 +651,17 @@ Kodo applies the following boundaries: first credential form ships. Audit data excludes API keys, access and refresh tokens, device and authorization codes, encrypted payloads, raw provider responses, and secret-bearing URLs. -* Encryption-key rotation re-encrypts credentials in bounded batches and keeps - the previous decryption key only for the documented migration window. +* Encryption-key rotation uses a distribute, activate, CAS-safe re-encrypt, and + retire sequence; the old key remains through application rollback and backup + retention. * Disconnecting instructs the user how to revoke the credential at the provider when Kodo cannot revoke it directly. Operational documentation covers initial encryption-key provisioning, backup handling, rotation, provider revocation, device-flow failures, refresh failure rates, and safe rollback across integration schema and encryption-key changes. +Encryption keys are backed up and access-controlled separately from database +backups. == Question to resolve From d677923f743c1c1d4e7c8c3754ce2ffde3423a23 Mon Sep 17 00:00:00 2001 From: "Darwin D. Wu" Date: Fri, 28 Aug 2026 00:46:02 -0700 Subject: [PATCH 19/29] docs(rfd): front-load credential security phase --- rfd/0005/IMPLEMENTATION.org | 56 +++++++++++++++++++++---------------- rfd/0005/README.adoc | 29 +++++++++++++------ 2 files changed, 52 insertions(+), 33 deletions(-) diff --git a/rfd/0005/IMPLEMENTATION.org b/rfd/0005/IMPLEMENTATION.org index dbf48d8..7093c9d 100644 --- a/rfd/0005/IMPLEMENTATION.org +++ b/rfd/0005/IMPLEMENTATION.org @@ -25,7 +25,7 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations once its implementation and required compatibility suite pass; surface later contract regressions as actionable integration-unavailable errors. -* Phase 1: Build encrypted integration storage and OpenAI API keys +* Phase 1A: Establish the credential security foundation - [ ] Generate migrations for one durable integration per user/provider pair, separate connection and validation states, non-secret status metadata, @@ -81,6 +81,27 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations redirects for every credential-bearing validation, inference, authorization, polling, exchange, and refresh request; classify every ~3xx~, including same-origin redirects, as provider unavailable without a follow-up request. +- [ ] Add every provider secret field to Phoenix parameter filtering and test + that filtered request logs, changeset errors, exception inspection, and + LiveView events cannot expose credential material. +- [ ] Add the credential-free security audit framework and API-key lifecycle + events; permit internal actor and integration IDs only in audit records, + never metric labels. +- [ ] Verify provider credentials never reach runner registration, runner + WebSockets, tool subprocesses, session events, browser responses, exception + inspection, or telemetry metadata. +- [ ] Test encryption round trips and redaction, ownership isolation, + ciphertext swaps between users and providers, tampering, wrong and missing + keys, unknown payload versions, nonce uniqueness, lifecycle transitions, + generation fencing, provider mismatches, owned text and object generation, + evaluation ownership, safe provider errors, same-origin and cross-origin + redirect rejection, no credential-bearing follow-up request, and absence of + an environment-key fallback. +- [ ] Do not expose a credential form or permit a production provider request + until every Phase 1A security boundary and test is complete. + +* Phase 1B: Add OpenAI API integration and settings + - [ ] Build a shared responsive settings shell with Account and Integrations navigation, a framed desktop/tablet workspace, compact persistent rail, deep-linkable detail pane, clear return action, accessible narrow-screen @@ -102,9 +123,6 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations after sudo expiry, require authenticated ownership but not fresh sudo for cancellation, and keep status viewing, automatic validation, and ordinary inference outside sudo mode. -- [ ] Add every provider secret field to Phoenix parameter filtering before - shipping its form, and test that filtered request logs, changeset errors, - exception inspection, and LiveView events cannot expose credential material. - [ ] Add LiveView selector tests for the settings shell, section navigation, provider cards, contextual forms, and device-flow states, plus representative desktop, tablet, and narrow-mobile browser checks for the sketched layouts. @@ -117,18 +135,12 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations confirmed presets affect only defaults and new sessions, and disconnecting a required integration derives an unavailable mapping without rewriting or silently rerouting existing sessions. -- [ ] Test encryption round trips and redaction, ownership isolation, concurrent - users with different OpenAI keys, key replacement during a session, missing - integration behavior, ciphertext swaps between users and providers, - tampering, wrong and missing keys, unknown payload versions, nonce - uniqueness, lifecycle transitions, generation-fenced asynchronous updates, - catalog aliases, inline model specs, OpenRouter underlying vendors, malformed - models, credential/provider mismatches, billing provenance, text and object - generation ownership, evaluation ownership, safe provider errors, secret - redaction, the absence of environment-key fallback, cross-user sensitive - actions, expired sudo mode, stale LiveView sockets, post-sudo authorization - completion, ownership-checked cancellation, and explicit route-change billing - confirmation. +- [ ] Test concurrent users with different OpenAI keys, key replacement during + a session, missing integration behavior, catalog aliases, inline model specs, + OpenRouter underlying vendors, malformed models, billing provenance, + cross-user sensitive actions, expired sudo mode, stale LiveView sockets, + post-sudo authorization completion, ownership-checked cancellation, and + explicit route-change billing confirmation. * Phase 2: Add Anthropic and OpenRouter API keys @@ -225,11 +237,10 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations - [ ] Add onboarding and session/model-settings feedback that links users to every missing provider integration required by their selected role mapping. -- [ ] Add bounded-cardinality telemetry and credential-free security audit logs - for API-key submission and replacement, authorization start/completion and - cancellation, disconnection, validation outcomes, execution-route changes, - refresh outcomes, and reauthorization; permit internal actor and integration - IDs only in audit records, never metric labels. +- [ ] Add bounded-cardinality telemetry for integration operations and complete + credential-free audit coverage for authorization start/completion and + cancellation, validation outcomes, execution-route changes, refresh outcomes, + and reauthorization using the Phase 1A audit framework. - [ ] Implement and test staged encryption-key rotation: distribute the new key as non-current, verify every replica accepts both keys, make it current for writes, and re-encrypt old rows in bounded batches using credential-generation @@ -238,9 +249,6 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations and backup-retention windows close; test concurrent replacement and disconnection, interrupted-batch resume, mixed-version reads, rollback while both keys remain configured, and refusal to start with an incomplete ring. -- [ ] Verify provider credentials never reach runner registration, runner - WebSockets, tool subprocesses, session events, browser responses, exception - inspection, or telemetry metadata. - [ ] Document encryption-key provisioning, separately protected key backups, database-backup retention, credential and provider revocation, staged key rotation, device-flow troubleshooting, refresh failure alerts, and rollback diff --git a/rfd/0005/README.adoc b/rfd/0005/README.adoc index d075439..5411d17 100644 --- a/rfd/0005/README.adoc +++ b/rfd/0005/README.adoc @@ -722,15 +722,25 @@ capabilities, token refresh, and restart recovery. Phase 0 defines the production contract and regression diagnostics; it is not a gate for omitting the required ChatGPT/Codex integration. -=== Phase 1: Build encrypted integration storage +=== Phase 1A: Establish the credential security foundation Add the user-owned integration schema, authenticated encryption and key -versioning, scoped context operations, safe inspection, explicit live-test -credentials, and fail-closed request-time resolution. Add the shared responsive -settings shell, improve the existing Account presentation without changing its -behavior, and ship the authenticated integration page with OpenAI Platform -API-key connection first. This establishes the full path from UI through -request-local ReqLLM options without a provider-key environment fallback. +versioning, scoped context operations, parameter filtering, safe inspection, +fixed outbound endpoint policy, credential-free audit framework, typed +request-local credential boundary, and fail-closed resolution. Security tests +must prove ownership isolation and prevent secret egress to logs, browsers, +runners, telemetry, or redirected requests. This phase is deployable without a +credential UI; Kodo must not accept a credential or send a production provider +request until the complete foundation passes. + +=== Phase 1B: Add OpenAI API integration and settings + +Add the shared responsive settings shell, improve the existing Account +presentation without changing its behavior, and ship the authenticated +integration page with OpenAI Platform API-key connection first. Wire model +resolution, explicit live-test credentials, validation, lifecycle audit events, +and request-local ReqLLM options through the Phase 1A boundary without a +provider-key environment fallback. === Phase 2: Add Anthropic and OpenRouter @@ -749,8 +759,9 @@ that leave affected integrations unavailable while Kodo fixes the contract. === Phase 4: Harden operations Complete operational hardening, exercise multi-node refresh and key rotation, -and publish operator guidance. The encryption key remains the only -integration-related boot secret. +and publish operator guidance. All Phase 4 requirements are production-launch +requirements even though they follow provider implementation. The encryption +key remains the only integration-related boot secret. == Implementation From 4f848d2fb8c513ec72c24a75da5664185d7f45d2 Mon Sep 17 00:00:00 2001 From: "Darwin D. Wu" Date: Fri, 28 Aug 2026 00:49:27 -0700 Subject: [PATCH 20/29] docs(rfd): clarify request-time environment fallback --- rfd/0005/README.adoc | 27 +++++++++++++++------------ 1 file changed, 15 insertions(+), 12 deletions(-) diff --git a/rfd/0005/README.adoc b/rfd/0005/README.adoc index 5411d17..4f2351c 100644 --- a/rfd/0005/README.adoc +++ b/rfd/0005/README.adoc @@ -9,9 +9,9 @@ == Goal Let each Kodo user connect the LLM providers used by their agent sessions -without requiring an operator to supply an OpenAI API key when the control -plane boots. The first provider set includes OpenAI Platform API keys, -Anthropic API keys, OpenRouter API keys, and OpenAI Codex access through a +without requiring an operator to configure a deployment-wide OpenAI API key for +successful model requests. The first provider set includes OpenAI Platform API +keys, Anthropic API keys, OpenRouter API keys, and OpenAI Codex access through a linked ChatGPT subscription. Provider credentials remain in the control plane. Runners continue to receive @@ -22,9 +22,11 @@ access tokens, or refresh tokens. Kodo delegates provider authentication to ReqLLM. With no request-local credential, ReqLLM reads `OPENAI_API_KEY` from the control-plane environment. -This creates one deployment-wide provider identity, makes credential changes an -operational restart concern, and prevents users from choosing separately billed -provider accounts. +This lookup occurs when a model request is made; the key is not required for +the control plane to boot, but model requests fail without it. The fallback +creates one deployment-wide provider identity, makes credential changes a +deployment-configuration concern, and prevents users from choosing separately +billed provider accounts. The existing execution boundary is otherwise suitable. Agent inference runs in the Phoenix control plane, sessions have a durable owning user, and model role @@ -620,12 +622,13 @@ Ordinary tests use fake adapters and fake typed credentials without external provider access. Neither path adds environment lookup or owner inference to the production adapter. -At the end of this RFD, no LLM API key or OAuth token is required at boot. A -credential-encryption key remains required because encrypted database secrets -need a root of trust outside the database. Installations with no connected -provider can boot and serve account and integration pages, but cannot start a -model turn until the user connects every provider required by the selected -role mapping. +At the end of this RFD, no LLM API key or OAuth token is read from deployment +configuration at boot or request time. The credential-encryption key ring is a +production-readiness requirement because encrypted database secrets need a +root of trust outside the database. Installations with a valid key ring but no +connected provider can boot and serve account and integration pages, but cannot +start a model turn until the user connects every provider required by the +selected role mapping. == Security and operations From 55e7a004e4be1c56c1d8b2e001beebc98f7dc6ca Mon Sep 17 00:00:00 2001 From: "Darwin D. Wu" Date: Fri, 28 Aug 2026 00:51:41 -0700 Subject: [PATCH 21/29] docs(rfd): clarify in-flight disconnect behavior --- rfd/0005/IMPLEMENTATION.org | 11 ++++++++++- rfd/0005/README.adoc | 18 ++++++++++++++++-- 2 files changed, 26 insertions(+), 3 deletions(-) diff --git a/rfd/0005/IMPLEMENTATION.org b/rfd/0005/IMPLEMENTATION.org index 7093c9d..1939ce2 100644 --- a/rfd/0005/IMPLEMENTATION.org +++ b/rfd/0005/IMPLEMENTATION.org @@ -77,6 +77,10 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations immediately before every generation call, and normalize Req/ReqLLM failures into bounded domain errors before persistence, inspection, logging, telemetry, or browser display. +- [ ] Snapshot credential generation and billing provenance when each external + provider request starts; allow that request to complete after replacement or + disconnection, but require every later call and tool continuation to resolve + again and fail closed when the integration is unusable. - [ ] Centralize exact HTTPS provider-origin allowlists and disable automatic redirects for every credential-bearing validation, inference, authorization, polling, exchange, and refresh request; classify every ~3xx~, including @@ -119,6 +123,9 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations the intended action across reauthentication; and implement the OpenAI Platform API-key provider card, contextual form, connection status, loading state, and accessible feedback. +- [ ] State in disconnect confirmation that future Kodo requests stop while an + already-sent provider request and its billing may continue, and link to + provider-side revocation when Kodo cannot revoke the credential directly. - [ ] Permit an active generation-fenced authorization attempt to complete after sudo expiry, require authenticated ownership but not fresh sudo for cancellation, and keep status viewing, automatic validation, and ordinary @@ -140,7 +147,9 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations OpenRouter underlying vendors, malformed models, billing provenance, cross-user sensitive actions, expired sudo mode, stale LiveView sockets, post-sudo authorization completion, ownership-checked cancellation, and - explicit route-change billing confirmation. + explicit route-change billing confirmation. Cover request start followed by + disconnect, completion with original provenance, failure of the next request + or tool continuation, and rejection of late refresh or validation updates. * Phase 2: Add Anthropic and OpenRouter API keys diff --git a/rfd/0005/README.adoc b/rfd/0005/README.adoc index 4f2351c..1c3ed7c 100644 --- a/rfd/0005/README.adoc +++ b/rfd/0005/README.adoc @@ -78,7 +78,9 @@ in which case the session owner must have every required integration. Kodo preflights every role integration before accepting a turn and resolves the current credential again immediately before each provider request. Key replacement, token refresh, disconnection, or revocation therefore takes effect -without rewriting session history. +for requests that have not started, without rewriting session history. It does +not retroactively cancel a provider request to which Kodo has already sent a +credential. The first release permits an explicit execution-route change between `openai` Platform billing and `openai_codex` ChatGPT subscription billing while @@ -349,6 +351,16 @@ resolved boundary: canonical provider, authentication type, and billing path material. This provenance remains stable even if the integration or mapping is later replaced. +Once a provider request has resolved its credential and begun the external HTTP +operation, replacement or disconnection does not alter that request. Its +response may complete and be persisted with the billing provenance captured at +start because Kodo cannot reliably retract transmitted credentials, stop +provider computation, or prevent already-incurred billing. Each later provider +request, including a tool-call continuation or another role call in the same +turn, resolves again and fails closed if the integration is then unusable. A +late validation or refresh result remains generation-fenced and cannot restore +or overwrite a disconnected integration. + Kodo classifies Req and ReqLLM failures into bounded domain errors before they leave the LLM boundary. Session events, exception inspection, logs, telemetry, and browser responses may contain only the provider, operation, bounded outcome @@ -658,7 +670,9 @@ Kodo applies the following boundaries: retire sequence; the old key remains through application rollback and backup retention. * Disconnecting instructs the user how to revoke the credential at the - provider when Kodo cannot revoke it directly. + provider when Kodo cannot revoke it directly. The confirmation states that + disconnect prevents future Kodo requests but cannot guarantee cancellation + of a request already sent to the provider or its associated billing. Operational documentation covers initial encryption-key provisioning, backup handling, rotation, provider revocation, device-flow failures, refresh failure From 5e9aa8475f4bb308fca286f76a1fa1dd3857b257 Mon Sep 17 00:00:00 2001 From: "Darwin D. Wu" Date: Fri, 28 Aug 2026 01:06:53 -0700 Subject: [PATCH 22/29] docs(rfd): avoid credential fingerprints --- rfd/0005/IMPLEMENTATION.org | 6 +++++- rfd/0005/README.adoc | 12 ++++++++++-- 2 files changed, 15 insertions(+), 3 deletions(-) diff --git a/rfd/0005/IMPLEMENTATION.org b/rfd/0005/IMPLEMENTATION.org index 1939ce2..5bbe5c9 100644 --- a/rfd/0005/IMPLEMENTATION.org +++ b/rfd/0005/IMPLEMENTATION.org @@ -40,6 +40,9 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations Kodo credential-encryption key ring from all LLM provider credentials, validates every current and previous key before replica readiness, and fails closed on missing keys or payload authentication failures. +- [ ] Do not derive or persist API-key hashes, suffixes, or fingerprints and + never fingerprint OAuth tokens; permit only provider-supplied labels that are + explicitly classified as non-secret by the reviewed provider contract. - [ ] Implement scope-first integration context operations for listing, connecting, replacing, validating, and disconnecting credentials without accepting browser-supplied ownership; disconnect clears the encrypted payload @@ -100,7 +103,8 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations generation fencing, provider mismatches, owned text and object generation, evaluation ownership, safe provider errors, same-origin and cross-origin redirect rejection, no credential-bearing follow-up request, and absence of - an environment-key fallback. + an environment-key fallback. Assert credential-derived identifiers are absent + from persistence, UI, logs, telemetry, and audit records. - [ ] Do not expose a credential form or permit a production provider request until every Phase 1A security boundary and test is complete. diff --git a/rfd/0005/README.adoc b/rfd/0005/README.adoc index 1c3ed7c..6780667 100644 --- a/rfd/0005/README.adoc +++ b/rfd/0005/README.adoc @@ -150,8 +150,11 @@ Validation is non-destructive and provider-specific. A transient validation failure does not expose or invalidate the credential. Kodo records a bounded status and safe error code, while the detailed provider error is redacted before logging or display. A saved credential is never rendered back to the -browser; the UI shows only provider, status, validation time, and a non-secret -fingerprint where useful. +browser. The UI shows provider, billing identity, status, connection and +validation times, and an explicitly non-secret provider-supplied key or account +label where that provider contract supplies one. Kodo does not derive, persist, +or display an API-key hash, suffix, or fingerprint, and never fingerprints an +OAuth access or refresh token. OpenRouter is a provider in its own right rather than a transparent fallback for direct provider integrations. Models routed through OpenRouter use the @@ -666,6 +669,11 @@ Kodo applies the following boundaries: first credential form ships. Audit data excludes API keys, access and refresh tokens, device and authorization codes, encrypted payloads, raw provider responses, and secret-bearing URLs. +* Credential-derived hashes, suffixes, and fingerprints do not appear in the + database, UI, logs, telemetry, or audit records. If a future operational need + requires API-key correlation, a separate proposal must use a versioned, + domain-separated keyed HMAC restricted to internal access; an unkeyed hash or + OAuth-token fingerprint is never acceptable. * Encryption-key rotation uses a distribute, activate, CAS-safe re-encrypt, and retire sequence; the old key remains through application rollback and backup retention. From d720b2caa6f7664569963036f46293032789654d Mon Sep 17 00:00:00 2001 From: "Darwin D. Wu" Date: Fri, 28 Aug 2026 01:12:56 -0700 Subject: [PATCH 23/29] docs(rfd): cascade integrations on user deletion --- rfd/0005/IMPLEMENTATION.org | 17 +++++++++++++---- rfd/0005/README.adoc | 14 +++++++++++++- 2 files changed, 26 insertions(+), 5 deletions(-) diff --git a/rfd/0005/IMPLEMENTATION.org b/rfd/0005/IMPLEMENTATION.org index 5bbe5c9..20fa401 100644 --- a/rfd/0005/IMPLEMENTATION.org +++ b/rfd/0005/IMPLEMENTATION.org @@ -32,6 +32,9 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations credential generation, and encryption-key version. Treat this as a fresh encrypted schema with no plaintext, environment-key, or legacy integration import path. +- [ ] Enforce ~ON DELETE CASCADE~ from users to integrations and from each + integration to authorization attempts and claim state; account deletion must + not depend on a successful provider call. - [ ] Add authenticated encryption with unique nonces, versioned keys, and AEAD associated data binding integration ID, user ID, provider, authentication type, and payload format version; keep provider account and workspace @@ -104,7 +107,9 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations evaluation ownership, safe provider errors, same-origin and cross-origin redirect rejection, no credential-bearing follow-up request, and absence of an environment-key fallback. Assert credential-derived identifiers are absent - from persistence, UI, logs, telemetry, and audit records. + from persistence, UI, logs, telemetry, and audit records. Delete a user with + API-key integrations and racing validation work, then assert all owned rows + are gone and the late task performs no state write or provider request. - [ ] Do not expose a credential form or permit a production provider request until every Phase 1A security boundary and test is complete. @@ -204,6 +209,9 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations - [ ] Store access tokens, refresh tokens, account identifiers, expiries, and credential generations in the encrypted integration payload without logging device or token material. +- [ ] Test user deletion with OAuth credentials, an active authorization + attempt, a held claim, and in-flight refresh; all rows cascade and every late + completion becomes a no-op. - [ ] Run just-in-time refreshes as bounded tasks under ~Kodo.ControlPlaneTaskSupervisor~; coordinate callers directly through the database rather than adding a refresh coordinator ~GenServer~, long-lived @@ -263,8 +271,9 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations disconnection, interrupted-batch resume, mixed-version reads, rollback while both keys remain configured, and refusal to start with an incomplete ring. - [ ] Document encryption-key provisioning, separately protected key backups, - database-backup retention, credential and provider revocation, staged key - rotation, device-flow troubleshooting, refresh failure alerts, and rollback - from each rollout phase without plaintext or provider-environment fallback. + database-backup and write-ahead-log retention, account deletion limits, + credential and provider revocation, staged key rotation, device-flow + troubleshooting, refresh failure alerts, and rollback from each rollout phase + without plaintext or provider-environment fallback. - [ ] Run ~mix precommit~ and the opt-in provider compatibility checks before declaring the RFD implemented. diff --git a/rfd/0005/README.adoc b/rfd/0005/README.adoc index 6780667..9e36e6c 100644 --- a/rfd/0005/README.adoc +++ b/rfd/0005/README.adoc @@ -250,6 +250,15 @@ provider pair. Context functions take the authenticated scope as their first argument and apply the owning user in the query. Changesets never cast a `user_id` supplied by a browser. +Deleting a Kodo user cascades in the database to all of that user's integration +rows and from each integration to its device-authorization attempts and claim +state. Provider availability or revocation success never blocks account +deletion. Any validation, polling, refresh, or rotation task that races with +deletion observes the missing ownership-checked row and terminates without +persisting state or issuing a later provider request. The deletion flow advises +the user to revoke credentials at providers where Kodo cannot do so, but does +not promise provider-side revocation. + Connection and validation are separate state axes. Connection is one of `disconnected`, `connected`, or `reauthorization_required`. Validation is one of `unverified`, `valid`, `invalid`, or `unavailable`. Connecting, replacing, @@ -686,7 +695,10 @@ Operational documentation covers initial encryption-key provisioning, backup handling, rotation, provider revocation, device-flow failures, refresh failure rates, and safe rollback across integration schema and encryption-key changes. Encryption keys are backed up and access-controlled separately from database -backups. +backups. User deletion removes live database payloads but cannot selectively +erase ciphertext already present in retained database backups or write-ahead +logs; those copies remain governed by the documented backup expiry and key +retention policy. == Question to resolve From d6539a2da69e6db74e95f2738ac41bf173ee577d Mon Sep 17 00:00:00 2001 From: "Darwin D. Wu" Date: Fri, 28 Aug 2026 01:15:53 -0700 Subject: [PATCH 24/29] docs(rfd): distinguish test opt-ins from feature flags --- rfd/0005/IMPLEMENTATION.org | 2 ++ rfd/0005/README.adoc | 5 +++++ 2 files changed, 7 insertions(+) diff --git a/rfd/0005/IMPLEMENTATION.org b/rfd/0005/IMPLEMENTATION.org index 20fa401..0f6df6c 100644 --- a/rfd/0005/IMPLEMENTATION.org +++ b/rfd/0005/IMPLEMENTATION.org @@ -24,6 +24,8 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations - [ ] Ship ChatGPT Subscription as a Beta without a deployment feature flag once its implementation and required compatibility suite pass; surface later contract regressions as actionable integration-unavailable errors. +- [ ] Treat opt-in live-provider and compatibility checks only as controls for + externally billed tests, not as deployment or product feature flags. * Phase 1A: Establish the credential security foundation diff --git a/rfd/0005/README.adoc b/rfd/0005/README.adoc index 9e36e6c..63f392c 100644 --- a/rfd/0005/README.adoc +++ b/rfd/0005/README.adoc @@ -226,6 +226,11 @@ retry through another client identity or fall back to Platform billing. Ordinary transient, quota, and authorization errors remain scoped to the affected integration. +No provider in this RFD is gated by a deployment feature flag once its required +phase acceptance checks pass. References to opt-in live-provider or +compatibility checks describe tests that contact and may bill an external +provider; they are test controls, not product availability controls. + Anthropic subscription OAuth is outside this RFD. Anthropic permits API keys and supported cloud-provider credentials for third-party applications but currently prohibits third parties from collecting or intermediating Claude From 0d83c7e71670a4e58cf7ae37a1db87e26837bc74 Mon Sep 17 00:00:00 2001 From: "Darwin D. Wu" Date: Fri, 28 Aug 2026 01:22:54 -0700 Subject: [PATCH 25/29] docs(rfd): bound plaintext credential lifetime --- rfd/0005/IMPLEMENTATION.org | 19 ++++++++++++++++--- rfd/0005/README.adoc | 35 +++++++++++++++++++++++++++++++---- 2 files changed, 47 insertions(+), 7 deletions(-) diff --git a/rfd/0005/IMPLEMENTATION.org b/rfd/0005/IMPLEMENTATION.org index 0f6df6c..92afe56 100644 --- a/rfd/0005/IMPLEMENTATION.org +++ b/rfd/0005/IMPLEMENTATION.org @@ -85,6 +85,16 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations immediately before every generation call, and normalize Req/ReqLLM failures into bounded domain errors before persistence, inspection, logging, telemetry, or browser display. +- [ ] Batch preflight metadata through indexed user/provider queries without + decryption; immediately before each credential-bearing external operation, + perform the bounded current integration or authorization-attempt lookup and + local decryption without holding a database transaction or lock across the + request. +- [ ] Keep plaintext credentials out of GenServers, ETS, persistent terms, + session state, and unnecessary process messages; discard operation-local + references after use without claiming BEAM memory zeroization. Do not add an + encrypted cache without measured database pressure and a separate consistency + review. - [ ] Snapshot credential generation and billing provenance when each external provider request starts; allow that request to complete after replacement or disconnection, but require every later call and tool continuation to resolve @@ -111,7 +121,9 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations an environment-key fallback. Assert credential-derived identifiers are absent from persistence, UI, logs, telemetry, and audit records. Delete a user with API-key integrations and racing validation work, then assert all owned rows - are gone and the late task performs no state write or provider request. + are gone and the late task performs no state write or provider request. Test + application-level non-retention in process state and messages without + asserting physical memory erasure. - [ ] Do not expose a credential form or permit a production provider request until every Phase 1A security boundary and test is complete. @@ -275,7 +287,8 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations - [ ] Document encryption-key provisioning, separately protected key backups, database-backup and write-ahead-log retention, account deletion limits, credential and provider revocation, staged key rotation, device-flow - troubleshooting, refresh failure alerts, and rollback from each rollout phase - without plaintext or provider-environment fallback. + troubleshooting, refresh failure alerts, BEAM debugging and memory-inspection + restrictions, crash/core dump and swap handling, and rollback from each + rollout phase without plaintext or provider-environment fallback. - [ ] Run ~mix precommit~ and the opt-in provider compatibility checks before declaring the RFD implemented. diff --git a/rfd/0005/README.adoc b/rfd/0005/README.adoc index 63f392c..350425d 100644 --- a/rfd/0005/README.adoc +++ b/rfd/0005/README.adoc @@ -353,6 +353,24 @@ refresh tokens, expiry, generation, and rotation state remain in Kodo. No request uses `System.put_env/2`, `Application.put_env/3`, ReqLLM's global key store, or a shared OAuth file. +Status pages and turn preflight query only non-secret integration metadata and +batch all required role providers into one indexed query. Immediately before +each credential-bearing external operation, Kodo reads the current integration +or authorization-attempt state through its ownership index and decrypts the +required fields in the Phoenix process. An operation is one validation, +inference or streaming call, tool-call continuation, device poll or exchange, +or token refresh. Kodo does not hold a database transaction or row lock during +the external request. + +This adds a small bounded set of indexed reads per external operation and scales +with active provider calls rather than the number of Kodo replicas. Since +provider calls are orders of magnitude slower than the lookup and local AEAD +decryption, Kodo does not introduce a plaintext credential cache in a +`GenServer`, ETS, persistent terms, or session state. A future generation-aware +encrypted cache requires measured database pressure and a separate consistency +review because stale cache entries weaken immediate replacement and +disconnection semantics. + Credential resolution occurs before Kodo persists a model invocation as started. A missing integration, a `disconnected` or `reauthorization_required` connection, or an `invalid` validation returns a @@ -398,9 +416,10 @@ identifier, and expiry in one encrypted payload. Refresh is just-in-time, bounded work rather than a long-lived process per integration. A request that encounters a near-expiry token starts the external HTTP operation under Kodo's existing task supervisor after acquiring the -database lease. The task holds decrypted credentials only for the operation's -duration. A task or node failure leaves durable credential state unchanged and -the expired lease available for takeover. +database lease. The task decrypts immediately before use, retains references +only for the bounded operation, and discards them afterward so they become +eligible for garbage collection. A task or node failure leaves durable +credential state unchanged and the expired lease available for takeover. Callers coordinate directly through the database; Kodo does not add a refresh coordinator `GenServer`, a `DynamicSupervisor` child, or an in-memory timer per @@ -688,6 +707,11 @@ Kodo applies the following boundaries: requires API-key correlation, a separate proposal must use a versioned, domain-separated keyed HMAC restricted to internal access; an unkeyed hash or OAuth-token fingerprint is never acceptable. +* Kodo minimizes plaintext credential references and does not retain them in + long-lived processes, caches, or unnecessary process messages. BEAM immutable + binaries, garbage collection, and HTTP client internals do not provide a + guarantee of immediate memory zeroization, so tests assert application-level + non-retention rather than physical erasure. * Encryption-key rotation uses a distribute, activate, CAS-safe re-encrypt, and retire sequence; the old key remains through application rollback and backup retention. @@ -703,7 +727,10 @@ Encryption keys are backed up and access-controlled separately from database backups. User deletion removes live database payloads but cannot selectively erase ciphertext already present in retained database backups or write-ahead logs; those copies remain governed by the documented backup expiry and key -retention policy. +retention policy. Production guidance restricts BEAM debugging and memory +inspection and protects or disables crash dumps, core dumps, and swap according +to the deployment environment. Guaranteed zeroization would require a separate +native or isolated secret-handling boundary and is outside this RFD. == Question to resolve From 7dde69c66856bd075540d947fcae9a1d96f5e5f0 Mon Sep 17 00:00:00 2001 From: "Darwin D. Wu" Date: Fri, 28 Aug 2026 01:33:24 -0700 Subject: [PATCH 26/29] docs(rfd): define unbilled credential validation --- rfd/0005/IMPLEMENTATION.org | 31 +++++++++++++++++------ rfd/0005/README.adoc | 50 +++++++++++++++++++++---------------- 2 files changed, 52 insertions(+), 29 deletions(-) diff --git a/rfd/0005/IMPLEMENTATION.org b/rfd/0005/IMPLEMENTATION.org index 92afe56..9a11ad4 100644 --- a/rfd/0005/IMPLEMENTATION.org +++ b/rfd/0005/IMPLEMENTATION.org @@ -146,6 +146,14 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations the intended action across reauthentication; and implement the OpenAI Platform API-key provider card, contextual form, connection status, loading state, and accessible feedback. +- [ ] Save OpenAI API keys encrypted as ~unverified~ before asynchronously + probing ~GET /v1/models~ in a bounded generation-fenced task. Mark only a + confirmed authentication rejection invalid; classify network, TLS, timeout, + redirect, rate-limit, provider, and insufficient-permission failures as + validation unavailable. +- [ ] Never run billed inference during automatic validation. Offer a Test model + request only after explicit confirmation identifies the exact provider and + model and warns of possible charges; require ownership but not fresh sudo. - [ ] State in disconnect confirmation that future Kodo requests stop while an already-sent provider request and its billing may continue, and link to provider-side revocation when Kodo cannot revoke the credential directly. @@ -173,25 +181,32 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations explicit route-change billing confirmation. Cover request start followed by disconnect, completion with original provenance, failure of the next request or tool continuation, and rejection of late refresh or validation updates. + Verify save succeeds during outage, stale validation cannot overwrite a newer + generation, permission-limited probes do not mark a key invalid, and no + automatic path sends a billed model request. * Phase 2: Add Anthropic and OpenRouter API keys - [ ] Add Anthropic API-key connection, fixed endpoint configuration, - provider-specific non-destructive validation, and safe error classification. + unbilled asynchronous ~GET /v1/models~ validation, and safe generation-fenced + error classification. - [ ] Add OpenRouter API-key connection, fixed endpoint configuration, - provider-specific non-destructive validation, and explicit OpenRouter billing - identity for routed models. -- [ ] Document whether validation for each API-key provider is unbilled or an - explicit potentially billed Test action, and represent saved-but-unverified - credentials separately from valid, invalid, and validation-unavailable - credentials; do not classify quota, entitlement, or provider outages as - credential invalidity. + unbilled asynchronous ~GET /api/v1/key~ validation, and explicit OpenRouter + billing identity for routed models. +- [ ] Represent saved-but-unverified credentials separately from valid, invalid, + and validation-unavailable credentials. Keep unverified and unavailable + credentials usable, block only confirmed invalid credentials, and treat quota, + billing, entitlement, restricted validation permission, and provider outages + as non-invalid outcomes. - [ ] Extend integration settings and model-selection feedback to distinguish OpenAI API, Anthropic, and OpenRouter providers without displaying saved credentials. - [ ] Test a role mapping that uses different connected providers for primary, search, and review, including absence or revocation of one required integration during a turn. +- [ ] Test each provider probe and outcome class, generation fencing, usable + unverified/unavailable credentials, blocking confirmed-invalid credentials, + and explicit cost confirmation for each optional Test model request. - [ ] Add opt-in live-provider checks for OpenAI API, Anthropic, and OpenRouter using explicit test-user integrations while keeping ordinary tests independent of external provider access. diff --git a/rfd/0005/README.adoc b/rfd/0005/README.adoc index 350425d..3008311 100644 --- a/rfd/0005/README.adoc +++ b/rfd/0005/README.adoc @@ -149,12 +149,35 @@ code being forwarded to an unexpected destination. Validation is non-destructive and provider-specific. A transient validation failure does not expose or invalidate the credential. Kodo records a bounded status and safe error code, while the detailed provider error is redacted -before logging or display. A saved credential is never rendered back to the -browser. The UI shows provider, billing identity, status, connection and -validation times, and an explicitly non-secret provider-supplied key or account -label where that provider contract supplies one. Kodo does not derive, persist, -or display an API-key hash, suffix, or fingerprint, and never fingerprints an -OAuth access or refresh token. +before logging or display. + +Saving or replacing an API key first commits its encrypted payload as +`connected` and `unverified`, then starts a bounded asynchronous validation task +under the control-plane task supervisor. The unbilled probes are OpenAI +`GET /v1/models`, Anthropic `GET /v1/models`, and OpenRouter +`GET /api/v1/key`, using each provider's required authentication headers. A +successful probe marks the integration `valid`. Only a provider-specific, +confirmed authentication rejection marks it `invalid`; Kodo does not treat +every `403` as proof of a bad key when restricted-key permissions may explain +the response. Network and TLS failures, timeouts, redirects, `429`, `5xx`, and +insufficient validation permission produce `unavailable`. Quota, billing, and +model-entitlement failures remain request outcomes. Every result compares the +credential generation captured by the task so it cannot update a replacement +or disconnection. + +Kodo never runs billed inference automatically. A user may explicitly confirm +a Test model request after the UI identifies the exact provider and model and +warns that the request may incur provider charges. Testing requires +authenticated ownership and confirmation but not renewed sudo mode. Saving +remains successful during provider outages, and `unverified` or `unavailable` +credentials remain usable while `invalid` blocks provider requests. + +A saved credential is never rendered back to the browser. The UI shows +provider, billing identity, status, connection and validation times, and an +explicitly non-secret provider-supplied key or account label where that +provider contract supplies one. Kodo does not derive, persist, or display an +API-key hash, suffix, or fingerprint, and never fingerprints an OAuth access or +refresh token. OpenRouter is a provider in its own right rather than a transparent fallback for direct provider integrations. Models routed through OpenRouter use the @@ -732,21 +755,6 @@ inspection and protects or disables crash dumps, core dumps, and swap according to the deployment environment. Guaranteed zeroization would require a separate native or isolated secret-handling boundary and is outside this RFD. -== Question to resolve - -The remaining question does not reopen the settled ownership, encryption, -request-local credential, no-runner-secret, or model-routing boundaries. It -must be answered before the affected providers ship. - -=== Credential validation and possible cost - -Providers do not all expose an equivalent unbilled credential-introspection -endpoint. Before each API-key provider ships, Kodo must identify a reliable -non-billed validation request or decide that validation is an explicit model -request that may incur a small charge. Saving a credential must remain possible -during a provider outage; the UI must distinguish saved, unverified, valid, and -invalid states and disclose any billed Test action before it runs. - == Alternatives considered === Continue deployment environment keys From e8f60ab1a77cf6110d26f077a2487433da2d473f Mon Sep 17 00:00:00 2001 From: "Darwin D. Wu" Date: Sat, 29 Aug 2026 12:47:11 -0700 Subject: [PATCH 27/29] docs(rfd): define refresh outcome precedence --- rfd/0005/IMPLEMENTATION.org | 27 ++++++++++++++++-------- rfd/0005/README.adoc | 41 +++++++++++++++++++++++++------------ 2 files changed, 47 insertions(+), 21 deletions(-) diff --git a/rfd/0005/IMPLEMENTATION.org b/rfd/0005/IMPLEMENTATION.org index 9a11ad4..57d794f 100644 --- a/rfd/0005/IMPLEMENTATION.org +++ b/rfd/0005/IMPLEMENTATION.org @@ -233,8 +233,9 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations cleanup owner only if periodic cleanup becomes necessary. - [ ] Make a new attempt atomically supersede the prior attempt and fence cancellation, expiry, completion, replacement, and disconnection by attempt - generation; install OAuth credentials only when both attempt and expected - integration generations still match. + generation. Starting reauthorization advances the integration generation and + the new attempt captures that post-advance value; install OAuth credentials + only when both attempt and expected integration generations still match. - [ ] Store access tokens, refresh tokens, account identifiers, expiries, and credential generations in the encrypted integration payload without logging device or token material. @@ -250,9 +251,17 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations and credential-generation compare-and-update protection for every success and failure-derived state change. - [ ] Let the first generation-matching successful response atomically persist - rotated tokens even after lease expiry, and make a stale or takeover - ~invalid_grant~ reload and wait for a possible generation advance before - requiring reauthorization. + rotated tokens even after lease expiry. Record ~invalid_grant~ as provisional + ~reauthorization_required~ without advancing the source generation, so a + success already produced from that generation may replace it after any pause; + fence it permanently when user replacement, starting or completing + reauthorization, or disconnection advances the generation. +- [ ] Require every refresh success to contain an access token and valid expiry; + retain the prior refresh token and account ID when omitted, persist a rotated + refresh token when present, and reject an account-ID change as requiring + reauthorization without changing billing identity. +- [ ] Treat bounded waiting after takeover ~invalid_grant~ only as a UI-churn + optimization, not an outcome-selection boundary. - [ ] Add the "ChatGPT Subscription (Beta)" connection flow, billing explanation, progress state, cancellation, quota errors, disconnect, and reauthorization to the authenticated integrations page. @@ -271,9 +280,11 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations contention, multi-node and concurrent-tab device claims, duplicate exchange and completion, LiveView closure and remount, node restart, superseded and stale attempts, cancellation/replacement/disconnection races, late refresh - success after lease expiry, takeover ~invalid_grant~, node pause, replacement - during refresh, permanently lost rotated-token responses, refresh restart - recovery, revocation, exact-model route changes, incompatible route + success after lease expiry, takeover ~invalid_grant~, terminal-first success + after a pause longer than every waiter timeout, node pause, replacement during + refresh, omitted and rotated refresh tokens, omitted and changed account IDs, + permanently lost rotated-token responses, refresh restart recovery, + revocation, exact-model route changes, incompatible route rejection, in-flight route freezing, normalized cross-route replay, no automatic route fallback, same-origin and cross-origin redirect rejection, malicious verification URLs, absence of a credential-bearing follow-up diff --git a/rfd/0005/README.adoc b/rfd/0005/README.adoc index 3008311..c99fdb8 100644 --- a/rfd/0005/README.adoc +++ b/rfd/0005/README.adoc @@ -433,8 +433,12 @@ validation and inference use the session owner's integration. API keys do not refresh. Codex access tokens refresh shortly before expiry, using the stored refresh token and OpenAI's token endpoint. A successful -refresh atomically replaces the access token, rotated refresh token, account -identifier, and expiry in one encrypted payload. +refresh must return a new access token and valid expiry. It atomically writes +those values, retains the existing refresh token or account identifier when the +response omits either one, and replaces the refresh token when the response +rotates it. A different returned account identifier is a contract and billing- +identity violation: Kodo discards that response and requires reauthorization +rather than silently changing the linked account. Refresh is just-in-time, bounded work rather than a long-lived process per integration. A request that encounters a near-expiry token starts the external @@ -464,17 +468,28 @@ overlap the replacement after resuming. Every attempt captures the credential generation it read. A successful response may atomically replace the encrypted payload and advance that generation even if its lease expired while the task or node was paused, but -only if no refresh, replacement, reconnection, or disconnection has already -advanced it. The first successful response wins; later responses reload the -stored result. Failure-derived state changes use the same generation fencing. - -An `invalid_grant` or equivalent terminal response marks the integration as -requiring authorization and discards unusable access for request purposes only -after its generation check succeeds. A stale-lease or takeover attempt that -receives `invalid_grant` first reloads and waits for a bounded interval so a -late successful response can advance the generation; it must not overwrite -that success with a terminal state. Transient provider or network failures -preserve the credential, release or expire the lease, and return a retryable +only if no successful refresh, replacement, reauthorization, or disconnection +has already advanced it. The first successful response wins; later responses +reload the stored result. User replacement, starting or completing +reauthorization, and disconnection always advance the generation and +irrevocably fence every old refresh response. A new authorization attempt +captures the post-advance integration generation. + +An `invalid_grant` or refresh identity mismatch records +`reauthorization_required` with a bounded refresh-specific reason after its +generation check succeeds, but does not advance the source credential +generation or erase the encrypted payload. This state is terminal for new +requests but provisional in refresh-outcome precedence: a successful response +already produced from that same generation may replace it and advance the +generation, even after an arbitrarily long process pause. A successful response +cannot replace reauthorization initiated or completed by the user, replacement, +or disconnection because those actions advance the generation. + +A stale-lease or takeover attempt that receives `invalid_grant` reloads and may +wait for a bounded interval before displaying reauthorization so a common late +success avoids UI churn. The wait is only an optimization and is never the +correctness boundary. Transient provider or network failures preserve the +credential, release or expire the lease, and return a retryable provider-unavailable error. This protocol minimizes concurrent refresh but cannot make an external token From 1e8603440cd0e1ec30bfc3f6260f52297a145b80 Mon Sep 17 00:00:00 2001 From: "Darwin D. Wu" Date: Sun, 30 Aug 2026 00:24:55 -0700 Subject: [PATCH 28/29] docs(rfd): bind device attempt ciphertext --- rfd/0005/IMPLEMENTATION.org | 19 ++++++++++++----- rfd/0005/README.adoc | 42 +++++++++++++++++++++++++------------ 2 files changed, 43 insertions(+), 18 deletions(-) diff --git a/rfd/0005/IMPLEMENTATION.org b/rfd/0005/IMPLEMENTATION.org index 57d794f..4e54cfe 100644 --- a/rfd/0005/IMPLEMENTATION.org +++ b/rfd/0005/IMPLEMENTATION.org @@ -215,9 +215,13 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations - [ ] Generate persistence for one active, expiring, user-scoped, single-use device authorization attempt per ~openai_codex~ integration, including - encrypted sensitive fields, attempt and expected integration generations, - provider deadline and polling interval, database-time claim state, and - bounded cleanup. + encrypted sensitive fields, encryption-key and payload-format versions, + immutable attempt and expected integration generations, provider deadline and + polling interval, database-time claim state, and bounded cleanup. +- [ ] Encrypt every attempt payload with a unique nonce and AEAD associated data + binding attempt ID, integration ID, user ID, provider, immutable attempt + generation, and payload-format version; fail closed on swaps, tampering, + unknown formats, and unavailable keys. - [ ] Implement device-code creation, interval-aware asynchronous polling, cancellation, expiry, terminal errors, and authorization-code exchange using Req rather than embedded browser scripts. @@ -284,7 +288,10 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations after a pause longer than every waiter timeout, node pause, replacement during refresh, omitted and rotated refresh tokens, omitted and changed account IDs, permanently lost rotated-token responses, refresh restart recovery, - revocation, exact-model route changes, incompatible route + revocation, cross-user and cross-attempt ciphertext swaps, attempt tampering, + missing attempt keys, unknown attempt payload versions, routine rotation with + an active old-key attempt, emergency attempt cancellation, exact-model route + changes, incompatible route rejection, in-flight route freezing, normalized cross-route replay, no automatic route fallback, same-origin and cross-origin redirect rejection, malicious verification URLs, absence of a credential-bearing follow-up @@ -309,7 +316,9 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations - [ ] Retain the old key until no payload uses it and the application rollback and backup-retention windows close; test concurrent replacement and disconnection, interrupted-batch resume, mixed-version reads, rollback while - both keys remain configured, and refusal to start with an incomplete ring. + both keys remain configured, active authorization attempts on an old key, + emergency cancellation for a compromised key, and refusal to start when any + integration or attempt key version is missing from the ring. - [ ] Document encryption-key provisioning, separately protected key backups, database-backup and write-ahead-log retention, account deletion limits, credential and provider revocation, staged key rotation, device-flow diff --git a/rfd/0005/README.adoc b/rfd/0005/README.adoc index c99fdb8..ff82497 100644 --- a/rfd/0005/README.adoc +++ b/rfd/0005/README.adoc @@ -207,11 +207,20 @@ and are not written to logs or telemetry. The database permits one active attempt per user and `openai_codex` integration. Each attempt records an attempt generation, expected integration generation, state, provider deadline and polling interval, encrypted sensitive -fields, and a database-time claim lease. Starting a new attempt atomically -cancels and supersedes the prior attempt. Cancellation, expiry, completion, -replacement, and disconnection transitions compare the attempt generation; -completion installs credentials only while the attempt remains active and both -its attempt and expected integration generations still match. +fields with encryption-key and payload-format versions, and a database-time +claim lease. Starting a new attempt atomically cancels and supersedes the prior +attempt. Cancellation, expiry, completion, replacement, and disconnection +transitions compare the attempt generation; completion installs credentials +only while the attempt remains active and both its attempt and expected +integration generations still match. + +The attempt generation is assigned with its encrypted payload and is immutable +for that attempt. Attempt ciphertext uses a unique nonce and AEAD associated +data containing the attempt ID, integration ID, owning user, provider, attempt +generation, and attempt-payload format version. Moving a valid device code, +authorization code, PKCE verifier, or polling payload to another attempt, user, +integration, provider, or generation therefore fails authentication rather +than linking the wrong provider account. Device polling is connection-driven rather than replica-driven. The node that handles creation or a LiveView mount attempts one conditional claim update @@ -316,10 +325,13 @@ Kodo requires a deployment-level credential-encryption key ring, supplied independently of all LLM providers, and records a key version so ciphertext can be rotated. Every production replica validates the complete configured current and non-current key ring before becoming ready to serve integration operations. -Production refuses to persist or decrypt integrations when a required key is -absent or malformed. An AEAD association or authentication failure fails closed -with a bounded corruption error and never falls back or sends the credential to -a provider; recovery requires credential replacement or an operator procedure. +Readiness verifies that every key version referenced by an integration or +authorization-attempt row is present. Production refuses to persist or decrypt +either payload type when a required key is absent or malformed. An AEAD +association or authentication failure fails closed with a bounded corruption +error and never falls back or sends the credential to a provider; recovery +requires credential replacement, restarting authorization, or an operator +procedure. Database encryption does not remove the need to protect the control-plane process; it protects database copies, backups, and read-only database access from directly revealing provider credentials. @@ -338,10 +350,14 @@ The old key remains available for decryption through the documented application rollback and backup-retention window. Rollback restores compatible application behavior while both keys are still configured; it never removes key material first, downgrades ciphertext to plaintext, or enables a provider-environment -fallback. Operators remove an old key only after the database has no payloads -at that version, all replicas use the new ring, the rollback window has closed, -and backups that require the old key have expired or remain paired with a -separately protected copy of that key. +fallback. Routine rotation does not re-encrypt short-lived authorization +attempts: new attempts use the current key, while old keys remain until attempts +at those versions expire and are deleted. Operators remove an old key only +after the database has no integration or attempt payloads at that version, all +replicas use the new ring, the rollback window has closed, and backups that +require the old key have expired or remain paired with a separately protected +copy of that key. Emergency retirement of a compromised key cancels and deletes +affected attempts and requires those users to restart authorization. The credential payload is opaque to Ecto query code and never appears in session events, model mappings, changeset errors, exception inspection, From 303c3a28ddaec33d251d70ee814addb0f6b7f5ec Mon Sep 17 00:00:00 2001 From: "Darwin D. Wu" Date: Sun, 30 Aug 2026 00:29:47 -0700 Subject: [PATCH 29/29] docs(rfd): persist turn route snapshots --- rfd/0005/IMPLEMENTATION.org | 37 +++++++++++++++++++---------- rfd/0005/README.adoc | 47 +++++++++++++++++++++++-------------- 2 files changed, 54 insertions(+), 30 deletions(-) diff --git a/rfd/0005/IMPLEMENTATION.org b/rfd/0005/IMPLEMENTATION.org index 4e54cfe..ac2c1df 100644 --- a/rfd/0005/IMPLEMENTATION.org +++ b/rfd/0005/IMPLEMENTATION.org @@ -13,8 +13,10 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations response, Kodo tool calls, structured output, and every reasoning effort required by current role mappings. - [ ] Record exact model identities available through both ~openai~ and - ~openai_codex~ and verify each route preserves the same role capability - contract; do not treat merely similar models as interchangeable. + ~openai_codex~ and create versioned compatibility records by provider-neutral + selector, source/destination routes, and role-contract version for tools, + structured output, reasoning, context, and modalities; do not treat matching + slugs, catalog fallback, or merely similar models as sufficient evidence. - [ ] Verify refresh-token rotation, control-plane restart recovery, quota errors, revocation, and required reauthorization without using a shared OAuth file. @@ -64,8 +66,10 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations integration using the resulting ~LLMDB.Model.provider~; replace model-string splitting with that resolved provider when storing derived display data. - [ ] Persist exact model identity separately from execution provider and - billing route in every session role mapping; user and repository default - changes affect new sessions only. + billing route in every session role mapping. Define identity as the exact + provider-neutral selector, including an immutable version when both routes + expose one, without claiming rolling aliases are identical backend snapshots; + user and repository default changes affect new sessions only. - [ ] Require exact equality between resolved model and credential providers before recording an invocation or making an HTTP request, with actionable missing, invalid, disconnected, reauthorization-required, malformed-model, @@ -271,13 +275,18 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations reauthorization to the authenticated integrations page. - [ ] Add an explicit per-session execution-route change between OpenAI Platform and ChatGPT Subscription only when the destination supports every - affected role's exact preserved model and capability contract; show billing - confirmation and append a durable route-change event. -- [ ] Freeze the route for an in-flight turn and its continuations, apply a - confirmed route change at the next turn boundary, and replay normalized - visible assistant/tool history without provider-specific reasoning, - signature, or cache state. -- [ ] Route requests whose current execution route is ~openai_codex~ only + affected role's exact provider-neutral selector and has a versioned + compatibility record for its capability contract; show billing confirmation + and reject the whole atomic role set when any role is incompatible. +- [ ] Under the existing durable session lock, assign each accepted turn a route + revision and persist its complete per-role model, execution-route, and + capability-contract snapshot. Write confirmed route changes as pending + revisions under the same lock and apply them only to later accepted turns. +- [ ] Use the persisted turn snapshot for primary, search, review, every tool + continuation, and process/node restart recovery; replay normalized visible + assistant/tool history after a route change without provider-specific + reasoning, signature, or cache state. +- [ ] Route requests whose persisted turn snapshot selects ~openai_codex~ only through the user's ChatGPT integration and prohibit silent fallback to OpenAI API billing. - [ ] Test authorization ownership and expiry, polling backoff, two-node refresh @@ -291,8 +300,10 @@ Implements [[file:README.adoc][RFD 5: Store user-owned LLM provider integrations revocation, cross-user and cross-attempt ciphertext swaps, attempt tampering, missing attempt keys, unknown attempt payload versions, routine rotation with an active old-key attempt, emergency attempt cancellation, exact-model route - changes, incompatible route - rejection, in-flight route freezing, normalized cross-route replay, no + changes, rolling-alias and immutable-snapshot compatibility, whole-set + rejection when one role is incompatible, route-change/turn-acceptance races, + process and node restart during a turn, in-flight route freezing, normalized + cross-route replay, no automatic route fallback, same-origin and cross-origin redirect rejection, malicious verification URLs, absence of a credential-bearing follow-up request, and secret redaction. diff --git a/rfd/0005/README.adoc b/rfd/0005/README.adoc index ff82497..7102df8 100644 --- a/rfd/0005/README.adoc +++ b/rfd/0005/README.adoc @@ -71,26 +71,39 @@ accounts for one provider may be added later with an explicit integration selection in model settings; the first release does not add that complexity. Model role mappings continue to own model selection. Each session preserves -the exact primary, search, and review model identities selected at creation; -later user or repository default changes affect new sessions only. Mappings do -not contain credentials. Their execution routes may use different providers, -in which case the session owner must have every required integration. Kodo -preflights every role integration before accepting a turn and resolves the -current credential again immediately before each provider request. Key -replacement, token refresh, disconnection, or revocation therefore takes effect -for requests that have not started, without rewriting session history. It does -not retroactively cancel a provider request to which Kodo has already sent a -credential. +the exact primary, search, and review model identities selected at creation. An +exact identity is Kodo's provider-neutral wire selector, including an immutable +snapshot or version when both routes expose one; a rolling alias remains that +exact selector but is not a claim that two billing routes use identical backend +weights or deployment snapshots. Later user or repository default changes +affect new sessions only. Mappings do not contain credentials. Their execution +routes may use different providers, in which case the session owner must have +every required integration. Kodo preflights every role integration before +accepting a turn and resolves the current credential again immediately before +each provider request. Key replacement, token refresh, disconnection, or +revocation therefore takes effect for requests that have not started, without +rewriting session history. It does not retroactively cancel a provider request +to which Kodo has already sent a credential. The first release permits an explicit execution-route change between `openai` Platform billing and `openai_codex` ChatGPT subscription billing while preserving the role's exact model identity. Kodo accepts the change only when both routes expose that exact model and satisfy the same role capability -contract. It records a durable route-change event, shows the destination -billing path for confirmation, and applies the new route at the next turn -boundary; an in-flight turn and all its continuations finish on the old route. -Quota, disconnection, or provider failure never triggers this change -automatically. +contract. A versioned compatibility record for the exact selector, source and +destination routes, and role-contract version must confirm tools, structured +output, reasoning options, context requirements, and input modalities. Matching +slugs or ReqLLM catalog fallback alone is insufficient evidence. + +Turn acceptance and route changes use the same durable session lock. Accepting +a turn atomically stores a route revision and complete per-role model, route, +and capability-contract snapshot. Primary, search, review, tool continuations, +and execution resumed after a process or node restart use that turn snapshot, +never the session's then-current mapping. A confirmed route change writes a +pending revision under the lock and affects only subsequently accepted turns; +an in-flight turn and all its continuations finish on its old snapshot. Changes +to multiple affected roles are atomic and Kodo rejects the whole change if any +role lacks exact-selector compatibility. Quota, disconnection, or provider +failure never triggers a route change automatically. After a route change, Kodo replays normalized visible assistant content, tool calls, and tool results while omitting provider-specific reasoning signatures, @@ -368,8 +381,8 @@ the deployment procedure. == Request-time resolution -Kodo combines the session's preserved model identity with its current execution -route and resolves that provider-specific model specification through +Kodo combines each accepted turn's preserved model identity and route snapshot +and resolves that provider-specific model specification through `ReqLLM.model/1` before looking up a credential. The `provider` on the resulting `LLMDB.Model` is canonical and must equal the selected route; provider text derived from a stored model string is display data and cannot select a