Add Custom model endpoint provider by paullizer · Pull Request #1228 · microsoft/simplechat

Paul Lizer (paullizer) · 2026-08-12T14:36:12Z

Summary

add a user-facing Custom model endpoint provider for OpenAI API, Azure OpenAI API, and Anthropic
support manual model configuration across global, personal, and group scopes with type-specific model/deployment names, API-key authentication, response-length settings, and Anthropic Version
wire Custom models through chat, streaming, agents, workflows, summaries, metadata extraction, multimodal requests, and model catalogs
preserve existing authorization, governance, and Key Vault secret-storage behavior
enforce HTTPS, blocked-address policy, optional private-host governance, runtime revalidation, redirect refusal, sanitized provider errors, and DNS address pinning against rebinding
document the feature, update release notes, and bump the application version to 0.250.168

Validation

26 targeted functional and route-policy test scripts passed
modified Python modules compiled successfully
modified JavaScript files passed node --check
git diff --check passed
two final code-review passes reported no remaining actionable findings

Support OpenAI, Azure OpenAI, and Anthropic API contracts across global, personal, and group endpoint scopes with manual models, API-key authentication, runtime integration, and hardened outbound networking. Refs #1222 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Preserve the Custom model endpoint release notes alongside the latest Development history and advance the application version to 0.250.172. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

…228-conflicts # Conflicts: # application/single_app/config.py # application/single_app/functions_documents.py # application/single_app/functions_model_endpoint_runtime.py # application/single_app/functions_settings.py # application/single_app/functions_workflow_runner.py # application/single_app/model_endpoint_clients.py # application/single_app/route_backend_chats.py # application/single_app/route_backend_conversation_export.py # application/single_app/route_backend_settings.py # application/single_app/route_frontend_admin_settings.py # application/single_app/templates/admin_settings.html # docs/explanation/release_notes.md

…228-conflicts # Conflicts: # application/single_app/config.py # docs/explanation/release_notes.md

…endpoint normalization Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

…228-conflicts

Take Development through the revert of the Custom model endpoint provider (microsoft#1431 reverting microsoft#1228). Only two files conflicted, both mechanical. * config.py: kept VERSION 0.261.011. The revert returned Development to 0.261.009, so this is still ahead and needs no renumber. * release_notes.md: kept the v0.261.011 DLP section and took Development's side of the region below it, so the reverted v0.261.010 Custom model endpoint section is not reinstated. * Regenerated docs/explanation/release-notes/ so the generated pages drop the reverted v0.261.010 entry with the source. docs/_data/app_surface.yml was regenerated and is unchanged, since the revert did not alter the application surface this PR documents. Re-verified the DLP enforcement paths against the reverted file contents rather than assuming they carried over, because microsoft#1228 had touched both files: * save_chunks and save_chunks_batch still evaluate upload DLP before any embedding call, and the embedding character clamp still only narrows sanitized_chunk_text. * perform_web_search still has no user_message fallback, so only the DLP-approved query can reach the outbound search. * Video enhanced-citation upload still gates on the caller-supplied enable_enhanced_citations flag that upload DLP clears. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

…ease Development published v0.261.010 and v0.261.011 for the Azure Managed Redis work while this branch was using the same numbers, so both release-notes sections and config.py collided. Development's sections are authoritative and are kept exactly as published. This branch's nine sections move above them, to v0.261.012 through v0.261.020, and config.py becomes 0.261.020. The restored PR #1228 entry is renumbered too: its original v0.261.010 claim was vacated when that PR was reverted, so it re-lands alongside the fixes stacked on it. Version references in this branch's tests and feature documentation are updated to match, so each still names the version its behaviour ships in. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

…ease Development published v0.261.012 for the Yamcs basic authentication work while this branch was using that number for the restored PR #1228 entry, so the release notes and config.py collided again. Development's sections are kept exactly as published. This branch's nine sections shift up one, to v0.261.013 through v0.261.021, and config.py becomes 0.261.021. Version references in this branch's tests and feature documentation move with them. The sections belonging to this branch are now identified by content rather than by a hardcoded list of numbers, so a further collision while this PR is open is resolved the same way without guessing which sections are whose. The generated release-notes pages are rebuilt from the merged source rather than hand-resolved, since they are build output. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Re-applies the changes from PR microsoft#1228, which was merged prematurely and reverted by PR microsoft#1431. This restores the feature onto current Development as a working base so the remaining provider-coverage gaps can be closed before it is proposed for merge again. Refs microsoft#1228 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

SimpleChat decided what a model could do by pattern-matching its name, so any model the shipped catalog did not know about -- an on-premises or customer supplied model reached through a Custom endpoint -- silently got the wrong answer for vision, tool calling, streaming, and reasoning. A model named "corp-llm-v2" was treated as having no capabilities at all. Capability answers now resolve through per-model override, endpoint override, catalog entry, then the original name heuristic, so the catalog can be incomplete without blocking an administrator and unknown models behave exactly as they did before. Two matching rules keep this honest. "family" is never used to match, because members of one family disagree on capabilities: the phi-4 family holds both the multimodal and the text-only Phi models, and each gpt-5.x family holds a non-vision "-chat" member. And a longer identifier prefix wins, with a digit-leading remainder rejected as a version continuation rather than a variant, so gpt-5.1-chat-v2 resolves to gpt-5.1-chat and gpt-5.3 no longer resolves to gpt-5. The catalog gains supportsStreaming and reasoning for all 65 existing models and 10 Google Gemini records, a provider it did not cover at all. Claude, Llama 4, and Phi-4 multimodal are now correctly recognised as vision-capable, and the gpt-5.x "-chat" variants correctly as not. A published JSON schema plus a coverage test means a malformed record fails a test rather than degrading capability answers at runtime. Refs microsoft#1228 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Custom endpoints supported exactly three API types, each hard-coded in five places: the allowlist, the request-model resolver, the protocol inference chain, the admin template option list, and the admin JavaScript. Adding a provider meant editing all five, and missing one failed silently. An API type is now one declarative record carrying its wire protocol, model identifier field, URL policy, accepted auth types, and version field. The admin option list, model identifier label, version fields, validation, and protocol inference all derive from it. The three existing API types are unchanged and an unregistered api_type is still refused. Google Gemini is registered, reached through its OpenAI-compatible surface so it keeps running on the validated-DNS pinned transport rather than needing a vendor SDK with its own transport. This also fixes URL construction for any surface that already carries a version segment. SimpleChat appended /v1 unconditionally, turning Gemini's .../v1beta/openai base into .../v1beta/openai/v1 and a 404. The URL policy is now per provider, so /v1 is appended only where it belongs. The registry reaches the browser as server-rendered inline JSON, through the modal's data-api-types attribute and a window helper in base.html for scripts that build model lists without the endpoint editor present. No hard-coded api_type comparison remains in the admin or workspace scripts, which the existing UI contract test now asserts. Refs microsoft#1228 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Every Custom endpoint failure produced the same sentence, "Custom model request failed.", with the cause discarded by "from None" before it could reach the log. A wrong path, a wrong key, a wrong model name, a TLS failure, and a blocked address were indistinguishable, and nothing anywhere explained which had happened. Seven failure paths did this. Sanitizing what reaches the browser is right, because an upstream error body can echo back a URL, a header, or an API key. Destroying it before it reaches the server log is not. The browser message stays sanitized and now carries a short reference id. The real cause is recorded server-side with the API type, the resolved request URL, the upstream status code, and the upstream body, stamped with the same id so an administrator can join a reported message to the entry that explains it. The resolved request URL is included deliberately: URL normalization rewrites what the administrator typed, and that rewrite was previously invisible, so a 404 caused by a rewritten path could not be diagnosed at all. Credentials are redacted before anything is logged, covering api_key fields, Authorization and bearer tokens, x-api-key, x-goog-api-key, key query parameters, and sk- style keys. Detail is truncated so a large upstream body cannot flood the log, and a logging failure is swallowed so it can never replace the model error. The existing provider test asserted the message was exactly "Custom model request failed."; it now asserts the security property, that no provider detail leaks, plus the presence of a reference id. Refs microsoft#1228 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

SimpleChat appended /v1 to every Custom OpenAI-compatible endpoint, even when the configured URL already said where the API lived. A gateway at https://apim.example.com/inference/chat/completions was called at https://apim.example.com/inference/v1/, and any base carrying its own version segment was broken the same way. The rewrite was also invisible, so a misdirected request looked identical to a correct one. /v1 is now appended only when the URL does not already name the API surface. A last path segment matching a version, such as v1, v2, or v1beta, is left alone, and a full operation URL is treated as stating the base exactly, so its suffix is stripped and the remainder used as given. Some gateways mount the API at a path that cannot be inferred, where both the given path and a /v1 child are plausible. Rather than guess, the endpoint editor gains a "Use this URL exactly as entered" option, stored as connection.url_mode and honoured for any API type. Test Connection now reports the URL that was actually called, so the rewrite is always verifiable instead of silent. Refs microsoft#1228 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

SimpleChat supports one response mode, streaming, but some responses cannot be streamed by the model. Tool calling is the clearest case: a tool call has to arrive complete, so the request is made without streaming. That path already pushed the completed answer through the streaming interface, but as a single chunk, so the user saw nothing and then everything at once, which reads as a hang. Agents and plugins rely on tool calling, so this was a common path rather than an exotic one. A completed answer is now split at word boundaries and delivered progressively. Chunking is lossless, which matters because the frontend accumulates chunks by concatenation: the tokenizer preserves every character, including runs of whitespace and newlines, so the reassembled text is identical to the original. Function calls cannot be partially delivered, so non-text items ride on the final message alongside the finish reason and usage metadata. Emitting metadata exactly once keeps token usage from being multiplied by the chunk count. This also restores usage reporting on streaming responses. stream_options is how an OpenAI-compatible stream reports token usage, and it was stripped from every request, including from endpoints that accept it. Support is now declared per provider and the option is dropped only for surfaces that reject it. Refs microsoft#1228 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

The gate named "allow private Custom endpoint hosts" did not permit the two address forms on-premises deployments actually use. An IP address and a short single-label host name were both rejected even with the gate enabled, and both were refused with a message claiming the URL was an IP address, which was simply wrong for the short host name and named no setting that would allow it. Even once an address was accepted, the connection could not complete. The outbound transport trusts only certifi public roots and deliberately ignores SSL_CERT_FILE, so a gateway presenting an internally issued certificate could never be validated. On-premises inference was unreachable in practice. With the gate enabled, IP addresses, short host names, and private-range hosts are now accepted, and administrators can name a CA bundle to trust an internal issuer. Ambient environment variables are still ignored, so widening trust stays a recorded configuration decision, and a bundle that cannot be loaded fails the request rather than silently falling back to public roots. Plaintext HTTP gets its own second gate for isolated networks, labelled with its consequence, and requires the private-hosts gate as well. None of this weakens the outbound protections. Loopback, link-local, and cloud metadata addresses stay blocked with every gate enabled, and addresses are still revalidated at connection time. Saving an endpoint no longer requires the host to resolve from the application tier, so configuration can be seeded or restored ahead of connectivity. Only resolution is tolerated; policy violations are still refused at save time, and are reported as policy violations rather than resolution failures. Egress proxies remain unsupported on purpose: with a proxy the CONNECT target is resolved by the proxy, so connection-time pinning would protect nothing, and replacing it with a host allowlist is a security decision rather than plumbing. Refs microsoft#1228 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Custom endpoints accepted one scheme: an API key sent in whichever header the built-in providers happened to use. That covers OpenAI and Anthropic and nothing else, so a gateway reading the key from a different header, a corporate gateway issuing short-lived tokens, and an appliance requiring a client certificate were all unreachable. The API key header name and value prefix are now configurable, which is what lets one scheme cover Authorization: Bearer, Anthropic's x-api-key, Google's x-goog-api-key, and bespoke gateway headers. Each provider supplies its own default so the common case needs no configuration. Static bearer tokens are supported, as is the OAuth2 client credentials grant. Tokens are cached per token URL, client, and scope, and refreshed ahead of expiry so one cannot lapse mid-request. Two things about the token endpoint matter. It is a different host from the inference endpoint, so it is validated against the same outbound policy and cannot become an unchecked request target: a token URL pointing at cloud metadata is refused exactly as an inference endpoint would be. And its error body routinely echoes the client id or secret, so failures are sanitized to the browser and recorded server-side with credentials redacted. mTLS is modelled as a transport concern rather than an auth type, because a client certificate combines with any scheme. Certificates are referenced by file path, never by value, so a private key is mounted into the deployment instead of being written to the configuration database and replicated with it. Refs microsoft#1228 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Validate the OAuth2 token endpoint when the token is fetched, not only when the endpoint is saved. Checking at save time leaves the request itself unguarded, because settings can be written by another path, restored from backup, or changed after validation, so code scanning was right to call this a server-side request forgery. The token URL is now revalidated against the same outbound policy as the inference endpoint and the request runs on the same pinned transport, so its addresses are validated at connection time. That transport refuses redirects, which the previous code avoided on the stated grounds that token endpoints commonly redirect. That reasoning was wrong: redirects belong to the browser-based authorization-code flow, while a client-credentials token endpoint answers a server-to-server POST with a JSON body. A regression test now asserts a blocked token URL is refused before any HTTP client is constructed. Remove backtracking from the version-segment pattern. Its optional suffix could begin with a digit, making it ambiguous with the preceding digits and quadratic on a long run of them: 8,000 characters took about 0.19s to reject and now takes 0.0003s. Requiring the suffix to begin with a letter matches exactly the same segments. Move the auth-type constants and their normalizer to the provider registry so validation no longer imports the authentication module. That removes the real coupling behind several cyclic-import findings and lets the token fetch import validation normally rather than through a deferred import whose comment would otherwise have become untrue. Drop genuinely dead imports, and declare the deliberate re-exports in functions_model_endpoint_types with __all__, since functions_settings and the provider tests import those constants through it. Close file handles in tests and stop importing the same module both ways. The remaining cyclic-import findings are pre-existing: the same single nine-module cycle is present on Development, and none of the modules in this change participate in an import-time cycle. Refs microsoft#1228 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Paul Lizer (paullizer) and others added 2 commits August 12, 2026 10:35

Merge Development and resolve conflicts

171bb2a

Preserve the Custom model endpoint release notes alongside the latest Development history and advance the application version to 0.250.172. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

github-advanced-security Bot found potential problems Aug 12, 2026

View reviewed changes

Comment thread application/single_app/functions_documents.py Dismissed

Paul Lizer (paullizer) mentioned this pull request Aug 19, 2026

Link release note issue references and archive pre-v0.230.001 notes #1292

Merged

Paul Lizer (paullizer) and others added 4 commits September 3, 2026 22:54

Merge remote-tracking branch 'origin/Development' into paullizer-pr-1…

08e1218

…228-conflicts # Conflicts: # application/single_app/config.py # docs/explanation/release_notes.md

Align model endpoint tests with Development admin template panes and …

d419752

…endpoint normalization Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Merge remote-tracking branch 'origin/Development' into paullizer-pr-1…

6905995

…228-conflicts

Paul Lizer (paullizer) merged commit 24b9336 into Development Sep 4, 2026
12 checks passed

Paul Lizer (paullizer) mentioned this pull request Sep 4, 2026

Revert "Add Custom model endpoint provider" #1431

Merged

Paul Lizer (paullizer) mentioned this pull request Sep 4, 2026

Re-land Custom model endpoint provider with the provider-coverage gaps closed #1437

Merged

Navigation Menu

Repository navigation

Add Custom model endpoint provider - #1228

Add Custom model endpoint provider#1228
Paul Lizer (paullizer) merged 6 commits into
Developmentfrom
paullizer-custom-model-endpoints

Paul Lizer (paullizer) commented Aug 12, 2026

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Paul Lizer (paullizer) commented Aug 12, 2026

Summary

Validation

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants