Skip to content

Tag native OAuth connections and failures in MCP analytics - #274

Merged
rgarcia merged 2 commits into
mainfrom
hypeship/mcp-auth-method-analytics
Nov 13, 2026
Merged

rgarcia merged 2 commits into
mainfrom
hypeship/mcp-auth-method-analytics

Conversation

@rgarcia

@rgarcia rgarcia commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Summary

MCP analytics could not tell Kernel-issued (native) OAuth apart from Clerk OAuth, and native authentication failures left no analytics record at all: they return 401/403/503 straight from the native path, before the scope resolution that emits mcp_connection_scope_failure. That makes a native rollout hard to watch, since a native-only failure or drop in connections is invisible.

  • $mcp_initialize events now report $mcp_auth_method: "native_oauth" for native connections. Clerk OAuth stays "oauth" and API keys stay "api_key", so existing queries keep their meaning.
  • A native credential the server refuses (rejected token, revoked grant, refused scope, issuer outage) now emits mcp_connection_scope_failure with connection_credential_type: "native_oauth" and the outcome (rejected or unavailable).
  • Every mcp_connection_scope_failure now records where it happened and what the client got: connection_failure_stage (authentication for this server refusing a native token, scope for the Kernel API refusing the connection) and response_status_code (the status this server returned). upstream_status_code keeps one meaning: the status the Kernel API returned, so it is set only for scope failures. Clerk token verification failures are still not recorded.

The credential type is passed from the route, which already knows which path authenticated the request; /auth/context reports jwt/oauth for both OAuth kinds.

Tests

  • Route: a native initialize produces $mcp_auth_method: "native_oauth"; a revoked native grant, a refused scope and an issuer outage each capture an authentication failure with the status returned; API-key refusals capture scope failures with both statuses.
  • connectionAnalyticsFromContext reports the route's credential type and nothing when it disagrees with /auth/context.
  • bunx tsc --noEmit and bun test (1191 tests) pass.

@vercel

vercel Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
mcp Ready Ready Preview Nov 13, 2026 6:01pm UTC

@masnwilliams masnwilliams left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Native connections are tagged native_oauth, native refusals now leave a record, and the tests cover both. tsc --noEmit and the route, auth-context, and analytics suites pass locally.

Worth doing here or in a quick follow-up

  • Say which stage the failure came from. A native token can now produce a rejected/401 native_oauth event from two places: the issuer refusing it in authenticateNativeOAuth, or /auth/context refusing the exchanged token in handleMcpRequestWithIdentity. The two events are identical, but they're the two failures you'd most want to tell apart during a rollout. A stage: "authentication" | "scope" field would separate them. The comment on McpConnectionScopeFailureAnalytics (and the matching test comment) also still says only scope resolution emits this event. Related: Clerk verifyToken failures emit nothing, so native and Clerk failure rates won't compare directly. Fine either way, but worth saying explicitly in the comment.

Optional

  • The two capture sites build upstreamStatusCode with opposite ternaries for the same rule. NativeOAuthResult's failure arms already fit McpConnectionContextFailure, so captureMcpConnectionScopeFailure could take { stage, credentialType, failure } and read statusCode in one place. I tried this locally and the production code typechecks unchanged; only the captured assertions need reshaping.
  • authMethod: isApiKey ? "api_key" : credentialType takes its value from two sources. Gating on credentialType === "api_key" ? isApiKey : isOAuth and returning authMethod: credentialType would make the route's credential type the only source. It also gives the new native_oauth assertion in auth-context.test.ts something to check beyond the argument coming back unchanged.
  • The revocation test asserts captured after the 401 but not after the 403 refused-scope case. That would be worth asserting too.

@masnwilliams masnwilliams left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, passing the credential type down from the route is the right call. a few things, none blocking:

1. please settle what upstream_status_code means before merge. for api keys it's the status the kernel api actually returned. for native failures it's the 401/403 that native-oauth.ts assigns itself, which also covers purely local refusals (failed local verification, oversized token, native oauth not configured), and the issuer's real http status on RFC 8693 errors is never recorded. once these events ship, any dashboard grouping by this field mixes the two meanings and the historical events can't be relabeled. either leave it unset for native and add something like response_status_code, or document it as "the status we returned" for every failure. if the latter is the intent, a comment is enough.

2. (suggestion) the capture + response block is now duplicated in route.ts. the native failure type already fits McpConnectionContextFailure, so one helper covers both paths:

// A refused connection never reaches the instrumented server, so this
// capture is its only analytics record.
function refuseConnection(
  req: Request,
  credentialType: McpCredentialType,
  failure: McpConnectionContextFailure,
): Response {
  captureMcpConnectionScopeFailure({
    outcome: failure.status,
    credentialType,
    upstreamStatusCode: "statusCode" in failure ? failure.statusCode : undefined,
  });
  if (failure.status === "invalid")
    throw new Error("Unable to resolve Kernel connection scope");
  return connectionScopeFailureResponse(req, failure);
}

then both call sites become if (x.status !== "ok") return refuseConnection(req, <type>, x);. tsc and the full suite pass with this locally.

3. (suggestion) connectionAnalyticsFromContext takes authMethod from two sources. "api key or not" comes from /auth/context, "which oauth" from the caller, and the new param is ignored for api-key contexts. making the route's credential type the only source and keeping the context check as a guard reads more directly:

if (!(credentialType === "api_key" ? isApiKey : isOAuth)) return null;
return { authMethod: credentialType, /* ... */ };

only changes behavior when the two disagree (emits nothing instead of a mixed label); a test for the mismatched case would pin it.

nits

  • the comment on McpConnectionScopeFailureAnalytics still says the event is about scope resolution; it now also covers native authentication failures.
  • observeConnections in route.test.ts reads like the route's observeConnection but actually forces isMcpAnalyticsEnabled; something like forceAnalyticsEnabled would be clearer.
  • clerk jwt verification failures are now the only oauth path with no failure event. fine for a follow-up.

@rgarcia
rgarcia merged commit 78c8a8d into main Nov 13, 2026
10 checks passed
@rgarcia
rgarcia deleted the hypeship/mcp-auth-method-analytics branch November 13, 2026 18:04

This branch was successfully deployed

1 active deployment
Preview — 210adaa3 Deployed Nov 13, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants