Skip to content

feat: add private ingress request boundary - #5922

Open
TristanSpeakEasy wants to merge 6 commits into
agentlink/ais-666-private-oauth-authorityfrom
agentlink/ais-667-private-request-boundary
Open

feat: add private ingress request boundary#5922
TristanSpeakEasy wants to merge 6 commits into
agentlink/ais-666-private-oauth-authorityfrom
agentlink/ais-667-private-request-boundary

Conversation

@TristanSpeakEasy

@TristanSpeakEasy TristanSpeakEasy commented Sep 1, 2026

Copy link
Copy Markdown
Member

Why

Private MCP traffic needs a serving boundary that can prove which provisioned ingress forwarded a request before trusting its Host or advisory network identity. The public application mux cannot provide that isolation because it also exposes management, callback, hook, and marketplace surfaces.

What changed

  • Added provider-neutral request identity parsing, Tailscale header normalization, active-ingress lookup, Kubernetes TokenReview verification, bounded credential caching, per-source and global rate limiting, coalesced authority rechecks, and fail-closed Host/identity middleware.
  • Added /app/gram netingress-attestor with a separate pod-local health listener. It allowlists private MCP/OAuth/install routes, overwrites workload attestation from a rotating projected token, preserves MCP Authorization, strips unsupported identity/client-IP headers, and supports streaming and cancellation.
  • Added an optional TLS-only GRAM_NETINGRESS_ADDRESS private application listener with its own mux, observability/recovery middleware, serving-policy version, TokenReview boundary, and concurrent graceful drain.
  • Defined one typed private route catalog consumed by the route guard and both /mcp and /x/mcp handler registration, preventing allowlist/mount drift and excluding global callback paths.
  • Private endpoint resolution reloads live ingress authority and pins namespace plus organization before serving.

Review notes

  • Stacked on feat: pin private OAuth ingress authority #5896 (agentlink/ais-666-private-oauth-authority). Review only the AIS-667 commits on this branch.
  • Closest review areas: server/internal/netingress/attestation.go, middleware.go, attestor.go, and routes.go; composition and shutdown wiring are in server/cmd/gram/start.go.
  • The listener remains disabled when GRAM_NETINGRESS_ADDRESS is empty. This PR does not deploy the listener, grant product entitlement, enable the PostHog rollout, or add Kubernetes/NetworkPolicy resources; those remain separate follow-up tickets.
  • Tailscale identity remains advisory. TokenReview workload identity plus canonical Host binding establish ingress authority; ordinary Gram authentication and authorization still run afterwards.
  • Per-ingress NetworkPolicy in AIS-608 is required before deployment so only the matching ProxyGroup can reach each attestor. HTTPS, TokenReview, and Host binding remain independent defenses.

Validation

  • mise run test:server ./internal/netingress ./internal/middleware ./cmd/gram — 302 tests passed
  • mise run test:server -race ./internal/netingress — 63 tests passed
  • mise run test:server ./internal/xmcp — 54 tests passed
  • mise run test:server ./internal/mcp — 667 tests passed
  • mise build:server
  • mise lint:server
  • git diff --check
  • Independent security/correctness review; actionable findings addressed
  • All 17 Ultrareview threads replied to and resolved across two review cycles; the final asynchronous Cubic check for 0b72fe5d32 remains pending

Linear: AIS-667

Rollout / deployment notes

  • No migration or infrastructure change in this PR.
  • GRAM_NETINGRESS_ADDRESS defaults empty, so existing deployments keep only the public listener.
  • Enabling the private listener requires GRAM_NETINGRESS_TLS_CERT_FILE and GRAM_NETINGRESS_TLS_KEY_FILE.
  • The attestor requires an HTTPS upstream URL, GRAM_NETINGRESS_UPSTREAM_CA_FILE, expected private Host, and projected ServiceAccount token path. Its dedicated transport ignores ambient proxy variables.
  • Product entitlement, organization-targeted PostHog rollout, provider mutations, private Service/RBAC, and per-ingress NetworkPolicies remain off/out of scope.

@linear-code

linear-code Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

AIS-667

@changeset-bot

changeset-bot Bot commented Sep 1, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 9906711

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@TristanSpeakEasy
TristanSpeakEasy marked this pull request as ready for review September 1, 2026 03:17
@TristanSpeakEasy
TristanSpeakEasy requested a review from a team as a code owner September 1, 2026 03:17
@cubic-dev-ai

cubic-dev-ai Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Running ultrareview automatically — This adds a new security-critical network ingress boundary with TokenReview verification, credential caching, rate limiting, and fail-closed middleware—any subtle flaw could bypass authentication, so an ultrareview is warranted.. I'll post findings when complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ultrareview completed in 27m 10s

All reported issues were addressed across 22 files

Linked issue analysis

Linked issue: AIS-667: Attestor, TokenReview verifier, and isolated private listener

Status Acceptance criteria Notes
TokenReview validates audience, service-account subject, and token expiry; inactive or mismatched ingress combinations fail closed. The verifier sends the required audience, accepts only service-account subjects, rejects missing or expired JWTs, looks up active ingress bindings, and rechecks serving state for disabled, deleted, or changed ingresses.
Host mismatches are rejected and private ingress authority is bound to the verified ingress. Both the attestor and private middleware canonicalize Host and require it to match the expected or verified ingress DNS name.
The private listener exposes only the intended MCP/OAuth/install routes and no RPC, admin, hooks, marketplace, callback, or other global routes. A single private route catalog drives route guarding and private handler registration, with explicit reserved-path rejection and route-census coverage.
Attestation does not overwrite MCP Authorization, unsupported identity headers are stripped, and attestation tokens are absent from logs and errors. The attestor replaces only the dedicated attestation header, preserves Authorization, strips unsupported Tailscale and client-IP headers, and error paths use generic messages without including the token.
⚠️ Streaming POST/SSE/DELETE requests and cancellation pass through the attestor and private listener. The route catalog registers POST, GET/SSE, and DELETE runtime methods, and an in-process SSE cancellation test verifies propagation, but the supplied evidence does not independently exercise streaming POST and DELETE cancellation.

Tip: instead of fixing issues one by one fix them all with cubic

Re-trigger cubic

Comment thread server/cmd/gram/netingress_attestor.go Outdated
Comment thread server/cmd/gram/netingress_attestor.go
Comment thread server/internal/netingress/queries.sql Outdated
Comment thread server/cmd/gram/start.go
Comment thread server/cmd/gram/start.go Outdated
Comment thread server/internal/netingress/attestation.go Outdated
Comment thread server/internal/netingress/tailscale_identity.go Outdated
Comment thread server/internal/netingress/lookup.go
Comment thread server/internal/netingress/middleware.go Outdated
Comment thread server/internal/xmcp/service.go Outdated
@TristanSpeakEasy
TristanSpeakEasy requested a review from a team as a code owner September 1, 2026 04:58
@TristanSpeakEasy
TristanSpeakEasy force-pushed the agentlink/ais-667-private-request-boundary branch from d56988e to ad1bd9c Compare September 1, 2026 04:59
@blacksmith-sh

This comment has been minimized.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 16 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic

Comment thread server/internal/netingress/attestation.go Outdated
Comment thread server/internal/netingress/tailscale_identity.go Outdated
Comment thread server/internal/netingress/middleware.go Outdated
Comment thread server/internal/netingress/attestation_test.go
@TristanSpeakEasy

Copy link
Copy Markdown
Member Author

Addressed the Ultrareview summary from #5922 (review) across ad1bd9c and 764ac80. The code now re-resolves private endpoints from live ingress authority, requires TLS and explicit CA trust, disables ambient proxying, scopes serving-state rechecks by organization, reserves xMCP callbacks, bounds/coalesces verification work, unifies the reserved header, handles IPv6 authority safely, and tightens identity parsing. The wildcard attestor listener remains intentionally pod-reachable; AIS-608 per-ingress NetworkPolicy is the required source boundary before deployment. All 13 inline threads have detailed replies and are resolved.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 8 files (changes from recent commits).

Tip: Review your code locally with the cubic CLI to iterate faster.

Fix all with cubic | Re-trigger cubic

Comment thread server/internal/netingress/tailscale_identity.go Outdated
@TristanSpeakEasy
TristanSpeakEasy force-pushed the agentlink/ais-667-private-request-boundary branch from 0b72fe5 to 7570eed Compare September 2, 2026 00:10
@TristanSpeakEasy

Copy link
Copy Markdown
Member Author

Addressed the remaining actionable Cubic identity-parsing feedback in commit 7570eeda17: every raw RFC 2047 marker is now validated without rejecting literal markers produced by valid decoding. Original review: #5922 (review)

@TristanSpeakEasy
TristanSpeakEasy force-pushed the agentlink/ais-667-private-request-boundary branch from 7570eed to 8677957 Compare September 2, 2026 01:20
@TristanSpeakEasy
TristanSpeakEasy force-pushed the agentlink/ais-667-private-request-boundary branch from 8677957 to 9906711 Compare September 2, 2026 11:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant