Repository navigation
🤖 test: add the CoderMCPServer Kind E2E phases - #389
Conversation
Refs #155 Change-Id: I4efe2c67f8baae36128ae75dac8d3723db97f669 Signed-off-by: Thomas Kosiewski <tk@coder.com>
|
@codex security review |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
🛡️ Codex Security ReviewSecurity review completed. No security issues were found in this pull request. Reviewed commit: Only the user who started this review can view the report in Codex. ℹ️ About Codex security reviews in GitHubThis is an experimental Codex feature. Security reviews are triggered when:
Once complete, Codex will leave suggestions, or a comment if no findings are found. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7c5864d3fb
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Change-Id: I207bcc7046ff83339244700771d7e8d01703f9bb Signed-off-by: Thomas Kosiewski <tk@coder.com>
Change-Id: If189ddd1f40425390164d2e20dc9eecdd8de9c99 Signed-off-by: Thomas Kosiewski <tk@coder.com>
|
@codex review |
|
@codex security review |
🛡️ Codex Security ReviewSecurity review completed. No security issues were found in this pull request. Reviewed commit: Only the user who started this review can view the report in Codex. ℹ️ About Codex security reviews in GitHubThis is an experimental Codex feature. Security reviews are triggered when:
Once complete, Codex will leave suggestions, or a comment if no findings are found. |
|
Codex Review: Didn't find any major issues. Bravo. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
…report while a turn runs Change-Id: I77f799fa8c9d58f1e880f14f1f11f43edf5cf88f Signed-off-by: Thomas Kosiewski <tk@coder.com>
|
@codex review |
|
@codex security review |
🛡️ Codex Security ReviewSecurity review completed. No security issues were found in this pull request. Reviewed commit: Only the user who started this review can view the report in Codex. ℹ️ About Codex security reviews in GitHubThis is an experimental Codex feature. Security reviews are triggered when:
Once complete, Codex will leave suggestions, or a comment if no findings are found. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 32036c4de7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
… log in Change-Id: Ic2b43f6e0689ff70cb37f90d2973b469d0f1c80c Signed-off-by: Thomas Kosiewski <tk@coder.com>
|
@codex review |
|
@codex security review |
🛡️ Codex Security ReviewSecurity review completed. No security issues were found in this pull request. Reviewed commit: Only the user who started this review can view the report in Codex. ℹ️ About Codex security reviews in GitHubThis is an experimental Codex feature. Security reviews are triggered when:
Once complete, Codex will leave suggestions, or a comment if no findings are found. |
|
Codex Review: Didn't find any major issues. Nice work! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
1 participant
Summary
The Kind E2E driver now exercises the active
CoderMCPServercontroller against a real Coder (v2.37.2, no license). It uses the stub from #371 as both the MCP server and the OpenAI-compatible LLM, so a real Coder chat proves credential delivery and tool names. This is plan step M10 of #155 (plan 7.3, phases P0 to P7 and P9). P8, the tool lists, comes with M11.Phases (
hack/e2e-workspace-lifecycle.sh)The new steps run after the AI Gateway key phases and before the namespace deletion.
imagePullPolicy: Never, Servicesmcp-stubandmcp-stub-2). Creates anopenai-compatprovider and a model that point at the stub, through the Coder API.has_license: false, the stub is Ready. P4 (e) runs here (see below). Then the CCP opt-in lists the organization.http://mcp-stub.<ns>.svc:8080and anapi_keyCR withaccess.groups: [Everyone]Ready=True Applied. Coder's GET bystatus.configID: slug, URL,enabled,has_api_key. The ACL is exactly Everyone read. The CR and the operator log hold no key.tools/listrequest at the stub has the right Host and the SHA-256 of the local key. The LLM got exactlye2e-mcp__alpha,e2e-mcp__beta,e2e-mcp__Gamma.updated_atis unchanged.force_onwithoutallowForceOnCredentialNotOptedIn,CredentialRefused,CredentialDestinationNotAllowed,SlugConflict,MCPServersNotAllowed,ForceOnNotAllowed, and noconfigID. Coder's config list (IDs, slugs, URLs,enabled,updated_at) is the same before, after, and after deleting the CRs.spec.urltomcp-stub-2, then the new origin in the SecretCredentialDestinationNotAllowed, and Coder keeps the old URL, enabled. After: Ready, the new URL, and the next turn reachesmcp-stub-2with the matching key hash.access.groups: [], then[Everyone], with the tester user's sessiondisplayNameeditDisabledExternallyand Coder stays disabled. After the edit: Ready and enabled.ControlPlaneGonerelease.P4 (e) runs in P0, before the P1 server exists. Unlicensed Coder has one organization, and a CR that names a missing organization gets
OrganizationNotFound, notMCPServersNotAllowed. So the driver sets the CCP list to[e2e-no-such-org], checks the refusal and that Coder is unchanged, and then sets the real list. Changing the list after P1 would disable the P1 server.Keys are 48 random hex characters in 0600 files, masked in GitHub Actions logs, and reach
jq --rawfile,kubectl create/replace -f,grep -fandsha256sumonly through files. The driver prints only "key hash match" or fails with "no match". It reads the stub's/reportthrough the API server's service proxy, so no extra port-forward runs. The chats run as the tester: a chat created with the operator token never ran a turn on Kind. The driver logs in as the tester in P0, and on a local rerun that reuses the tester (409) it first sets this run's password. The tester's session token stays in the 0600 filemember.hdr, which cleanup removes. While a turn runs, the driver logs the stub's request methods and tool counts (no header values).Red evidence (
hack/e2e-workspace-lifecycle_test.sh)The offline stubs gain a fake
CoderMCPServercontroller, the fake Coder MCP routes, a fake stub report and a fake chat. Each phase has at least one failure row that breaks the checked behavior. Each row asserts that the driver exits non-zero with the FAIL message below and stops before the next mutation:mcp-licensedCoder has a license: the MCP phases test unlicensed Codermcp-never-readytimed out ... waiting for: MCP server e2e-mcp Readymcp-cr-leakCoderMCPServer e2e-mcp or the operator log holds the MCP server keymcp-hash-mismatch... with the key in ai-key-mcp-1: key hash no matchmcp-tools-missingGammathe LLM did not get exactly the tools ...mcp-no-rotation... with the key in ai-key-mcp-2: key hash no matchmcp-meta-resenda metadata-only Secret change wrote to Codermcp-wrong-reasonCredentialMalformedinstead ofCredentialDestinationNotAllowedMCP server e2e-mcp-origin stalled with ... want Ready=False CredentialDestinationNotAllowedmcp-url-earlyCoder config ... changed before the Secret allowed the new originmcp-acl-opena tester chat that selects e2e-mcp with access.groups [] answered HTTP 201, want 400mcp-kill-undonethe controller undid the disable of MCP server e2e-mcpmcp-delete-keptMCP server config e2e-mcp ... still exists in Coder after its object was deletedmcp-ns-stucktimed out ... waiting for: test, provider, control plane, and namespace coder deletedThe existing rows now include the MCP mutations in their expected mutation order, the receipt counts 48 passed cases, and the argv and output checks also cover the tester's session token.
Validation
bash ./hack/e2e-workspace-lifecycle_test.sh: ALL OFFLINE TESTS PASSED, also with jq 1.7.1, the jq of the CI runner (jq 1.8 accepted a manifest expression that 1.7 refuses, review round 1). Each scenario is now capped at 180 s instead of 60 s, because the longer lifecycle passed the old cap on a loaded host.tester-existsscenario checks that a reused tester gets the run's password before the login. Without the reset it fails (mutation checked).shellcheckon both scripts: clean.make verify-vendor,make build,make lint,GOFLAGS=-mod=vendor go tool govulncheck ./...,make test,make test-integration,go test -race ./internal/controller/, andmake docs-checkpass.make codegen,make manifests,make build-installerandmake docs-referenceleave no diff.hack/e2e-workspace-lifecycle*.sh.Size
396 changed lines, all hand-written: 238 in the driver and 158 in the offline tests.
Refs #155
Generated with
xum• Model:anthropic:claude-opus-5-5• Thinking:high• Cost:$232.23