Add 1ES lane for building Copilot CLI with latest MXC - #1116
Huzaifa Danish (huzaifa-d) wants to merge 15 commits into
Conversation
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Private-source logging, fail-open manifest sanitization, and unwired contract tests must be addressed.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review tier: Balanced
Findings: 2
New issues introduced by this change (3)
| Severity | Finding |
|---|---|
.github/workflows/Validation.CopilotCli.Mxc.Job.yml — This directly streams pnpm, Cargo, rustc, and bundler diagnostics from the private checkout into… |
|
scripts/ci/CopilotCliMxcBuild.psm1 — Validate the manifest before writing it. This branch suppresses deletion failures and then throws,… |
|
scripts/ci/test-copilot-cli-mxc-build.ps1 — This new contract suite is not invoked by any workflow, so future changes to the helper or… |
What changed in this PR
Adds a manual 1ES lane that builds the latest Copilot CLI against MXC main and records provenance.
Changes:
- Adds reusable and dispatch workflows for the build lane.
- Adds build helpers, orchestration, and contract tests.
- Documents the lane and its security boundaries.
| File | Description |
|---|---|
.github/workflows/Validation.Tests.Scheduled.yml |
Adds the manual plan. |
.github/workflows/Validation.CopilotCli.Mxc.Job.yml |
Defines checkout, build, evidence, and cleanup. |
scripts/ci/CopilotCliMxcBuild.psm1 |
Implements build and provenance helpers. |
scripts/ci/build-copilot-cli-with-mxc.ps1 |
Orchestrates compilation and staging. |
scripts/ci/test-copilot-cli-mxc-build.ps1 |
Adds contract tests. |
docs/ci-validation-infrastructure.md |
Documents the lane. |
.github/copilot-instructions.md |
Records the new CI architecture. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| $json = $manifest | ConvertTo-Json -Depth 4 | ||
| Set-Content -Path $ManifestPath -Value $json -Encoding utf8NoBOM | ||
| Write-Host "Manifest written to $ManifestPath" | ||
|
|
||
| # Post-write sanitization check — reject if the file contains tokens, | ||
| # private checkout paths, or environment secret names. | ||
| $content = Get-Content $ManifestPath -Raw | ||
| $suspiciousPatterns = @( | ||
| 'ghp_' # GitHub PAT prefix | ||
| 'gho_' # GitHub OAuth prefix | ||
| 'github_pat_' # Fine-grained PAT prefix | ||
| 'GHCP_CLI_' # Environment secret name fragment | ||
| '\\source\\cli' # Private CLI absolute path (backslash) | ||
| '/source/cli' # Private CLI absolute path (forward slash) | ||
| ) | ||
| foreach ($pattern in $suspiciousPatterns) { | ||
| if ($content -match [regex]::Escape($pattern)) { | ||
| Remove-Item $ManifestPath -Force -ErrorAction SilentlyContinue | ||
| throw "Manifest contains suspicious content matching '$pattern'. File removed." | ||
| } | ||
| } |
| @@ -0,0 +1,366 @@ | |||
| #Requires -Version 7.0 | |||
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Private-output retention concerns remain unresolved, and toolchain and MSVC discovery can select or reject incorrect installations.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review tier: Balanced
Findings: 2
New issues introduced by this change (2)
| Severity | Finding |
|---|---|
scripts/ci/CopilotCliMxcBuild.psm1 — vswhere -latest uses its default product filter, which can omit a standalone Visual Studio Build… |
|
scripts/ci/build-copilot-cli-with-mxc.ps1 — This unanchored search can select a commented or unrelated channel = ... occurrence before the… |
Pre-existing issues (3)
| Severity | Finding |
|---|---|
scripts/ci/CopilotCliMxcBuild.psm1 — Validate the manifest before writing it. This branch suppresses deletion failures and then throws,… View comment |
|
.github/workflows/Validation.CopilotCli.Mxc.Job.yml — This directly streams pnpm, Cargo, rustc, and bundler diagnostics from the private checkout into… View comment |
|
scripts/ci/test-copilot-cli-mxc-build.ps1 — This new contract suite is not invoked by any workflow, so future changes to the helper or… View comment |
| throw "rust-toolchain.toml not found at $toolchainFile" | ||
| } | ||
| $toolchainContent = Get-Content $toolchainFile -Raw | ||
| $channelMatch = [regex]::Match($toolchainContent, 'channel\s*=\s*"([^"]+)"') |
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Manifest sanitization can fail open during artifact upload, and the claimed direct-dispatch validation cannot run before the workflow reaches the default branch.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review tier: Balanced
Findings: 2
New issues introduced by this change (1)
| Severity | Finding |
|---|---|
docs/superpowers/plans/2026-09-08-cli-mxc-direct-dispatch.md — This command cannot validate this new workflow_dispatch workflow before merge: GitHub only… |
Pre-existing issues (5)
| Severity | Finding |
|---|---|
scripts/ci/CopilotCliMxcBuild.psm1 — Validate the manifest before writing it. This branch suppresses deletion failures and then throws,… View comment |
|
.github/workflows/Validation.CopilotCli.Mxc.Job.yml — This directly streams pnpm, Cargo, rustc, and bundler diagnostics from the private checkout into… View comment |
|
scripts/ci/build-copilot-cli-with-mxc.ps1 — This unanchored search can select a commented or unrelated channel = ... occurrence before the… View comment |
|
scripts/ci/CopilotCliMxcBuild.psm1 — vswhere -latest uses its default product filter, which can omit a standalone Visual Studio Build… View comment |
|
scripts/ci/test-copilot-cli-mxc-build.ps1 — This new contract suite is not invoked by any workflow, so future changes to the helper or… View comment |
Suppressed comments (1)
scripts/ci/CopilotCliMxcBuild.psm1:380
- The manifest is written before sanitization, and failure to delete a rejected file is ignored. Because the workflow uploads this path under
if: always(), a transient lock or restrictive ACL can leave suspicious content in place and publish it. Validate$jsonbefore creating the file so rejected content can never reach the artifact path.
$json = $manifest | ConvertTo-Json -Depth 4
Set-Content -Path $ManifestPath -Value $json -Encoding utf8NoBOM
Write-Host "Manifest written to $ManifestPath"
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
The dependency rewrite can discard SDK feature flags and silently produce a functionally different runtime.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review tier: Balanced
Findings: 1
New issues introduced by this change (1)
| Severity | Finding |
|---|---|
docs/superpowers/plans/2026-09-08-cli-mxc-direct-dispatch.md — This snippet has a closing fence but no opening fence, so the remainder of the implementation plan… |
Pre-existing issues (4)
| Severity | Finding |
|---|---|
scripts/ci/CopilotCliMxcBuild.psm1 — Validate the manifest before writing it. This branch suppresses deletion failures and then throws,… View comment |
|
scripts/ci/build-copilot-cli-with-mxc.ps1 — This unanchored search can select a commented or unrelated channel = ... occurrence before the… View comment |
|
scripts/ci/CopilotCliMxcBuild.psm1 — vswhere -latest uses its default product filter, which can omit a standalone Visual Studio Build… View comment |
|
scripts/ci/test-copilot-cli-mxc-build.ps1 — This new contract suite is not invoked by any workflow, so future changes to the helper or… View comment |
Issues resolved since last review (2)
| Severity | Finding |
|---|---|
docs/superpowers/plans/2026-09-08-cli-mxc-direct-dispatch.md — This command cannot validate this new workflow_dispatch workflow before merge: GitHub only… View resolved comment |
|
.github/workflows/Validation.CopilotCli.Mxc.Job.yml — This directly streams pnpm, Cargo, rustc, and bundler diagnostics from the private checkout into… View resolved comment |
Suppressed comments (2)
Previously missed (1) — in code that hasn't changed since the last review.
scripts/ci/CopilotCliMxcBuild.psm1:229
- Replacing the complete dependency entry drops any existing
featuresordefault-featuressettings along with the registry source. Sincemxc-sdkhas feature-gatedwslcandisolation_sessionsupport, provenance can pass while the resulting CLI runtime silently loses requested functionality. Preserve non-source dependency options while replacing only the registry/version source withpath, and add a contract case that includes feature flags.
scripts/ci/CopilotCliMxcBuild.psm1:140
- This selects the latest Visual Studio product without requiring the x64 C++ toolset. On a runner with multiple VS products, it can choose one without MSVC and then fail the
cl.execheck even though a usable Build Tools/VS installation exists. Query the required component (asscripts/generate-wslc-bindings.ps1:83already does).
$installPath = & $vswhere -latest -property installationPath 2>&1
| copilot-cli-build: | ||
| if: github.event_name == 'workflow_dispatch' && inputs.plan == 'copilot-cli-build' | ||
| environment: copilot | ||
| runs-on: 1es-mxc-windows-prerelease-t1-x64 | ||
| ``` |
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Copilot review overview
🔵 Needs a closer look
The Visual Studio lookup excludes the Build Tools SKU installed by the workflow, preventing compilation on the clean T1 image.
Review tier: Balanced
Findings: 1
Pre-existing issues (5)
| Severity | Finding |
|---|---|
scripts/ci/CopilotCliMxcBuild.psm1 — Validate the manifest before writing it. This branch suppresses deletion failures and then throws,… View comment |
|
scripts/ci/build-copilot-cli-with-mxc.ps1 — This unanchored search can select a commented or unrelated channel = ... occurrence before the… View comment |
|
scripts/ci/CopilotCliMxcBuild.psm1 — vswhere -latest uses its default product filter, which can omit a standalone Visual Studio Build… View comment |
|
scripts/ci/test-copilot-cli-mxc-build.ps1 — This new contract suite is not invoked by any workflow, so future changes to the helper or… View comment |
|
docs/superpowers/plans/2026-09-08-cli-mxc-direct-dispatch.md — This snippet has a closing fence but no opening fence, so the remainder of the implementation plan… View comment |
Suppressed comments (1)
scripts/ci/CopilotCliMxcBuild.psm1:140
vswhereexcludes the Build Tools SKU from its default product set. This lane installs only Visual Studio Build Tools, so on the documented clean T1 image this lookup returns no installation and the next check throws before compilation. Include all products (and preferably require the VC tools component) when resolving the installation.
$installPath = & $vswhere -latest -property installationPath 2>&1
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Copilot review overview
🔵 Needs a closer look
The Visual Studio lookup cannot discover the standalone Build Tools SKU installed by the workflow.
Review tier: Balanced
Findings: 1
Pre-existing issues (5)
| Severity | Finding |
|---|---|
scripts/ci/CopilotCliMxcBuild.psm1 — Validate the manifest before writing it. This branch suppresses deletion failures and then throws,… View comment |
|
scripts/ci/build-copilot-cli-with-mxc.ps1 — This unanchored search can select a commented or unrelated channel = ... occurrence before the… View comment |
|
scripts/ci/CopilotCliMxcBuild.psm1 — vswhere -latest uses its default product filter, which can omit a standalone Visual Studio Build… View comment |
|
scripts/ci/test-copilot-cli-mxc-build.ps1 — This new contract suite is not invoked by any workflow, so future changes to the helper or… View comment |
|
docs/superpowers/plans/2026-09-08-cli-mxc-direct-dispatch.md — This snippet has a closing fence but no opening fence, so the remainder of the implementation plan… View comment |
Suppressed comments (1)
scripts/ci/CopilotCliMxcBuild.psm1:140
- The workflow installs the standalone Visual Studio Build Tools SKU, but
vswheredoes not include that SKU in its default product set. On the documented T1 image (which has no IDE install), this therefore reports no Visual Studio installation and prevents the lane from reaching compilation. Query all products and require the x64 VC toolset, matching the established lookup inscripts/generate-wslc-bindings.ps1:83.
$installPath = & $vswhere -latest -property installationPath 2>&1
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Copilot review overview
🔵 Needs a closer look
Rejected manifest content may still be uploaded when its best-effort deletion fails.
Review tier: Balanced
Findings: 1
Pre-existing issues (4)
| Severity | Finding |
|---|---|
scripts/ci/CopilotCliMxcBuild.psm1 — Validate the manifest before writing it. This branch suppresses deletion failures and then throws,… View comment |
|
scripts/ci/build-copilot-cli-with-mxc.ps1 — This unanchored search can select a commented or unrelated channel = ... occurrence before the… View comment |
|
scripts/ci/test-copilot-cli-mxc-build.ps1 — This new contract suite is not invoked by any workflow, so future changes to the helper or… View comment |
|
docs/superpowers/plans/2026-09-08-cli-mxc-direct-dispatch.md — This snippet has a closing fence but no opening fence, so the remainder of the implementation plan… View comment |
Issues resolved since last review (1)
| Severity | Finding |
|---|---|
scripts/ci/CopilotCliMxcBuild.psm1 — vswhere -latest uses its default product filter, which can omit a standalone Visual Studio Build… View resolved comment |
Suppressed comments (2)
Previously missed (1) — in code that hasn't changed since the last review.
docs/superpowers/specs/2026-09-08-cli-mxc-direct-dispatch-design.md:7
- The PR description's Validation section links run 33927108140 as the “initial 1ES validation run” and says it is queued, but that run completed as cancelled after never receiving a runner, which is consistent with this paragraph. Update the PR description with the terminal result (and link the latest terminal proof once available) so the claimed validation is accurate.
scripts/ci/CopilotCliMxcBuild.psm1:384
- Validate the serialized JSON before writing it. As written, rejection deletes the file with
-ErrorAction SilentlyContinue; if that deletion fails, the workflow'salways()artifact step can upload the still-present manifest containing the suspicious value. Scan$jsonfirst and only create the file after it passes.
$json = $manifest | ConvertTo-Json -Depth 4
Set-Content -Path $ManifestPath -Value $json -Encoding utf8NoBOM
Write-Host "Manifest written to $ManifestPath"
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
The executable download and retained-version sanitization have security gaps, and terminal validation remains outstanding.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review tier: Balanced
Findings: 2
New issues introduced by this change (2)
| Severity | Finding |
|---|---|
.github/workflows/Validation.Tests.Scheduled.yml — Authenticate this download before executing it. This mutable network response is run without a… |
|
scripts/ci/CopilotCliMxcBuild.psm1 — This check accepts any single-line payload after the prefix, and that value is copied into both… |
Pre-existing issues (4)
| Severity | Finding |
|---|---|
scripts/ci/CopilotCliMxcBuild.psm1 — Validate the manifest before writing it. This branch suppresses deletion failures and then throws,… View comment |
|
scripts/ci/build-copilot-cli-with-mxc.ps1 — This unanchored search can select a commented or unrelated channel = ... occurrence before the… View comment |
|
scripts/ci/test-copilot-cli-mxc-build.ps1 — This new contract suite is not invoked by any workflow, so future changes to the helper or… View comment |
|
docs/superpowers/plans/2026-09-08-cli-mxc-direct-dispatch.md — This snippet has a closing fence but no opening fence, so the remainder of the implementation plan… View comment |
Suppressed comments (2)
Previously missed (2) — in code that hasn't changed since the last review.
docs/superpowers/plans/2026-09-08-cli-mxc-direct-dispatch.md:178
- This required validation step is still outstanding: the PR description cites run 33927108140 as queued, but that run is now cancelled, and the latest run for the current implementation is not terminal. Update the validation section with a terminal direct-dispatch run and its actual compilation/provenance result before merge.
docs/superpowers/plans/2026-09-08-cli-mxc-direct-dispatch.md:123 - The successful harness now increments
$passed17 times (11 assertion-helper calls and 6 direct checks), so this documented result is stale.
| Invoke-WebRequest ` | ||
| -Uri 'https://win.rustup.rs/x86_64' ` | ||
| -OutFile $rustupInit | ||
|
|
||
| & $rustupInit ` |
| if ($MxcSdkManifest -ne 'src/core/mxc-sdk/Cargo.toml') { | ||
| throw "mxcSdkManifest must be the repository-relative path 'src/core/mxc-sdk/Cargo.toml'." | ||
| } | ||
| if ($CliVersion -notmatch '^GitHub Copilot CLI [^\\/\r\n]+$') { |



📖 Description
Adds a manual-only
copilot-cli-buildvalidation plan that runs on1es-mxc-windows-prerelease-t1-x64and:microsoft/mxc@mainand privategithub/copilot-agent-runtime@main;copilotenvironment'sGHCP_CLI_SOURCE_READsecret only for private source checkout;mxc-sdk;copilot-mxc-testcommand;This first lane proves private checkout, combined compilation, staging, and provenance. It intentionally does not authenticate to Copilot or run sandbox capability scenarios yet.
🔗 References
🔍 Validation
pwsh -NoProfile -File scripts/ci/test-copilot-cli-mxc-build.ps1— 14 contract tests pass.git diff --checkpasses.✅ Checklist
Cargo.lock, thedependency-feed-checkcheck passes (not applicable;Cargo.lockis unchanged)📋 Issue Type
GitHub Actions runs the PR validation build automatically. The ADO pipeline
(
MXC-PR-Build) is the Azure version of the PR pipeline, kept in parity with the GitHubActions build; it runs on merge to
main, and Microsoft reviewers with write access can trigger iton a PR with
/azp run. See docs/pull-requests.md.If the
dependency-feed-checkcheck fails on a new dependency, the crate must be added tothe feed before the PR can pass. See docs/pull-requests.md
for the steps.
Microsoft Reviewers: Open in CodeFlow