Skip to content

Add 1ES lane for building Copilot CLI with latest MXC - #1116

Draft
Huzaifa Danish (huzaifa-d) wants to merge 15 commits into
mainfrom
user/modanish/cli-mxc-1es-workflow
Draft

Huzaifa Danish (huzaifa-d) wants to merge 15 commits into
mainfrom
user/modanish/cli-mxc-1es-workflow

Conversation

@huzaifa-d

@huzaifa-d Huzaifa Danish (huzaifa-d) commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

📖 Description

Adds a manual-only copilot-cli-build validation plan that runs on 1es-mxc-windows-prerelease-t1-x64 and:

  • checks out the latest microsoft/mxc@main and private github/copilot-agent-runtime@main;
  • uses the copilot environment's GHCP_CLI_SOURCE_READ secret only for private source checkout;
  • rewrites the disposable CLI checkout to use the local MXC mxc-sdk;
  • builds the runtime and required CLI native addon using the CLI-pinned Rust toolchain;
  • stages a job-local copilot-mxc-test command;
  • verifies Cargo provenance and source/bundled runtime hash equality;
  • uploads only a sanitized provenance manifest and summary, never private source or binaries.

This first lane proves private checkout, combined compilation, staging, and provenance. It intentionally does not authenticate to Copilot or run sandbox capability scenarios yet.

🔗 References

🔍 Validation

  • pwsh -NoProfile -File scripts/ci/test-copilot-cli-mxc-build.ps1 — 14 contract tests pass.
  • PowerShell parser checks pass for the module, orchestrator, and test harness.
  • Both changed workflow YAML files parse successfully.
  • git diff --check passes.
  • The initial 1ES workflow dispatch is queued on the selected hosted pool.

✅ Checklist

  • Signed the Contributor License Agreement
  • Linked to an issue
  • Updated documentation (if applicable)
  • Updated Copilot instructions (if build, architecture, or conventions changed)
  • If this PR changes Cargo.lock, the dependency-feed-check check passes (not applicable; Cargo.lock is unchanged)

📋 Issue Type

  • Bug fix
  • Feature
  • Task

GitHub Actions runs the PR validation build automatically. The ADO pipeline
(MXC-PR-Build) is the Azure version of the PR pipeline, kept in parity with the GitHub
Actions build; it runs on merge to main, and Microsoft reviewers with write access can trigger it
on a PR with /azp run. See docs/pull-requests.md.

If the dependency-feed-check check fails on a new dependency, the crate must be added to
the feed before the PR can pass. See docs/pull-requests.md
for the steps.

Microsoft Reviewers: Open in CodeFlow

huzaifa-msft and others added 2 commits September 4, 2026 15:49
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings September 4, 2026 23:01
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@microsoft-github-policy-service microsoft-github-policy-service Bot added the Copilot-Instructions PR modifies Copilot instruction files (.github/copilot-instructions.md or .github/instructions/) label Sep 4, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Private-source logging, fail-open manifest sanitization, and unwired contract tests must be addressed.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review tier: Balanced
Findings: 2 High severity · 1 Medium severity

New issues introduced by this change (3)
Severity Finding
High severity .github/​workflows/​Validation.CopilotCli.Mxc.Job.yml — This directly streams pnpm, Cargo, rustc, and bundler diagnostics from the private checkout into…
High severity scripts/​ci/​CopilotCliMxcBuild.psm1 — Validate the manifest before writing it. This branch suppresses deletion failures and then throws,…
Medium severity scripts/​ci/​test-copilot-cli-mxc-build.ps1 — This new contract suite is not invoked by any workflow, so future changes to the helper or…
What changed in this PR

Adds a manual 1ES lane that builds the latest Copilot CLI against MXC main and records provenance.

Changes:

  • Adds reusable and dispatch workflows for the build lane.
  • Adds build helpers, orchestration, and contract tests.
  • Documents the lane and its security boundaries.
File Description
.github/​workflows/​Validation.Tests.Scheduled.yml Adds the manual plan.
.github/​workflows/​Validation.CopilotCli.Mxc.Job.yml Defines checkout, build, evidence, and cleanup.
scripts/​ci/​CopilotCliMxcBuild.psm1 Implements build and provenance helpers.
scripts/​ci/​build-copilot-cli-with-mxc.ps1 Orchestrates compilation and staging.
scripts/​ci/​test-copilot-cli-mxc-build.ps1 Adds contract tests.
docs/​ci-validation-infrastructure.md Documents the lane.
.github/​copilot-instructions.md Records the new CI architecture.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread .github/workflows/Validation.CopilotCli.Mxc.Job.yml Outdated
Comment on lines +378 to +398
$json = $manifest | ConvertTo-Json -Depth 4
Set-Content -Path $ManifestPath -Value $json -Encoding utf8NoBOM
Write-Host "Manifest written to $ManifestPath"

# Post-write sanitization check — reject if the file contains tokens,
# private checkout paths, or environment secret names.
$content = Get-Content $ManifestPath -Raw
$suspiciousPatterns = @(
'ghp_' # GitHub PAT prefix
'gho_' # GitHub OAuth prefix
'github_pat_' # Fine-grained PAT prefix
'GHCP_CLI_' # Environment secret name fragment
'\\source\\cli' # Private CLI absolute path (backslash)
'/source/cli' # Private CLI absolute path (forward slash)
)
foreach ($pattern in $suspiciousPatterns) {
if ($content -match [regex]::Escape($pattern)) {
Remove-Item $ManifestPath -Force -ErrorAction SilentlyContinue
throw "Manifest contains suspicious content matching '$pattern'. File removed."
}
}
@@ -0,0 +1,366 @@
#Requires -Version 7.0
huzaifa-msft and others added 2 commits September 8, 2026 09:45
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Private-output retention concerns remain unresolved, and toolchain and MSVC discovery can select or reject incorrect installations.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review tier: Balanced
Findings: 2 High severity · 3 Medium severity

New issues introduced by this change (2)
Severity Finding
Medium severity scripts/​ci/​CopilotCliMxcBuild.psm1vswhere -latest uses its default product filter, which can omit a standalone Visual Studio Build…
Medium severity scripts/​ci/​build-copilot-cli-with-mxc.ps1 — This unanchored search can select a commented or unrelated channel = ... occurrence before the…
Pre-existing issues (3)
Severity Finding
High severity scripts/​ci/​CopilotCliMxcBuild.psm1 — Validate the manifest before writing it. This branch suppresses deletion failures and then throws,… View comment
High severity .github/​workflows/​Validation.CopilotCli.Mxc.Job.yml — This directly streams pnpm, Cargo, rustc, and bundler diagnostics from the private checkout into… View comment
Medium severity scripts/​ci/​test-copilot-cli-mxc-build.ps1 — This new contract suite is not invoked by any workflow, so future changes to the helper or… View comment

Comment thread scripts/ci/CopilotCliMxcBuild.psm1 Outdated
throw "rust-toolchain.toml not found at $toolchainFile"
}
$toolchainContent = Get-Content $toolchainFile -Raw
$channelMatch = [regex]::Match($toolchainContent, 'channel\s*=\s*"([^"]+)"')
huzaifa-msft and others added 4 commits September 8, 2026 10:02
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 8, 2026 17:16

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Manifest sanitization can fail open during artifact upload, and the claimed direct-dispatch validation cannot run before the workflow reaches the default branch.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review tier: Balanced
Findings: 2 High severity · 4 Medium severity

New issues introduced by this change (1)
Severity Finding
Medium severity docs/​superpowers/​plans/​2026-09-08-cli-mxc-direct-dispatch.md — This command cannot validate this new workflow_dispatch workflow before merge: GitHub only…
Pre-existing issues (5)
Severity Finding
High severity scripts/​ci/​CopilotCliMxcBuild.psm1 — Validate the manifest before writing it. This branch suppresses deletion failures and then throws,… View comment
High severity .github/​workflows/​Validation.CopilotCli.Mxc.Job.yml — This directly streams pnpm, Cargo, rustc, and bundler diagnostics from the private checkout into… View comment
Medium severity scripts/​ci/​build-copilot-cli-with-mxc.ps1 — This unanchored search can select a commented or unrelated channel = ... occurrence before the… View comment
Medium severity scripts/​ci/​CopilotCliMxcBuild.psm1vswhere -latest uses its default product filter, which can omit a standalone Visual Studio Build… View comment
Medium severity scripts/​ci/​test-copilot-cli-mxc-build.ps1 — This new contract suite is not invoked by any workflow, so future changes to the helper or… View comment
Suppressed comments (1)

scripts/ci/CopilotCliMxcBuild.psm1:380

  • The manifest is written before sanitization, and failure to delete a rejected file is ignored. Because the workflow uploads this path under if: always(), a transient lock or restrictive ACL can leave suspicious content in place and publish it. Validate $json before creating the file so rejected content can never reach the artifact path.
    $json = $manifest | ConvertTo-Json -Depth 4
    Set-Content -Path $ManifestPath -Value $json -Encoding utf8NoBOM
    Write-Host "Manifest written to $ManifestPath"

Comment thread docs/superpowers/plans/2026-09-08-cli-mxc-direct-dispatch.md Outdated
huzaifa-msft and others added 3 commits September 8, 2026 10:24
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The dependency rewrite can discard SDK feature flags and silently produce a functionally different runtime.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review tier: Balanced
Findings: 1 High severity · 3 Medium severity · 1 Low severity

New issues introduced by this change (1)
Severity Finding
Low severity docs/​superpowers/​plans/​2026-09-08-cli-mxc-direct-dispatch.md — This snippet has a closing fence but no opening fence, so the remainder of the implementation plan…
Pre-existing issues (4)
Severity Finding
High severity scripts/​ci/​CopilotCliMxcBuild.psm1 — Validate the manifest before writing it. This branch suppresses deletion failures and then throws,… View comment
Medium severity scripts/​ci/​build-copilot-cli-with-mxc.ps1 — This unanchored search can select a commented or unrelated channel = ... occurrence before the… View comment
Medium severity scripts/​ci/​CopilotCliMxcBuild.psm1vswhere -latest uses its default product filter, which can omit a standalone Visual Studio Build… View comment
Medium severity scripts/​ci/​test-copilot-cli-mxc-build.ps1 — This new contract suite is not invoked by any workflow, so future changes to the helper or… View comment
Issues resolved since last review (2)
Severity Finding
Medium severity docs/​superpowers/​plans/​2026-09-08-cli-mxc-direct-dispatch.md — This command cannot validate this new workflow_dispatch workflow before merge: GitHub only… View resolved comment
High severity .github/​workflows/​Validation.CopilotCli.Mxc.Job.yml — This directly streams pnpm, Cargo, rustc, and bundler diagnostics from the private checkout into… View resolved comment
Suppressed comments (2)

Previously missed (1) — in code that hasn't changed since the last review.

scripts/ci/CopilotCliMxcBuild.psm1:229

  • Replacing the complete dependency entry drops any existing features or default-features settings along with the registry source. Since mxc-sdk has feature-gated wslc and isolation_session support, provenance can pass while the resulting CLI runtime silently loses requested functionality. Preserve non-source dependency options while replacing only the registry/version source with path, and add a contract case that includes feature flags.

scripts/ci/CopilotCliMxcBuild.psm1:140

  • This selects the latest Visual Studio product without requiring the x64 C++ toolset. On a runner with multiple VS products, it can choose one without MSVC and then fail the cl.exe check even though a usable Build Tools/VS installation exists. Query the required component (as scripts/generate-wslc-bindings.ps1:83 already does).
    $installPath = & $vswhere -latest -property installationPath 2>&1

Comment on lines +47 to +51
copilot-cli-build:
if: github.event_name == 'workflow_dispatch' && inputs.plan == 'copilot-cli-build'
environment: copilot
runs-on: 1es-mxc-windows-prerelease-t1-x64
```
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 8, 2026 17:34

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The Visual Studio lookup excludes the Build Tools SKU installed by the workflow, preventing compilation on the clean T1 image.

Review tier: Balanced
Findings: 1 High severity · 3 Medium severity · 1 Low severity

Pre-existing issues (5)
Severity Finding
High severity scripts/​ci/​CopilotCliMxcBuild.psm1 — Validate the manifest before writing it. This branch suppresses deletion failures and then throws,… View comment
Medium severity scripts/​ci/​build-copilot-cli-with-mxc.ps1 — This unanchored search can select a commented or unrelated channel = ... occurrence before the… View comment
Medium severity scripts/​ci/​CopilotCliMxcBuild.psm1vswhere -latest uses its default product filter, which can omit a standalone Visual Studio Build… View comment
Medium severity scripts/​ci/​test-copilot-cli-mxc-build.ps1 — This new contract suite is not invoked by any workflow, so future changes to the helper or… View comment
Low severity docs/​superpowers/​plans/​2026-09-08-cli-mxc-direct-dispatch.md — This snippet has a closing fence but no opening fence, so the remainder of the implementation plan… View comment
Suppressed comments (1)

scripts/ci/CopilotCliMxcBuild.psm1:140

  • vswhere excludes the Build Tools SKU from its default product set. This lane installs only Visual Studio Build Tools, so on the documented clean T1 image this lookup returns no installation and the next check throws before compilation. Include all products (and preferably require the VC tools component) when resolving the installation.
    $installPath = & $vswhere -latest -property installationPath 2>&1

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The Visual Studio lookup cannot discover the standalone Build Tools SKU installed by the workflow.

Review tier: Balanced
Findings: 1 High severity · 3 Medium severity · 1 Low severity

Pre-existing issues (5)
Severity Finding
High severity scripts/​ci/​CopilotCliMxcBuild.psm1 — Validate the manifest before writing it. This branch suppresses deletion failures and then throws,… View comment
Medium severity scripts/​ci/​build-copilot-cli-with-mxc.ps1 — This unanchored search can select a commented or unrelated channel = ... occurrence before the… View comment
Medium severity scripts/​ci/​CopilotCliMxcBuild.psm1vswhere -latest uses its default product filter, which can omit a standalone Visual Studio Build… View comment
Medium severity scripts/​ci/​test-copilot-cli-mxc-build.ps1 — This new contract suite is not invoked by any workflow, so future changes to the helper or… View comment
Low severity docs/​superpowers/​plans/​2026-09-08-cli-mxc-direct-dispatch.md — This snippet has a closing fence but no opening fence, so the remainder of the implementation plan… View comment
Suppressed comments (1)

scripts/ci/CopilotCliMxcBuild.psm1:140

  • The workflow installs the standalone Visual Studio Build Tools SKU, but vswhere does not include that SKU in its default product set. On the documented T1 image (which has no IDE install), this therefore reports no Visual Studio installation and prevents the lane from reaching compilation. Query all products and require the x64 VC toolset, matching the established lookup in scripts/generate-wslc-bindings.ps1:83.
    $installPath = & $vswhere -latest -property installationPath 2>&1

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Rejected manifest content may still be uploaded when its best-effort deletion fails.

Review tier: Balanced
Findings: 1 High severity · 2 Medium severity · 1 Low severity

Pre-existing issues (4)
Severity Finding
High severity scripts/​ci/​CopilotCliMxcBuild.psm1 — Validate the manifest before writing it. This branch suppresses deletion failures and then throws,… View comment
Medium severity scripts/​ci/​build-copilot-cli-with-mxc.ps1 — This unanchored search can select a commented or unrelated channel = ... occurrence before the… View comment
Medium severity scripts/​ci/​test-copilot-cli-mxc-build.ps1 — This new contract suite is not invoked by any workflow, so future changes to the helper or… View comment
Low severity docs/​superpowers/​plans/​2026-09-08-cli-mxc-direct-dispatch.md — This snippet has a closing fence but no opening fence, so the remainder of the implementation plan… View comment
Issues resolved since last review (1)
Severity Finding
Medium severity scripts/​ci/​CopilotCliMxcBuild.psm1vswhere -latest uses its default product filter, which can omit a standalone Visual Studio Build… View resolved comment
Suppressed comments (2)

Previously missed (1) — in code that hasn't changed since the last review.

docs/superpowers/specs/2026-09-08-cli-mxc-direct-dispatch-design.md:7

  • The PR description's Validation section links run 33927108140 as the “initial 1ES validation run” and says it is queued, but that run completed as cancelled after never receiving a runner, which is consistent with this paragraph. Update the PR description with the terminal result (and link the latest terminal proof once available) so the claimed validation is accurate.

scripts/ci/CopilotCliMxcBuild.psm1:384

  • Validate the serialized JSON before writing it. As written, rejection deletes the file with -ErrorAction SilentlyContinue; if that deletion fails, the workflow's always() artifact step can upload the still-present manifest containing the suspicious value. Scan $json first and only create the file after it passes.
    $json = $manifest | ConvertTo-Json -Depth 4
    Set-Content -Path $ManifestPath -Value $json -Encoding utf8NoBOM
    Write-Host "Manifest written to $ManifestPath"

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The executable download and retained-version sanitization have security gaps, and terminal validation remains outstanding.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review tier: Balanced
Findings: 2 High severity · 3 Medium severity · 1 Low severity

New issues introduced by this change (2)
Severity Finding
High severity .github/​workflows/​Validation.Tests.Scheduled.yml — Authenticate this download before executing it. This mutable network response is run without a…
Medium severity scripts/​ci/​CopilotCliMxcBuild.psm1 — This check accepts any single-line payload after the prefix, and that value is copied into both…
Pre-existing issues (4)
Severity Finding
High severity scripts/​ci/​CopilotCliMxcBuild.psm1 — Validate the manifest before writing it. This branch suppresses deletion failures and then throws,… View comment
Medium severity scripts/​ci/​build-copilot-cli-with-mxc.ps1 — This unanchored search can select a commented or unrelated channel = ... occurrence before the… View comment
Medium severity scripts/​ci/​test-copilot-cli-mxc-build.ps1 — This new contract suite is not invoked by any workflow, so future changes to the helper or… View comment
Low severity docs/​superpowers/​plans/​2026-09-08-cli-mxc-direct-dispatch.md — This snippet has a closing fence but no opening fence, so the remainder of the implementation plan… View comment
Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

docs/superpowers/plans/2026-09-08-cli-mxc-direct-dispatch.md:178

  • This required validation step is still outstanding: the PR description cites run 33927108140 as queued, but that run is now cancelled, and the latest run for the current implementation is not terminal. Update the validation section with a terminal direct-dispatch run and its actual compilation/provenance result before merge.
    docs/superpowers/plans/2026-09-08-cli-mxc-direct-dispatch.md:123
  • The successful harness now increments $passed 17 times (11 assertion-helper calls and 6 direct checks), so this documented result is stale.

Comment on lines +110 to +114
Invoke-WebRequest `
-Uri 'https://win.rustup.rs/x86_64' `
-OutFile $rustupInit

& $rustupInit `
if ($MxcSdkManifest -ne 'src/core/mxc-sdk/Cargo.toml') {
throw "mxcSdkManifest must be the repository-relative path 'src/core/mxc-sdk/Cargo.toml'."
}
if ($CliVersion -notmatch '^GitHub Copilot CLI [^\\/\r\n]+$') {

This branch was successfully deployed

1 active deployment
copilot 242e5a61 Deployed Sep 8, 2026 by huzaifa-d via Copilot CLI + latest MXC #29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Copilot-Instructions PR modifies Copilot instruction files (.github/copilot-instructions.md or .github/instructions/)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants