From 77e7c72a90c091eeaea2190c3e9efabda0320b8b Mon Sep 17 00:00:00 2001 From: Missy Messa <47990216+missymessa@users.noreply.github.com> Date: Thu, 10 Sep 2026 17:45:18 -0700 Subject: [PATCH 1/3] Use Secret Manager values for GitHub App tokens (#17528) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: cb83d078-4ba7-4f9f-86e5-7840b221515c --- Documentation/OneLocBuild.md | 22 ++--- Documentation/OneLocBuildGitHubApp.md | 51 ++++++------ eng/common/Get-GitHubAppToken.ps1 | 81 ++++++++----------- eng/common/core-templates/job/onelocbuild.yml | 18 ++--- .../steps/get-github-app-token.yml | 49 ++++------- 5 files changed, 95 insertions(+), 126 deletions(-) diff --git a/Documentation/OneLocBuild.md b/Documentation/OneLocBuild.md index 36a21b93296..19a028479ce 100644 --- a/Documentation/OneLocBuild.md +++ b/Documentation/OneLocBuild.md @@ -27,12 +27,12 @@ Onboarding to OneLocBuild is a simple process: parameters: CreatePr: false ``` -Note: If you are running your PR builds and official builds off of the same definition, conditionalize -this step so OneLocBuild runs only in the supported `dnceng/internal` or `DevDiv/DevDiv` project: +Note: If you are running your PR builds and official builds off of the same definition and are on dnceng, +you will want to conditionalize this step with the following: ```yaml -- ${{ if and(or(eq(variables['System.TeamProject'], 'internal'), eq(variables['System.TeamProject'], 'DevDiv')), notin(variables['Build.Reason'], 'PullRequest')) }}: +- ${{ if and(ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}: ``` -This also prevents OneLocBuild from running during pull request validation. +To prevent OneLocBuild from running in the public project where it will fail. 3. Run the pipeline you want to use OneLocBuild on your test branch. 4. Open a ticket with the localization team using @@ -51,7 +51,7 @@ This also prevents OneLocBuild from running during pull request validation. Make sure to remove the `CreatePr: false` line from step 2. Additionally, if you added the YAML condition from step 2, make sure that your new YAML condition now looks like: ```yaml -- ${{ if and(or(eq(variables['System.TeamProject'], 'internal'), eq(variables['System.TeamProject'], 'DevDiv')), notin(variables['Build.Reason'], 'PullRequest'), eq(variables['Build.SourceBranch'], 'refs/heads/main')) }}: +- ${{ if and(ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest'), eq(variables['Build.SourceBranch'], 'refs/heads/main')) }}: ``` 7. If using a mirrored repository (your code is mirrored to a trusted repository which your official build uses), @@ -177,7 +177,7 @@ The most basic structure for calling the OneLocBuild template is: ```yaml jobs: -- ${{ if and(or(eq(variables['System.TeamProject'], 'internal'), eq(variables['System.TeamProject'], 'DevDiv')), notin(variables['Build.Reason'], 'PullRequest'), eq(variables['Build.SourceBranch'], 'refs/heads/main')) }}: +- ${{ if and(ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest'), eq(variables['Build.SourceBranch'], 'refs/heads/main')) }}: - template: /eng/common/templates/job/onelocbuild.yml parameters: LclSource: lclFilesfromPackage @@ -203,13 +203,15 @@ The parameters that can be passed to the template are as follows: | `LclSource` | `LclFilesInRepo` | This passes the `LclSource` input to the OneLocBuild task as described in [its documentation](https://ceapex.visualstudio.com/CEINTL/_wiki/wikis/CEINTL.wiki/107/Localization-with-OneLocBuild-Task?anchor=languageset%2C-languages-(required)). For most repos, this should be set to `LclFilesfromPackage`. | | `LclPackageId` | `''` | When `LclSource` is set to `LclFilesfromPackage`, this passes in the package ID as described in the [OneLocBuild task documentation](https://ceapex.visualstudio.com/CEINTL/_wiki/wikis/CEINTL.wiki/107/Localization-with-OneLocBuild-Task?anchor=scenario-2%3A-lcl-files-from-a-package). | | `CeapexServiceConnection` | `'dnceng-onelocbuild-ceapex'` | The project-scoped WIF service connection used to acquire a short-lived token for the Ceapex feeds. OneLocBuild supports only `dnceng/internal` and `DevDiv/DevDiv`; pipelines must be authorized to use the connection. | -| `GitHubAppServiceConnection` | `'dnceng-oneloc-githubapp'` | The dnceng/internal WIF service connection used to sign the App JWT. When the value remains the default, Arcade automatically uses `devdiv-oneloc-githubapp` in `DevDiv/DevDiv`. | -| `GitHubAppClientId` | `'Iv23lijBU8x3gc9lDOc9'` | The GitHub App's Client ID. | -| `GitHubAppKeyVaultName` | `'EngKeyVault'` | Key Vault holding the App's RSA signing key. | -| `GitHubAppKeyName` | `'oneloc-localization-app-key'` | Name of the App's RSA signing key in the Key Vault. | +| `GitHubAppId` | `$(oneloc-localization-app-app-id)` | Secret Manager-managed GitHub App ID from `OneLocBuildVariables`. | +| `GitHubAppPrivateKey` | `$(oneloc-localization-app-app-private-key)` | Secret Manager-managed PEM private key from `OneLocBuildVariables`. | | `condition` | `''` | Allows for conditionalizing the template's steps on build-time variables. | | `JobNameSuffix` | `''` | Allows for custom job name suffix. This is helpful for disambiguation in case of need for more then one OneLocBuild job run - e.g. as a way to set multiple package IDs. | +The previous Key Vault RSA signing parameters have been removed. See +[Authenticating OneLocBuild's GitHub check-in with the GitHub App](OneLocBuildGitHubApp.md#migrating-from-key-vault-rsa-signing) +for the required parameter migration. + It is recommended that you set `LclSource` and `LclPackageId` as shown in the example above. diff --git a/Documentation/OneLocBuildGitHubApp.md b/Documentation/OneLocBuildGitHubApp.md index 302480e3894..3fbdd99a7eb 100644 --- a/Documentation/OneLocBuildGitHubApp.md +++ b/Documentation/OneLocBuildGitHubApp.md @@ -23,8 +23,8 @@ whenever `RepoType` is `gitHub`. OneLocBuild supports the **`dnceng/internal`** **`DevDiv/DevDiv`** Azure DevOps projects. When those hold, the job runs [`get-github-app-token.yml`](/eng/common/core-templates/steps/get-github-app-token.yml), -which signs a JWT with the App's RSA key in Key Vault, exchanges it for an installation token, and -passes that token to the OneLocBuild task via `gitHubPatVariable`. +which signs a JWT with the Secret Manager-managed App private key, exchanges it for an installation +token, and passes that token to the OneLocBuild task via `gitHubPatVariable`. If App token minting or authentication fails, the job fails; there is no stored-PAT fallback. @@ -35,16 +35,15 @@ If App token minting or authentication fails, the job fails; there is no stored- 1. **The App must be installed on the GitHub org/account that owns your target repo, and your specific repository must be selected in that installation.** The App can only open a PR against a repository it is installed on. This is what actually grants the App permission to your repo. -2. **Your pipeline must run in `dnceng/internal` or `DevDiv/DevDiv` and be authorized to use that - project's App service connection.** +2. **Your pipeline must run in `dnceng/internal` or `DevDiv/DevDiv` and include its project's + Key Vault-backed `OneLocBuildVariables` variable group.** -The .NET Engineering Services team manages the App signing key and the project-scoped service -connections. Contact the First Responders to authorize an intended pipeline. +The .NET Engineering Services team manages the App credentials and variable groups. ### Step 1 — Request that your repository be added to the App installation -The App installation and the backing `dnceng/internal` service connection / Key Vault key are -managed by the .NET Engineering Services (dnceng) team. To have your repo added: +The App installation and backing Secret Manager values are managed by the .NET Engineering +Services (dnceng) team. To have your repo added: 1. Identify the **GitHub org** and **repository** your OneLoc check-in PR targets. For most repos this is the value of the `GitHubOrg` parameter (default `dotnet`) and your repo name. If you use @@ -74,28 +73,37 @@ OneLocBuild template call. For example: LclPackageId: 'LCL-JUNO-PROD-YOURREPO' ``` -Arcade automatically selects the project-scoped service connection: +The project-specific variable group supplies the App ID and private key: -| Azure DevOps project | Service connection | +| Azure DevOps project | Variable group | |---|---| -| `dnceng/internal` | `dnceng-oneloc-githubapp` | -| `DevDiv/DevDiv` | `devdiv-oneloc-githubapp` | +| `dnceng/internal` | `OneLocBuildVariables` (103) | +| `DevDiv/DevDiv` | `OneLocBuildVariables` (343) | -The App client ID, Key Vault, and key name are also centralized in the Arcade template. A pipeline -still needs one-time authorization to use its project's connection. +The variable group must contain `oneloc-localization-app-app-id` and +`oneloc-localization-app-app-private-key`, and the pipeline must be authorized to use the group. ### GitHub App parameters | **Parameter** | **Default** | **Notes** | |:-:|:-:|-| -| `GitHubAppServiceConnection` | `'dnceng-oneloc-githubapp'` | The Azure DevOps **WIF service connection** used by `dnceng/internal`. When the value remains the default, Arcade selects `devdiv-oneloc-githubapp` automatically in `DevDiv/DevDiv`. | -| `GitHubAppClientId` | `'Iv23lijBU8x3gc9lDOc9'` | The GitHub App's **Client ID** (used as the JWT `iss` claim). | -| `GitHubAppKeyVaultName` | `'EngKeyVault'` | The Key Vault holding the App's RSA signing key. | -| `GitHubAppKeyName` | `'oneloc-localization-app-key'` | The name of the RSA key inside that Key Vault (the App's private key). | +| `GitHubAppId` | `$(oneloc-localization-app-app-id)` | Secret Manager-managed GitHub App ID from `OneLocBuildVariables`. | +| `GitHubAppPrivateKey` | `$(oneloc-localization-app-app-private-key)` | Secret Manager-managed PEM private key from `OneLocBuildVariables`. | The token is minted for the installation on the `GitHubOrg` account (default `dotnet`), so make sure `GitHubOrg` (and `MirrorRepo`, if mirroring) point at the org/repo where the App is installed. +### Migrating from Key Vault RSA signing + +The Key Vault RSA signing path has been removed. OneLoc callers must remove +`GitHubAppServiceConnection`, `GitHubAppClientId`, `GitHubAppKeyVaultName`, and +`GitHubAppKeyName`; the default `GitHubAppId` and `GitHubAppPrivateKey` values use the +Secret Manager projections from `OneLocBuildVariables`. + +Direct callers of `get-github-app-token.yml` must replace `azureSubscription`, +`keyVaultName`, `keyName`, and `appClientId` with `appId` and `appPrivateKey`. There is no +fallback to the legacy RSA key. + ## Verifying it works 1. Run your pipeline from a branch where the OneLocBuild job runs. @@ -107,11 +115,8 @@ The token is minted for the installation on the `GitHubOrg` account (default `do ## Troubleshooting - **The App-token step is skipped.** The App path activates when `RepoType` is `gitHub`. -- **The pipeline pauses for service-connection authorization.** Authorize the pipeline to use - `dnceng-oneloc-githubapp` in `dnceng/internal` or `devdiv-oneloc-githubapp` in `DevDiv/DevDiv`. -- **Token minting fails with a Key Vault authorization error.** The service connection identity - needs the `Key Vault Crypto User` role (or at least the `Sign` action) on the App's key. Contact - First Responders. +- **The App ID or private key is empty.** Confirm the pipeline includes and is authorized to use + its project's `OneLocBuildVariables` group, and that the group maps both Secret Manager values. - **`404`/`Not Found` when requesting the installation token.** The App is not installed on the `GitHubOrg` account, or your repository was not selected in the installation. Complete Step 1. - **PR fails to open on your repo.** Ensure the App has `Contents` and `Pull requests` (read & diff --git a/eng/common/Get-GitHubAppToken.ps1 b/eng/common/Get-GitHubAppToken.ps1 index ea776bd6bc2..b9be6430032 100644 --- a/eng/common/Get-GitHubAppToken.ps1 +++ b/eng/common/Get-GitHubAppToken.ps1 @@ -1,13 +1,9 @@ # Mints a short-lived GitHub App installation access token by signing a JWT -# with a private key stored in Azure Key Vault (RSA, RS256). The signed JWT is -# exchanged with the GitHub API for a token scoped to a single installation. +# with an RSA private key (RS256). The signed JWT is exchanged with the GitHub +# API for a token scoped to a single installation. # # Requirements: -# - A GitHub App whose private key has been uploaded into Key Vault as an RSA -# key (the PEM converted to a Key Vault *key*, NOT stored as a secret). -# - The caller (the federated Azure service connection used to run this script) -# must have the `Key Vault Crypto User` role (or at minimum the `Sign` -# action) on that key. +# - A GitHub App ID and PEM private key supplied through the environment. # - The App must be installed on the target organization/account # (`InstallationOwner`) with the permissions/repositories it needs. # @@ -16,18 +12,6 @@ [CmdletBinding()] param( - # Name of the Key Vault that holds the GitHub App's RSA signing key. - [Parameter(Mandatory = $true)] - [string] $KeyVaultName, - - # Name of the RSA key inside the Key Vault (the App's private key). - [Parameter(Mandatory = $true)] - [string] $KeyName, - - # The GitHub App's Client ID (the value to put in the `iss` JWT claim). - [Parameter(Mandatory = $true)] - [string] $AppClientId, - # Login of the organization or user account whose installation we should # mint the token for (e.g. `dotnet`, `microsoft`). [Parameter(Mandatory = $true)] @@ -39,9 +23,7 @@ param( [Parameter(Mandatory = $false)] [string] $OutputVariableName ) - $ErrorActionPreference = 'Stop' -$PSNativeCommandUseErrorActionPreference = $true . $PSScriptRoot\pipeline-logging-functions.ps1 @@ -49,6 +31,17 @@ function ConvertTo-Base64Url([byte[]] $bytes) { return [Convert]::ToBase64String($bytes).TrimEnd('=').Replace('+', '-').Replace('/', '_') } +$appId = $env:GITHUB_APP_ID +$privateKey = $env:GITHUB_APP_PRIVATE_KEY +if ([string]::IsNullOrWhiteSpace($appId) -or [string]::IsNullOrWhiteSpace($privateKey)) { + Write-PipelineTelemetryError -Category 'Build' -Message "GITHUB_APP_ID and GITHUB_APP_PRIVATE_KEY must both be set. Verify both values are supplied as secret pipeline variables." + exit 1 +} +if ($appId -match '^\$\([^)]+\)$' -or $privateKey -match '^\$\([^)]+\)$') { + Write-PipelineTelemetryError -Category 'Build' -Message "The GitHub App ID or private key is an unresolved pipeline variable. Verify the pipeline resolves both secret variables before this task runs." + exit 1 +} + # Build JWT header and payload. Use [ordered] hashtables so JSON # serialization is deterministic. $jwtHeader = [ordered]@{ @@ -59,46 +52,38 @@ $now = [System.DateTimeOffset]::UtcNow $jwtPayload = [ordered]@{ iat = $now.AddMinutes(-1).ToUnixTimeSeconds() exp = $now.AddMinutes(5).ToUnixTimeSeconds() - iss = $AppClientId + iss = $appId } $headerEncoded = ConvertTo-Base64Url ([System.Text.Encoding]::UTF8.GetBytes(($jwtHeader | ConvertTo-Json -Compress))) $payloadEncoded = ConvertTo-Base64Url ([System.Text.Encoding]::UTF8.GetBytes(($jwtPayload | ConvertTo-Json -Compress))) $signingInput = "$headerEncoded.$payloadEncoded" -# Key Vault `sign` expects the *digest* (base64), not the raw bytes. -$sha256 = [System.Security.Cryptography.SHA256]::Create() -$digestBytes = $sha256.ComputeHash([System.Text.Encoding]::UTF8.GetBytes($signingInput)) -$digestBase64 = [Convert]::ToBase64String($digestBytes) +$sha256 = [System.Security.Cryptography.SHA256]::Create() +try { + $digestBytes = $sha256.ComputeHash([System.Text.Encoding]::UTF8.GetBytes($signingInput)) +} +finally { + $sha256.Dispose() +} -Write-Host "Signing JWT with key '$KeyName' in vault '$KeyVaultName'..." -$previousNativeCommandErrorPreference = $PSNativeCommandUseErrorActionPreference +Write-Host 'Signing JWT with the GitHub App private key...' +$rsa = [System.Security.Cryptography.RSA]::Create() try { - # Azure CLI can emit non-fatal Python warnings to stderr even when signing succeeds. - # Use the exit code to determine success for this invocation. - $PSNativeCommandUseErrorActionPreference = $false - $signatureBase64 = az keyvault key sign ` - --vault-name $KeyVaultName ` - --name $KeyName ` - --algorithm RS256 ` - --digest $digestBase64 ` - --query signature ` - --output tsv ` - --only-show-errors - $signExitCode = $LASTEXITCODE + $rsa.ImportFromPem($privateKey) + $signatureBytes = $rsa.SignHash( + $digestBytes, + [System.Security.Cryptography.HashAlgorithmName]::SHA256, + [System.Security.Cryptography.RSASignaturePadding]::Pkcs1) + $signatureUrl = ConvertTo-Base64Url $signatureBytes } catch { - Write-PipelineTelemetryError -Category 'Build' -Message "Failed to sign the JWT via Key Vault (key '$KeyName', vault '$KeyVaultName'): $_. Verify the service connection identity has the 'Key Vault Crypto User' role (Sign action) on the key." + Write-PipelineTelemetryError -Category 'Build' -Message "Failed to sign the GitHub App JWT with the supplied private key: $_" exit 1 } finally { - $PSNativeCommandUseErrorActionPreference = $previousNativeCommandErrorPreference -} -if ($signExitCode -ne 0 -or [string]::IsNullOrWhiteSpace($signatureBase64)) { - Write-PipelineTelemetryError -Category 'Build' -Message "'az keyvault key sign' exited with code $signExitCode for key '$KeyName' in vault '$KeyVaultName'. Verify the service connection identity has the 'Key Vault Crypto User' role (Sign action) on the key." - exit 1 + $rsa.Dispose() } -$signatureUrl = $signatureBase64.Trim().TrimEnd('=').Replace('+', '-').Replace('/', '_') $jwt = "$signingInput.$signatureUrl" $headers = @{ @@ -126,7 +111,7 @@ try { } while ($pageInstallationCount -eq 100) } catch { - Write-PipelineTelemetryError -Category 'Build' -Message "Failed to list GitHub App installations: $_. The signed JWT may be invalid or the App's Client ID ('$AppClientId') may be incorrect." + Write-PipelineTelemetryError -Category 'Build' -Message "Failed to list GitHub App installations: $_. The signed JWT may be invalid or the App ID may be incorrect." exit 1 } $matchingInstallations = @($installations | Where-Object { $_.account.login -ieq $InstallationOwner }) diff --git a/eng/common/core-templates/job/onelocbuild.yml b/eng/common/core-templates/job/onelocbuild.yml index d7a69f1c76e..04051f72936 100644 --- a/eng/common/core-templates/job/onelocbuild.yml +++ b/eng/common/core-templates/job/onelocbuild.yml @@ -8,11 +8,10 @@ parameters: # Project-scoped WIF service connection for Ceapex feed authentication. CeapexServiceConnection: 'dnceng-onelocbuild-ceapex' - # GitHub App authentication for the OneLoc check-in PR. - GitHubAppServiceConnection: 'dnceng-oneloc-githubapp' - GitHubAppClientId: 'Iv23lijBU8x3gc9lDOc9' - GitHubAppKeyVaultName: 'EngKeyVault' - GitHubAppKeyName: 'oneloc-localization-app-key' + # GitHub App authentication for the OneLoc check-in PR. These values come + # from the Key Vault-backed OneLocBuildVariables variable group. + GitHubAppId: $(oneloc-localization-app-app-id) + GitHubAppPrivateKey: $(oneloc-localization-app-app-private-key) SourcesDirectory: $(System.DefaultWorkingDirectory) CreatePr: true @@ -96,13 +95,8 @@ jobs: - template: /eng/common/core-templates/steps/get-github-app-token.yml parameters: is1ESPipeline: ${{ parameters.is1ESPipeline }} - ${{ if and(eq(variables['System.TeamProject'], 'DevDiv'), eq(parameters.GitHubAppServiceConnection, 'dnceng-oneloc-githubapp')) }}: - azureSubscription: 'devdiv-oneloc-githubapp' - ${{ else }}: - azureSubscription: ${{ parameters.GitHubAppServiceConnection }} - keyVaultName: ${{ parameters.GitHubAppKeyVaultName }} - keyName: ${{ parameters.GitHubAppKeyName }} - appClientId: ${{ parameters.GitHubAppClientId }} + appId: ${{ parameters.GitHubAppId }} + appPrivateKey: ${{ parameters.GitHubAppPrivateKey }} installationOwner: ${{ parameters.GitHubOrg }} outputVariableName: 'GitHubAppInstallationToken' condition: ${{ parameters.condition }} diff --git a/eng/common/core-templates/steps/get-github-app-token.yml b/eng/common/core-templates/steps/get-github-app-token.yml index 6d42a48d3c3..1dda75f8cd8 100644 --- a/eng/common/core-templates/steps/get-github-app-token.yml +++ b/eng/common/core-templates/steps/get-github-app-token.yml @@ -1,13 +1,9 @@ # Mints a short-lived GitHub App installation access token by signing a JWT -# with a private key stored in Azure Key Vault (RSA, RS256). The JWT is -# exchanged with the GitHub API for a token scoped to a single installation. +# with an RSA private key (RS256). The JWT is exchanged with the GitHub API +# for a token scoped to a single installation. # # Requirements (per GitHub App you want to authenticate as): -# - A GitHub App with its private key uploaded into Key Vault as an RSA key -# (PEM converted to a key, NOT stored as a secret). -# - The Azure service connection passed via `azureSubscription` must be -# granted the `Key Vault Crypto User` role (or at minimum `Sign` action) -# on that key. +# - A GitHub App ID and PEM private key supplied by secret pipeline variables. # - The App must be installed on the target organization/account # (`installationOwner`) with the permissions/repositories you need. # @@ -17,23 +13,11 @@ # enterprise classic-PAT lifetime policy. parameters: -# Azure DevOps service connection (federated) that can call -# `az keyvault key sign` on the App's signing key. -- name: azureSubscription +# Secret pipeline values produced by Secret Manager's github-app-secret type. +- name: appId type: string -# Name of the Key Vault that holds the GitHub App's RSA signing key. -- name: keyVaultName - type: string - -# Name of the RSA key inside the Key Vault (the App's private key). -- name: keyName - type: string - -# The GitHub App's Client ID (the value to put in the `iss` JWT claim). -# Prefer this over the numeric App ID; GitHub accepts either, but Client ID -# is the documented form going forward. -- name: appClientId +- name: appPrivateKey type: string # Login of the organization or user account whose installation we should @@ -61,19 +45,18 @@ parameters: default: Get GitHub App installation token steps: -- task: AzureCLI@2 +- task: PowerShell@2 displayName: ${{ parameters.displayName }} name: ${{ parameters.stepName }} ${{ if ne(parameters.condition, '') }}: condition: ${{ parameters.condition }} inputs: - azureSubscription: ${{ parameters.azureSubscription }} - scriptType: pscore - scriptLocation: inlineScript - inlineScript: | - & "$(System.DefaultWorkingDirectory)/eng/common/Get-GitHubAppToken.ps1" ` - -KeyVaultName '${{ parameters.keyVaultName }}' ` - -KeyName '${{ parameters.keyName }}' ` - -AppClientId '${{ parameters.appClientId }}' ` - -InstallationOwner '${{ parameters.installationOwner }}' ` - -OutputVariableName '${{ parameters.outputVariableName }}' + targetType: filePath + filePath: $(System.DefaultWorkingDirectory)/eng/common/Get-GitHubAppToken.ps1 + arguments: >- + -InstallationOwner '${{ parameters.installationOwner }}' + -OutputVariableName '${{ parameters.outputVariableName }}' + pwsh: true + env: + GITHUB_APP_ID: ${{ parameters.appId }} + GITHUB_APP_PRIVATE_KEY: ${{ parameters.appPrivateKey }} From 070e7d38500916b763e23abac43b13c53670b239 Mon Sep 17 00:00:00 2001 From: Missy Messa <47990216+missymessa@users.noreply.github.com> Date: Fri, 11 Sep 2026 12:35:44 -0700 Subject: [PATCH 2/3] Read OneLoc GitHub App credentials through WIF (#17541) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- Documentation/OneLocBuild.md | 9 ++- Documentation/OneLocBuildGitHubApp.md | 56 +++++++++------ eng/common/Get-GitHubAppToken.ps1 | 72 +++++++++++++++++-- eng/common/core-templates/job/onelocbuild.yml | 18 +++-- .../steps/get-github-app-token.yml | 38 ++++++---- 5 files changed, 144 insertions(+), 49 deletions(-) diff --git a/Documentation/OneLocBuild.md b/Documentation/OneLocBuild.md index 19a028479ce..50e6207eaad 100644 --- a/Documentation/OneLocBuild.md +++ b/Documentation/OneLocBuild.md @@ -203,11 +203,16 @@ The parameters that can be passed to the template are as follows: | `LclSource` | `LclFilesInRepo` | This passes the `LclSource` input to the OneLocBuild task as described in [its documentation](https://ceapex.visualstudio.com/CEINTL/_wiki/wikis/CEINTL.wiki/107/Localization-with-OneLocBuild-Task?anchor=languageset%2C-languages-(required)). For most repos, this should be set to `LclFilesfromPackage`. | | `LclPackageId` | `''` | When `LclSource` is set to `LclFilesfromPackage`, this passes in the package ID as described in the [OneLocBuild task documentation](https://ceapex.visualstudio.com/CEINTL/_wiki/wikis/CEINTL.wiki/107/Localization-with-OneLocBuild-Task?anchor=scenario-2%3A-lcl-files-from-a-package). | | `CeapexServiceConnection` | `'dnceng-onelocbuild-ceapex'` | The project-scoped WIF service connection used to acquire a short-lived token for the Ceapex feeds. OneLocBuild supports only `dnceng/internal` and `DevDiv/DevDiv`; pipelines must be authorized to use the connection. | -| `GitHubAppId` | `$(oneloc-localization-app-app-id)` | Secret Manager-managed GitHub App ID from `OneLocBuildVariables`. | -| `GitHubAppPrivateKey` | `$(oneloc-localization-app-app-private-key)` | Secret Manager-managed PEM private key from `OneLocBuildVariables`. | +| `GitHubAppServiceConnection` | `'dnceng-oneloc-githubapp'` | Project-scoped WIF service connection used to read the App credentials from Key Vault. DevDiv automatically uses `devdiv-oneloc-githubapp` when this default is unchanged. | +| `GitHubAppKeyVaultName` | `'EngKeyVault'` | Key Vault containing the Secret Manager-managed GitHub App credentials. | +| `GitHubAppIdSecretName` | `'oneloc-localization-app-app-id'` | Secret Manager projection containing the GitHub App ID. | +| `GitHubAppPrivateKeySecretName` | `'oneloc-localization-app-app-private-key'` | Secret Manager projection containing the PEM private key. | | `condition` | `''` | Allows for conditionalizing the template's steps on build-time variables. | | `JobNameSuffix` | `''` | Allows for custom job name suffix. This is helpful for disambiguation in case of need for more then one OneLocBuild job run - e.g. as a way to set multiple package IDs. | +GitHub OneLoc pipelines must be authorized to use their project's GitHub App WIF service +connection. The service connection has read access only to the two required Key Vault secrets. + The previous Key Vault RSA signing parameters have been removed. See [Authenticating OneLocBuild's GitHub check-in with the GitHub App](OneLocBuildGitHubApp.md#migrating-from-key-vault-rsa-signing) for the required parameter migration. diff --git a/Documentation/OneLocBuildGitHubApp.md b/Documentation/OneLocBuildGitHubApp.md index 3fbdd99a7eb..52558487d99 100644 --- a/Documentation/OneLocBuildGitHubApp.md +++ b/Documentation/OneLocBuildGitHubApp.md @@ -35,10 +35,12 @@ If App token minting or authentication fails, the job fails; there is no stored- 1. **The App must be installed on the GitHub org/account that owns your target repo, and your specific repository must be selected in that installation.** The App can only open a PR against a repository it is installed on. This is what actually grants the App permission to your repo. -2. **Your pipeline must run in `dnceng/internal` or `DevDiv/DevDiv` and include its project's - Key Vault-backed `OneLocBuildVariables` variable group.** +2. **Your pipeline must run in `dnceng/internal` or `DevDiv/DevDiv` and be authorized to use its + project's GitHub App WIF service connection.** The shared OneLoc job uses that identity to read + only the two required Secret Manager projections from Key Vault. -The .NET Engineering Services team manages the App credentials and variable groups. +The .NET Engineering Services team manages the App credentials, Key Vault permissions, and +service connections. ### Step 1 — Request that your repository be added to the App installation @@ -73,39 +75,52 @@ OneLocBuild template call. For example: LclPackageId: 'LCL-JUNO-PROD-YOURREPO' ``` -The project-specific variable group supplies the App ID and private key: +The project-specific WIF service connection reads the App ID and private key directly from +EngKeyVault: -| Azure DevOps project | Variable group | +| Azure DevOps project | Service connection | |---|---| -| `dnceng/internal` | `OneLocBuildVariables` (103) | -| `DevDiv/DevDiv` | `OneLocBuildVariables` (343) | +| `dnceng/internal` | `dnceng-oneloc-githubapp` | +| `DevDiv/DevDiv` | `devdiv-oneloc-githubapp` | -The variable group must contain `oneloc-localization-app-app-id` and -`oneloc-localization-app-app-private-key`, and the pipeline must be authorized to use the group. +Each identity has `Key Vault Secrets User` access scoped to only +`oneloc-localization-app-app-id` and `oneloc-localization-app-app-private-key`. Each pipeline must +be authorized to use its project's service connection. ### GitHub App parameters | **Parameter** | **Default** | **Notes** | |:-:|:-:|-| -| `GitHubAppId` | `$(oneloc-localization-app-app-id)` | Secret Manager-managed GitHub App ID from `OneLocBuildVariables`. | -| `GitHubAppPrivateKey` | `$(oneloc-localization-app-app-private-key)` | Secret Manager-managed PEM private key from `OneLocBuildVariables`. | +| `GitHubAppServiceConnection` | `dnceng-oneloc-githubapp` | WIF service connection used to read the App credentials. DevDiv automatically selects `devdiv-oneloc-githubapp` when this default is unchanged. | +| `GitHubAppKeyVaultName` | `EngKeyVault` | Key Vault containing the Secret Manager projections. | +| `GitHubAppIdSecretName` | `oneloc-localization-app-app-id` | Secret containing the GitHub App ID. | +| `GitHubAppPrivateKeySecretName` | `oneloc-localization-app-app-private-key` | Secret containing the PEM private key. | The token is minted for the installation on the `GitHubOrg` account (default `dotnet`), so make sure `GitHubOrg` (and `MirrorRepo`, if mirroring) point at the org/repo where the App is installed. ### Migrating from Key Vault RSA signing -The Key Vault RSA signing path has been removed. OneLoc callers must remove -`GitHubAppServiceConnection`, `GitHubAppClientId`, `GitHubAppKeyVaultName`, and -`GitHubAppKeyName`; the default `GitHubAppId` and `GitHubAppPrivateKey` values use the -Secret Manager projections from `OneLocBuildVariables`. +The Key Vault RSA signing path and pipeline-variable credential path have both been removed. -Direct callers of `get-github-app-token.yml` must replace `azureSubscription`, -`keyVaultName`, `keyName`, and `appClientId` with `appId` and `appPrivateKey`. There is no -fallback to the legacy RSA key. +OneLoc job callers using `GitHubAppId` and `GitHubAppPrivateKey` must remove those parameters. +Callers from the older RSA interface must remove `GitHubAppClientId` and `GitHubAppKeyName`; +`GitHubAppServiceConnection` and `GitHubAppKeyVaultName` retain their meanings. Standard callers +need no replacement parameters because the four service-connection and secret-name defaults apply +automatically. + +Direct callers of `get-github-app-token.yml` using `appId` and `appPrivateKey` must replace those +parameters with `azureSubscription`, `keyVaultName`, `appIdSecretName`, and +`appPrivateKeySecretName`. Direct callers from the older RSA interface keep `azureSubscription` +and `keyVaultName`, remove `keyName` and `appClientId`, and add the two secret-name parameters. +There is no fallback to the legacy RSA key. ## Verifying it works +Changes to the GitHub App credential retrieval path must pass a protected internal canary before +merge. Public pull-request builds intentionally cannot access the WIF service connection or App +private key, so syntax and unit checks alone do not validate this boundary. + 1. Run your pipeline from a branch where the OneLocBuild job runs. 2. In the build, confirm the **`Get GitHub App installation token`** step runs and succeeds before the `OneLocBuild` task. @@ -115,8 +130,9 @@ fallback to the legacy RSA key. ## Troubleshooting - **The App-token step is skipped.** The App path activates when `RepoType` is `gitHub`. -- **The App ID or private key is empty.** Confirm the pipeline includes and is authorized to use - its project's `OneLocBuildVariables` group, and that the group maps both Secret Manager values. +- **The App ID or private key cannot be read.** Confirm the pipeline is authorized to use its + project's GitHub App service connection and that its identity has `Key Vault Secrets User` + access to both configured secrets. - **`404`/`Not Found` when requesting the installation token.** The App is not installed on the `GitHubOrg` account, or your repository was not selected in the installation. Complete Step 1. - **PR fails to open on your repo.** Ensure the App has `Contents` and `Pull requests` (read & diff --git a/eng/common/Get-GitHubAppToken.ps1 b/eng/common/Get-GitHubAppToken.ps1 index b9be6430032..ec005e487c3 100644 --- a/eng/common/Get-GitHubAppToken.ps1 +++ b/eng/common/Get-GitHubAppToken.ps1 @@ -3,7 +3,9 @@ # API for a token scoped to a single installation. # # Requirements: -# - A GitHub App ID and PEM private key supplied through the environment. +# - A GitHub App ID and PEM private key stored as Azure Key Vault secrets. +# - The federated Azure service connection running this script must have +# `Get` access to those two secrets. # - The App must be installed on the target organization/account # (`InstallationOwner`) with the permissions/repositories it needs. # @@ -12,6 +14,18 @@ [CmdletBinding()] param( + # Name of the Key Vault holding the GitHub App credentials. + [Parameter(Mandatory = $true)] + [string] $KeyVaultName, + + # Secret Manager projection containing the GitHub App ID. + [Parameter(Mandatory = $true)] + [string] $AppIdSecretName, + + # Secret Manager projection containing the PEM private key. + [Parameter(Mandatory = $true)] + [string] $AppPrivateKeySecretName, + # Login of the organization or user account whose installation we should # mint the token for (e.g. `dotnet`, `microsoft`). [Parameter(Mandatory = $true)] @@ -24,24 +38,68 @@ param( [string] $OutputVariableName ) $ErrorActionPreference = 'Stop' +$PSNativeCommandUseErrorActionPreference = $true . $PSScriptRoot\pipeline-logging-functions.ps1 +if ($KeyVaultName -notmatch '^[A-Za-z][A-Za-z0-9-]{1,22}[A-Za-z0-9]$' -or $KeyVaultName.Contains('--')) { + Write-PipelineTelemetryError -Category 'Build' -Message "KeyVaultName '$KeyVaultName' is not a valid Azure Key Vault name." + exit 1 +} + function ConvertTo-Base64Url([byte[]] $bytes) { return [Convert]::ToBase64String($bytes).TrimEnd('=').Replace('+', '-').Replace('/', '_') } -$appId = $env:GITHUB_APP_ID -$privateKey = $env:GITHUB_APP_PRIVATE_KEY -if ([string]::IsNullOrWhiteSpace($appId) -or [string]::IsNullOrWhiteSpace($privateKey)) { - Write-PipelineTelemetryError -Category 'Build' -Message "GITHUB_APP_ID and GITHUB_APP_PRIVATE_KEY must both be set. Verify both values are supplied as secret pipeline variables." +$previousNativeCommandErrorPreference = $PSNativeCommandUseErrorActionPreference +try { + # Azure CLI can emit non-fatal Python warnings to stderr. + $PSNativeCommandUseErrorActionPreference = $false + $keyVaultAccessToken = az account get-access-token ` + --resource https://vault.azure.net ` + --query accessToken ` + --output tsv ` + --only-show-errors + $tokenExitCode = $LASTEXITCODE +} +catch { + Write-PipelineTelemetryError -Category 'Build' -Message "Failed to acquire an Azure Key Vault access token: $_" exit 1 } -if ($appId -match '^\$\([^)]+\)$' -or $privateKey -match '^\$\([^)]+\)$') { - Write-PipelineTelemetryError -Category 'Build' -Message "The GitHub App ID or private key is an unresolved pipeline variable. Verify the pipeline resolves both secret variables before this task runs." +finally { + $PSNativeCommandUseErrorActionPreference = $previousNativeCommandErrorPreference +} +if ($tokenExitCode -ne 0 -or [string]::IsNullOrWhiteSpace($keyVaultAccessToken)) { + Write-PipelineTelemetryError -Category 'Build' -Message "'az account get-access-token' exited with code $tokenExitCode while acquiring an Azure Key Vault access token." exit 1 } +function Get-KeyVaultSecret([string] $SecretName) { + # Use the data-plane REST API because `az keyvault secret show` can fail + # with Errno 22 on hosted Windows agents when reading these projections. + $escapedSecretName = [Uri]::EscapeDataString($SecretName) + $secretUri = "https://$KeyVaultName.vault.azure.net/secrets/$escapedSecretName`?api-version=7.4" + try { + $response = Invoke-RestMethod ` + -Uri $secretUri ` + -Headers @{ Authorization = "Bearer $keyVaultAccessToken" } ` + -Method Get + } + catch { + Write-PipelineTelemetryError -Category 'Build' -Message "Failed to read secret '$SecretName' from vault '$KeyVaultName': $_. Verify the secret exists and the service connection has 'Key Vault Secrets User' access to it." + exit 1 + } + if ([string]::IsNullOrWhiteSpace($response.value)) { + Write-PipelineTelemetryError -Category 'Build' -Message "Secret '$SecretName' in vault '$KeyVaultName' is empty." + exit 1 + } + return [string] $response.value +} + +Write-Host "Reading GitHub App credentials from vault '$KeyVaultName'..." +$appId = Get-KeyVaultSecret $AppIdSecretName +$privateKey = Get-KeyVaultSecret $AppPrivateKeySecretName + # Build JWT header and payload. Use [ordered] hashtables so JSON # serialization is deterministic. $jwtHeader = [ordered]@{ diff --git a/eng/common/core-templates/job/onelocbuild.yml b/eng/common/core-templates/job/onelocbuild.yml index 04051f72936..adcdfa17536 100644 --- a/eng/common/core-templates/job/onelocbuild.yml +++ b/eng/common/core-templates/job/onelocbuild.yml @@ -8,10 +8,11 @@ parameters: # Project-scoped WIF service connection for Ceapex feed authentication. CeapexServiceConnection: 'dnceng-onelocbuild-ceapex' - # GitHub App authentication for the OneLoc check-in PR. These values come - # from the Key Vault-backed OneLocBuildVariables variable group. - GitHubAppId: $(oneloc-localization-app-app-id) - GitHubAppPrivateKey: $(oneloc-localization-app-app-private-key) + # GitHub App authentication for the OneLoc check-in PR. + GitHubAppServiceConnection: 'dnceng-oneloc-githubapp' + GitHubAppKeyVaultName: 'EngKeyVault' + GitHubAppIdSecretName: 'oneloc-localization-app-app-id' + GitHubAppPrivateKeySecretName: 'oneloc-localization-app-app-private-key' SourcesDirectory: $(System.DefaultWorkingDirectory) CreatePr: true @@ -95,8 +96,13 @@ jobs: - template: /eng/common/core-templates/steps/get-github-app-token.yml parameters: is1ESPipeline: ${{ parameters.is1ESPipeline }} - appId: ${{ parameters.GitHubAppId }} - appPrivateKey: ${{ parameters.GitHubAppPrivateKey }} + ${{ if and(eq(variables['System.TeamProject'], 'DevDiv'), eq(parameters.GitHubAppServiceConnection, 'dnceng-oneloc-githubapp')) }}: + azureSubscription: 'devdiv-oneloc-githubapp' + ${{ else }}: + azureSubscription: ${{ parameters.GitHubAppServiceConnection }} + keyVaultName: ${{ parameters.GitHubAppKeyVaultName }} + appIdSecretName: ${{ parameters.GitHubAppIdSecretName }} + appPrivateKeySecretName: ${{ parameters.GitHubAppPrivateKeySecretName }} installationOwner: ${{ parameters.GitHubOrg }} outputVariableName: 'GitHubAppInstallationToken' condition: ${{ parameters.condition }} diff --git a/eng/common/core-templates/steps/get-github-app-token.yml b/eng/common/core-templates/steps/get-github-app-token.yml index 1dda75f8cd8..3eeb5a4c1bc 100644 --- a/eng/common/core-templates/steps/get-github-app-token.yml +++ b/eng/common/core-templates/steps/get-github-app-token.yml @@ -3,7 +3,9 @@ # for a token scoped to a single installation. # # Requirements (per GitHub App you want to authenticate as): -# - A GitHub App ID and PEM private key supplied by secret pipeline variables. +# - A GitHub App ID and PEM private key stored as Azure Key Vault secrets. +# - The Azure service connection passed via `azureSubscription` must have +# `Get` access to those two secrets. # - The App must be installed on the target organization/account # (`installationOwner`) with the permissions/repositories you need. # @@ -13,11 +15,18 @@ # enterprise classic-PAT lifetime policy. parameters: -# Secret pipeline values produced by Secret Manager's github-app-secret type. -- name: appId +# Azure DevOps service connection (federated) that can read the App credentials. +- name: azureSubscription type: string -- name: appPrivateKey +# Name of the Key Vault holding Secret Manager's github-app-secret projections. +- name: keyVaultName + type: string + +- name: appIdSecretName + type: string + +- name: appPrivateKeySecretName type: string # Login of the organization or user account whose installation we should @@ -45,18 +54,19 @@ parameters: default: Get GitHub App installation token steps: -- task: PowerShell@2 +- task: AzureCLI@2 displayName: ${{ parameters.displayName }} name: ${{ parameters.stepName }} ${{ if ne(parameters.condition, '') }}: condition: ${{ parameters.condition }} inputs: - targetType: filePath - filePath: $(System.DefaultWorkingDirectory)/eng/common/Get-GitHubAppToken.ps1 - arguments: >- - -InstallationOwner '${{ parameters.installationOwner }}' - -OutputVariableName '${{ parameters.outputVariableName }}' - pwsh: true - env: - GITHUB_APP_ID: ${{ parameters.appId }} - GITHUB_APP_PRIVATE_KEY: ${{ parameters.appPrivateKey }} + azureSubscription: ${{ parameters.azureSubscription }} + scriptType: pscore + scriptLocation: inlineScript + inlineScript: | + & "$(System.DefaultWorkingDirectory)/eng/common/Get-GitHubAppToken.ps1" ` + -KeyVaultName '${{ parameters.keyVaultName }}' ` + -AppIdSecretName '${{ parameters.appIdSecretName }}' ` + -AppPrivateKeySecretName '${{ parameters.appPrivateKeySecretName }}' ` + -InstallationOwner '${{ parameters.installationOwner }}' ` + -OutputVariableName '${{ parameters.outputVariableName }}' From a59de24eae8b3c704edf7395930a8558f508e976 Mon Sep 17 00:00:00 2001 From: Missy Messa Date: Fri, 2 Oct 2026 10:45:18 -0700 Subject: [PATCH 3/3] Preserve release/10.0 OneLoc project guards Keep the release branch's internal and DevDiv project checks while backporting the GitHub App credential changes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4e96402b-01cb-4650-acf6-4ee514e5268c --- Documentation/OneLocBuild.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/Documentation/OneLocBuild.md b/Documentation/OneLocBuild.md index 50e6207eaad..48b868fd17c 100644 --- a/Documentation/OneLocBuild.md +++ b/Documentation/OneLocBuild.md @@ -27,12 +27,12 @@ Onboarding to OneLocBuild is a simple process: parameters: CreatePr: false ``` -Note: If you are running your PR builds and official builds off of the same definition and are on dnceng, -you will want to conditionalize this step with the following: +Note: If you are running your PR builds and official builds off of the same definition, conditionalize +this step so OneLocBuild runs only in the supported `dnceng/internal` or `DevDiv/DevDiv` project: ```yaml -- ${{ if and(ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest')) }}: +- ${{ if and(or(eq(variables['System.TeamProject'], 'internal'), eq(variables['System.TeamProject'], 'DevDiv')), notin(variables['Build.Reason'], 'PullRequest')) }}: ``` -To prevent OneLocBuild from running in the public project where it will fail. +This also prevents OneLocBuild from running during pull request validation. 3. Run the pipeline you want to use OneLocBuild on your test branch. 4. Open a ticket with the localization team using @@ -51,7 +51,7 @@ To prevent OneLocBuild from running in the public project where it will fail. Make sure to remove the `CreatePr: false` line from step 2. Additionally, if you added the YAML condition from step 2, make sure that your new YAML condition now looks like: ```yaml -- ${{ if and(ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest'), eq(variables['Build.SourceBranch'], 'refs/heads/main')) }}: +- ${{ if and(or(eq(variables['System.TeamProject'], 'internal'), eq(variables['System.TeamProject'], 'DevDiv')), notin(variables['Build.Reason'], 'PullRequest'), eq(variables['Build.SourceBranch'], 'refs/heads/main')) }}: ``` 7. If using a mirrored repository (your code is mirrored to a trusted repository which your official build uses), @@ -177,7 +177,7 @@ The most basic structure for calling the OneLocBuild template is: ```yaml jobs: -- ${{ if and(ne(variables['System.TeamProject'], 'public'), notin(variables['Build.Reason'], 'PullRequest'), eq(variables['Build.SourceBranch'], 'refs/heads/main')) }}: +- ${{ if and(or(eq(variables['System.TeamProject'], 'internal'), eq(variables['System.TeamProject'], 'DevDiv')), notin(variables['Build.Reason'], 'PullRequest'), eq(variables['Build.SourceBranch'], 'refs/heads/main')) }}: - template: /eng/common/templates/job/onelocbuild.yml parameters: LclSource: lclFilesfromPackage