diff --git a/CHANGELOG.md b/CHANGELOG.md index 450e0573..804fff43 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -51,7 +51,9 @@ The patch number restarted from 1, below the 2.0.x versions already published. - `bv` recognizes a CI runner [as the .NET CLI does](docs/environment-variables.md#variables-that-mark-a-ci-runner), and passes `ContinuousIntegrationBuild=true` to the build pipeline on every one. - `bv release` runs [on any machine](docs/tool-commands/release.md#preconditions), with `repository.url` naming the repository the release goes to. - `bv release` fetches the tags of the [release remote](docs/tool-commands/release.md#the-release-remote) before it checks the versions. -- `release.allowUnknownHost` lets `bv release` [tag and push without a release page](docs/tool-commands/release.md#publishing) on a Git host `bv` does not know. +- `release.allowUnknownHost` lets `bv release` [tag and push without a release page](docs/tool-commands/release.md#publishing-on-an-unknown-host) on a Git host `bv` does not know. +- `bv release` [publishes on GitLab](docs/tool-commands/release.md#publishing-on-gitlab), with the assets in the generic package registry of the project, and `hosts.gitlab.tokenEnv` names the token variable. +- `bv release` takes a host whose name ends in `.ghe.com` for [GitHub Enterprise Cloud with data residency](docs/tool-commands/release.md#publishing-on-github), whose API is at `https://api.`. - [`bv info`](docs/tool-commands/info.md) prints the CI platform, the Git host, the release remote, and the identity a release would use. ### Changes to existing features @@ -95,6 +97,7 @@ The patch number restarted from 1, below the 2.0.x versions already published. - **BREAKING CHANGE**: A file-based app is a [project type of its own](docs/internal-use-properties.md#project-type), `BV_IsFileBasedAppProject`, so `BV_IsExeProject` is `false` for it. - **BREAKING CHANGE**: `bv` no longer reads `.config/dotnet-tools.json`, and stops with an error that [names the move](docs/directory-structure.md#migration-from-configdotnet-toolsjson) when the home directory holds one. - The [`ReleaseAssetList` module](docs/sdk-modules/release-asset-list.md#generatereleaseassetlist-property) writes no release asset list for a file-based app. +- On GitHub, `bv release` refuses [two assets with the same file name](docs/tool-commands/release.md#publishing-on-github) before it pushes the commits. - A [`notice:` level](docs/command-line.md#verbosity), shown from `minimal` verbosity up, sits between `warning:` and `info:`. - The tasks of Buildvana SDK show each [message level](docs/command-line.md#verbosity) from the same verbosity as `bv`. - `bv` runs from any directory under the [home directory](docs/command-line.md#home-directory), which it finds as Buildvana SDK does and makes the current directory. @@ -110,7 +113,7 @@ The patch number restarted from 1, below the 2.0.x versions already published. - `bv release` runs the [build pipeline](docs/tool-commands/release.md#the-steps) once, from `clean` through `pack`, after the release commit, and no longer runs `clean` through `test` before it. - [The pages under `docs/`](docs/README.md) moved to kebab-case names, `docs/modules/` to `docs/sdk-modules/`, and `docs/DependencyManagement.md` to `docs/tool-commands/dependencies.md`, so a link to an old name breaks. - The `StandardAnalyzers` module adds [`StyleCop.Analyzers` 1.3.133-beta](docs/sdk-modules/standard-analyzers.md#package-references), which supports C# 8 to 15 and adds several new rules. -- `bv release` reaches the [GitHub API of the host of the repository](docs/tool-commands/release.md#publishing), `api.github.com` or `/api/v3`, where it reached `api.github.com` alone. +- `bv release` reaches the [GitHub API of the host of the repository](docs/tool-commands/release.md#publishing-on-github), `api.github.com` or `/api/v3`, where it reached `api.github.com` alone. - `bv release` passes `ContinuousIntegrationBuild=true` to [its pipeline run](docs/tool-commands/release.md#the-steps) on every platform. ### Bugs fixed in this release diff --git a/buildvana.example.jsonc b/buildvana.example.jsonc index 3c597d87..fb042192 100644 --- a/buildvana.example.jsonc +++ b/buildvana.example.jsonc @@ -146,6 +146,12 @@ // Name of the environment variable that holds the GitHub access token. "tokenEnv": "GITHUB_TOKEN", }, + + // Configuration for the GitLab host. + "gitlab": { + // Name of the environment variable that holds the GitLab access token. + "tokenEnv": "GITLAB_TOKEN", + }, }, // Configuration for Git-related behavior. diff --git a/docs/configuration-file.md b/docs/configuration-file.md index ca2e0ec2..51c3bc75 100644 --- a/docs/configuration-file.md +++ b/docs/configuration-file.md @@ -86,7 +86,7 @@ A setting whose member names are data, such as `dependencies.policies` and `dotn | ------------------------------------- | ------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `$schema` | none | The URL of the JSON schema, whose version segment [`bv self-update`](tool-commands/self-update.md#what-the-command-updates) moves with the other pins. | | `repository.url` | none | The URL of the repository that [`bv release`](tool-commands/release.md#preconditions) publishes to, and without it `bv release` fails. | -| `repository.host` | `auto` | The [kind of Git host](tool-commands/release.md#preconditions) of the repository: `auto` or `github`. | +| `repository.host` | `auto` | The [kind of Git host](tool-commands/release.md#preconditions) of the repository: `auto`, `github`, or `gitlab`. | | `release.branches` | none | The regular expressions of the branches that produce [public releases](sdk-modules/versioning.md#releasebranches-setting), and by default no branch does. | | `release.configuration` | `dotnet.configuration` | The [build configuration of `bv release`](tool-commands/release.md#the-build-configuration). | | `release.checkPublicApi` | `true` | Whether the public API files take part in [the version spec change](tool-commands/release.md#the-version-spec-change) of `bv release`. | @@ -114,6 +114,7 @@ A setting whose member names are data, such as `dependencies.policies` and `dotn | `nuget.feeds.release.source` | none | The URL of the feed that [stable versions](tool-commands/release.md#publishing) are pushed to. | | `nuget.feeds.release.apiKeyEnv` | none | The environment variable holding the [API key](environment-variables.md#secret-carrying-variables-named-by-the-configuration-file) of the release feed. | | `hosts.github.tokenEnv` | `GITHUB_TOKEN` | The environment variable holding the [GitHub token](environment-variables.md#secret-carrying-variables-named-by-the-configuration-file) of the release operations. | +| `hosts.gitlab.tokenEnv` | `GITLAB_TOKEN` | The environment variable holding the [GitLab token](environment-variables.md#secret-carrying-variables-named-by-the-configuration-file) of the release operations. | | `git.identity.name` | none | The name of the [author of the commits of `bv release`](tool-commands/release.md#preconditions). | | `git.identity.email` | none | The email address of the [author of the commits of `bv release`](tool-commands/release.md#preconditions). | | `dependencies.scopes.netsdk` | `major` | The [update policy](tool-commands/dependencies.md#update-policies) of the `netsdk` scope, the .NET SDK version in `global.json`. | diff --git a/docs/environment-variables.md b/docs/environment-variables.md index ca5e8795..315aced7 100644 --- a/docs/environment-variables.md +++ b/docs/environment-variables.md @@ -64,9 +64,10 @@ The variable matters only together with `GITLAB_CI`, which is what selects the G ### `CI_SERVER_URL` -GitLab CI sets it to the URL of the GitLab instance that runs the job. -`bv` reads it as the server of the GitLab CI platform. -`bv` knows no GitLab host, so the variable has no effect on a release. +GitLab CI sets it to the URL of the GitLab instance that runs the job, such as `https://gitlab.com`. +When `repository.url` names a repository on the host of that URL, `bv release` takes the host for GitLab. +A self-managed GitLab instance is recognized this way on its own runners. +The variable matters only together with `GITLAB_CI`. ### `DOTNET_CLI_CONSOLE_USE_DEFAULT_ENCODING` @@ -163,6 +164,10 @@ The [configuration file](configuration-file.md) names the environment variable t - `hosts.github.tokenEnv` names the variable holding the GitHub token of the release operations. The token is also the password of the fetch and the push of `bv release`. The default name is `GITHUB_TOKEN`. +- `hosts.gitlab.tokenEnv` names the variable holding the GitLab token of the release operations. + The token is also the password of the fetch and the push of `bv release`. + The default name is `GITLAB_TOKEN`. + The name `CI_JOB_TOKEN` selects the job token of GitLab CI, as [Publishing on GitLab](tool-commands/release.md#publishing-on-gitlab) says. - `nuget.feeds.release.apiKeyEnv` and `nuget.feeds.prerelease.apiKeyEnv` name the variables holding the API keys of the NuGet push feeds. --- diff --git a/docs/getting-started.md b/docs/getting-started.md index 07fea50f..370f7dd3 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -32,7 +32,7 @@ Every command below runs in the directory you clone into, which becomes the [hom ## Create the repository -Create an empty repository on your Git server, clone it, and enter the directory: +Create an empty repository on your Git host, clone it, and enter the directory: ```shell git clone MyProduct diff --git a/docs/introduction.md b/docs/introduction.md index d9d68816..3c2517f3 100644 --- a/docs/introduction.md +++ b/docs/introduction.md @@ -15,7 +15,7 @@ This page says what Buildvana is made of, what it does for a project, and what i - [Operating systems](#operating-systems) - [Project types](#project-types) - [Programming languages](#programming-languages) - - [Git servers](#git-servers) + - [Git hosts](#git-hosts) - [CI platforms](#ci-platforms) - [Toolchain](#toolchain) @@ -54,29 +54,29 @@ Each table below states a status for every item. ### Operating systems -| Operating system | Status | -| ---------------- | --------- | -| Windows | Supported | -| Linux | Supported | -| macOS | Untested | +| Operating system | Status | +| ---------------- | -------------- | +| Windows | Supported | +| Linux | Supported | +| macOS | ??? (untested) | ### Project types -| Project type | Status | -| -------------------------------------------------------- | ----------- | -| Multi-platform and cross-platform projects | Supported | -| Libraries | Supported | -| Console apps | Supported | -| File-based apps | Supported | -| Windows Forms | Supported | -| ASP.NET | Supported | -| Projects using the `Microsoft.Build.NoTargets` SDK | Supported | -| [Avalonia UI](https://avaloniaui.net) | Supported | -| WPF | Untested | -| [UNO Platform](https://platform.uno) | Untested | -| .NET MAUI | Untested | -| Legacy projects, with no `Sdk` attribute | Unsupported | -| Projects whose SDK does not layer on `Microsoft.NET.Sdk` | Unsupported | +| Project type | Status | +| -------------------------------------------------------- | -------------- | +| Multi-platform and cross-platform projects | Supported | +| Libraries | Supported | +| Console apps | Supported | +| File-based apps | Supported | +| Windows Forms | Supported | +| ASP.NET | Supported | +| Projects using the `Microsoft.Build.NoTargets` SDK | Supported | +| [Avalonia UI](https://avaloniaui.net) | Supported | +| WPF | ??? (untested) | +| [UNO Platform](https://platform.uno) | ??? (untested) | +| .NET MAUI | ??? (untested) | +| Legacy projects, with no `Sdk` attribute | Unsupported | +| Projects whose SDK does not layer on `Microsoft.NET.Sdk` | Unsupported | ### Programming languages @@ -87,12 +87,16 @@ Each table below states a status for every item. | F# | Unsupported | | Other languages | Unsupported | -### Git servers +### Git hosts -| Server | Status | Limits | -| ---------------------------- | --------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| GitHub and GitHub Enterprise | Supported | | -| Other servers | Supported with limits | Buildvana SDK does not configure SourceLink.
`bv release` creates no release page, and runs only with [`release.allowUnknownHost`](tool-commands/release.md#publishing). | +| Git host | Status | Limits | +| ------------------------------------------- | --------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| GitHub | Supported | | +| GitHub Enterprise Server | Supported (untested) | [SourceLink](https://github.com/dotnet/sourcelink) needs a `SourceLinkGitHubHost` item. | +| GitHub Enterprise Cloud with data residency | Supported (untested) | SourceLink needs a `SourceLinkGitHubHost` item, with `ContentUrl` set to `https://raw.`. | +| GitLab | Supported (untested) | | +| Self-managed GitLab | Supported (untested) | SourceLink needs a `SourceLinkGitLabHost` item. | +| Other Git hosts | Supported with limits | Buildvana SDK does not configure SourceLink.
`bv release` creates no release page, and runs only with [`release.allowUnknownHost`](tool-commands/release.md#publishing-on-an-unknown-host). | ### CI platforms @@ -101,12 +105,12 @@ Generic CI is any other runner that the .NET CLI recognizes as one. Local is a machine where `bv` recognizes no CI runner, such as a developer machine. Every CI platform runs every command, `bv release` included. -| CI platform | Status | -| ---------------------------------------------------------------------- | --------- | -| GitHub Actions | Supported | -| GitLab CI | Supported | -| [Generic CI](environment-variables.md#variables-that-mark-a-ci-runner) | Supported | -| Local | Supported | +| CI platform | Status | +| ---------------------------------------------------------------------- | -------------------- | +| GitHub Actions | Supported | +| GitLab CI | Supported (untested) | +| [Generic CI](environment-variables.md#variables-that-mark-a-ci-runner) | Supported (untested) | +| Local | Supported (untested) | ### Toolchain diff --git a/docs/tool-commands/release.md b/docs/tool-commands/release.md index d49b4f59..b34f7b22 100644 --- a/docs/tool-commands/release.md +++ b/docs/tool-commands/release.md @@ -17,6 +17,9 @@ This page says what the command checks, what it does step by step, and what it u - [The changelog](#the-changelog) - [The post-release commit](#the-post-release-commit) - [Publishing](#publishing) + - [Publishing on GitHub](#publishing-on-github) + - [Publishing on GitLab](#publishing-on-gitlab) + - [Publishing on an unknown host](#publishing-on-an-unknown-host) - [Rollback](#rollback) - [Options](#options) - [The build configuration](#the-build-configuration) @@ -33,7 +36,7 @@ bv release [OPTIONS] `bv release` runs on every [CI platform](../introduction.md#ci-platforms), Local included. A release from a developer machine goes through the same steps as a release from a CI runner. -The Git host decides where the release is published: GitHub, or a host `bv` does not know. +The Git host decides where the release is published: GitHub, GitLab, or a host `bv` does not know. Before building anything, `bv release` checks what a build cannot change: @@ -41,8 +44,9 @@ Before building anything, `bv release` checks what a build cannot change: 2. The clone has a remote that names that repository, as [The release remote](#the-release-remote) says. 3. `bv` knows the Git host of `repository.url`, or `release.allowUnknownHost` is `true`. `bv` takes `repository.host` when it is not `auto`. - Otherwise, `github.com` is GitHub. + Otherwise, `github.com` and any host whose name ends in `.ghe.com` are GitHub, and `gitlab.com` is GitLab. Otherwise, a host whose name is the host name of [`GITHUB_SERVER_URL`](../environment-variables.md#github_server_url) is a GitHub Enterprise Server, on GitHub Actions. + Otherwise, a host whose name is the host name of [`CI_SERVER_URL`](../environment-variables.md#ci_server_url) is a self-managed GitLab instance, on GitLab CI. Any other host is unknown. 4. `HEAD` is on a branch, and the branch produces [public releases](../sdk-modules/versioning.md#public-releases) under `release.branches`. 5. A committer identity exists for the commits of the release. @@ -53,6 +57,7 @@ Before building anything, `bv release` checks what a build cannot change: `bv` commits with that identity, and writes nothing to the Git configuration. 6. The host supplies Git credentials. On GitHub, they are `x-access-token` and the token that [`hosts.github.tokenEnv`](../environment-variables.md#secret-carrying-variables-named-by-the-configuration-file) names. + On GitLab, they are a username and the token that `hosts.gitlab.tokenEnv` names, as [Publishing on GitLab](#publishing-on-gitlab) says. An unknown host supplies none. Without them, `bv` warns that the fetch and the push may fail, and goes on. 7. `bv` fetches the tags of the release remote. @@ -87,7 +92,9 @@ When no remote matches, the release fails, and the message names the remotes of ## The steps -1. On GitHub, create a provisional draft release, named ` [provisional]`, so that a token without permission fails the release early. +1. Check that the token can publish on the host, so that a token without permission fails the release early. + On GitHub, create a provisional draft release, named ` [provisional]`. + On GitLab, read the releases of the project, and check the role of the token, as [Publishing on GitLab](#publishing-on-gitlab) says. 2. Rewrite `VERSION` when the version spec change is not `none`, with the tag `versioning.prereleaseTag` names. 3. For a stable version, move the unshipped public and internal API into the shipped one. Every `PublicAPI.Unshipped.txt` next to a `PublicAPI.Shipped.txt` is emptied into it, and a line starting with `*REMOVED*` removes its API from the shipped file. @@ -106,9 +113,9 @@ When no remote matches, the release fails, and the message names the remotes of 12. Run the [`release/post-release` hook](../hooks.md#the-releasepost-release-hook), and record the files it changed. 13. Rewrite the self-references when `release.dogfood` is `true`, as [The post-release commit](#the-post-release-commit) says. 14. Create the post-release commit, when the hook or the rewrites changed a file. -15. Create the release tag, push the commits, and push the tag, as [The release tag](#the-release-tag) says. -16. Push the packages to the NuGet feed, as [Publishing](#publishing) says. -17. Collect the release assets, as [Publishing](#publishing) says. +15. Collect the release assets, and have the host check them, as [Publishing](#publishing) says. +16. Create the release tag, push the commits, and push the tag, as [The release tag](#the-release-tag) says. +17. Push the packages to the NuGet feed, as [Publishing](#publishing) says. 18. Publish the release on the host, and write the `version` step output. Every step is undone when a later step fails, as [Rollback](#rollback) says. @@ -215,9 +222,17 @@ The release assets are collected from two sources: - every `.nupkg` file, and the `.snupkg` file next to it, when one exists. A line with the wrong shape, or one naming a file that does not exist, is a warning, and the asset is skipped. +`bv` collects the assets before it pushes the commits, so that an asset the host refuses fails the release before anything leaves the clone. -On GitHub, `bv` reaches the API of the host of `repository.url`. +`bv` then publishes the release on the Git host, as the sections below say. +On GitHub Actions, `bv` appends `version=` to the file [`GITHUB_OUTPUT`](../environment-variables.md#github_output) names, for the later steps of the job. +The last line of a release is `notice: Published release with assets.` + +### Publishing on GitHub + +`bv` reaches the API of the host of `repository.url`. For `github.com`, the API is at `https://api.github.com`. +A host whose name ends in `.ghe.com` belongs to GitHub Enterprise Cloud with data residency, and its API is at `https://api.`. `bv` takes any other host for a GitHub Enterprise Server, whose API is at `https:///api/v3`. An asset upload goes to the address that GitHub states in the answer to the creation of the draft release. @@ -225,13 +240,60 @@ An asset upload goes to the address that GitHub states in the answer to the crea The release is named after the version, its tag is the release tag, and the prerelease flag follows the version. Its description is a link to `CHANGELOG.md` on the release branch, followed by the notes GitHub generates. +GitHub names each asset after its file, so two assets of a release differ in file name. +Two assets with the same file name fail the release before the push of the commits. + +### Publishing on GitLab + +`bv` reaches the API of the host of `repository.url` at `https:///api/v4`, on `gitlab.com` and on a self-managed instance alike. +`bv` recognizes a self-managed instance on its own GitLab CI runners. +On any other CI platform, set `repository.host` to `gitlab`. + +The token that `hosts.gitlab.tokenEnv` names is one of two kinds: + +- a personal, project, or group access token, which pushes with the username `oauth2`; +- the job token of GitLab CI, when `hosts.gitlab.tokenEnv` is `CI_JOB_TOKEN`, which pushes with the username `gitlab-ci-token`. + +GitLab exposes the job token through `CI_JOB_TOKEN` alone, so the name of the variable decides the kind. +The comparison is case-sensitive. +A job token copied into a variable of another name counts as an access token, and GitLab refuses it. + +The token needs the Maintainer role on the project, because a rollback deletes the generic package of the release. +With an access token, `bv` reads the role of the user of the token before the release commit. +A role below Maintainer fails the release. +A user who is not a member of the project passes the check, such as an administrator of the instance. +Such a user either can delete the package, or lacks push access and fails at the push of the release commit. +GitLab states no role for a job token, so `bv` checks none. + +A job token has the access of the user who started the job. +It pushes only when the project enables "Allow Git push requests to the repository". +The setting is under **Settings > CI/CD > Job token permissions**, and is generally available from GitLab 18.4. + +`bv` uploads each asset to the generic package registry of the project, as one file of one package. +The package is named after the repository, the last segment of the path of `repository.url`. +Its version is the release tag. +`bv` then creates the release, named after the version, on the release tag. +Each asset gets a release link of type `package`, which points at the package file. +The name of the link is the description of the asset, and its direct asset path is `/`. + +GitLab has no prerelease flag, so a prerelease looks like any other release. +GitLab generates no release notes, so the description of the release is a link to `CHANGELOG.md` on the release branch. + +GitLab accepts an asset only when its file name meets the rules of the generic package registry and of release links: + +- the file name holds only ASCII letters, digits, `.`, `_`, and `-`, and its first character may also be `+`; +- the last character is a letter or a digit; +- the file name has 3 to 127 characters. + +Two assets of a release also differ in file name and in description. +An asset that breaks a rule fails the release before the push of the commits. + +### Publishing on an unknown host + On a host `bv` does not know, there is no release page, and `bv` says so with a notice. The tag, the commits, and the packages are pushed all the same. The new changelog section gets a heading without a link, and its relative file links stay as they are, each with a notice. -On GitHub Actions, `bv` appends `version=` to the file [`GITHUB_OUTPUT`](../environment-variables.md#github_output) names, for the later steps of the job. -The last line of a release is `notice: Published release with assets.` - --- ## Rollback @@ -241,7 +303,8 @@ When a step fails, `bv release` undoes what the earlier steps did, in reverse or - it deletes the release tag from the release remote, when it was pushed, and from the clone; - it resets the branch to the commit before the release commit, post-release commit included; - when the commits were pushed, it force-pushes the previous tip; -- on GitHub, it deletes the release, draft or published. +- on GitHub, it deletes the release, draft or published; +- on GitLab, it deletes the release, then the generic package, each when it exists. A rollback action that fails is a warning, and the rollback goes on. Once the release is published and the step output is written, nothing is undone. diff --git a/schemas/buildvana.schema.json b/schemas/buildvana.schema.json index 0624c874..416a4a91 100644 --- a/schemas/buildvana.schema.json +++ b/schemas/buildvana.schema.json @@ -25,7 +25,8 @@ "description": "Kind of Git host of the repository.", "enum": [ "auto", - "github" + "github", + "gitlab" ], "default": "auto" } @@ -371,6 +372,18 @@ } }, "additionalProperties": false + }, + "gitlab": { + "description": "Configuration for the GitLab host.", + "type": "object", + "properties": { + "tokenEnv": { + "description": "Name of the environment variable that holds the GitLab access token.", + "type": "string", + "default": "GITLAB_TOKEN" + } + }, + "additionalProperties": false } }, "additionalProperties": false diff --git a/src/Buildvana.Core.Configuration/BuildvanaConfigFactory.cs b/src/Buildvana.Core.Configuration/BuildvanaConfigFactory.cs index 348753e7..eb3a8608 100644 --- a/src/Buildvana.Core.Configuration/BuildvanaConfigFactory.cs +++ b/src/Buildvana.Core.Configuration/BuildvanaConfigFactory.cs @@ -169,6 +169,7 @@ private static NuGetConfig ComposeNuGet(NuGetJsonConfig? json) private static HostsConfig ComposeHosts(HostsJsonConfig? json) => new() { GitHub = ComposeGitHub(json?.GitHub), + GitLab = ComposeGitLab(json?.GitLab), }; // A blank tokenEnv cannot name a variable, so it counts as not stated at all. @@ -180,6 +181,15 @@ private static GitHubConfig ComposeGitHub(GitHubJsonConfig? json) }; } + // A blank tokenEnv cannot name a variable, so it counts as not stated at all. + private static GitLabConfig ComposeGitLab(GitLabJsonConfig? json) + { + var defaults = new GitLabConfig(); + return new GitLabConfig { + TokenEnv = NormalizeBlankToNull(json?.TokenEnv) ?? defaults.TokenEnv, + }; + } + // The schema requires name and email whenever git.identity is stated, and forbids blank values for // them, so a wire identity that reaches the factory carries both. private static GitConfig ComposeGit(GitJsonConfig? json) diff --git a/src/Buildvana.Core.Configuration/GitLabJsonConfig.cs b/src/Buildvana.Core.Configuration/GitLabJsonConfig.cs new file mode 100644 index 00000000..bf68e5bf --- /dev/null +++ b/src/Buildvana.Core.Configuration/GitLabJsonConfig.cs @@ -0,0 +1,18 @@ +// Copyright (C) Tenacom and Contributors. Licensed under the MIT license. +// See the LICENSE file in the project root for full license information. + +using System.ComponentModel; +using JetBrains.Annotations; + +namespace Buildvana.Core.Configuration; + +/// +/// Models the hosts.gitlab section of a Buildvana configuration file. +/// +[UsedImplicitly(ImplicitUseTargetFlags.WithMembers)] +public sealed record GitLabJsonConfig +{ + /// Gets the name of the environment variable holding the GitLab access token. + [Description("Name of the environment variable that holds the GitLab access token.")] + public string? TokenEnv { get; init; } +} diff --git a/src/Buildvana.Core.Configuration/HostsJsonConfig.cs b/src/Buildvana.Core.Configuration/HostsJsonConfig.cs index b285100e..3e0ac89b 100644 --- a/src/Buildvana.Core.Configuration/HostsJsonConfig.cs +++ b/src/Buildvana.Core.Configuration/HostsJsonConfig.cs @@ -17,4 +17,9 @@ public sealed record HostsJsonConfig [JsonPropertyName("github")] [Description("Configuration for the GitHub host.")] public GitHubJsonConfig? GitHub { get; init; } + + /// Gets the hosts.gitlab section. + [JsonPropertyName("gitlab")] + [Description("Configuration for the GitLab host.")] + public GitLabJsonConfig? GitLab { get; init; } } diff --git a/src/Buildvana.Runtime/GitLabConfig.cs b/src/Buildvana.Runtime/GitLabConfig.cs new file mode 100644 index 00000000..17960652 --- /dev/null +++ b/src/Buildvana.Runtime/GitLabConfig.cs @@ -0,0 +1,19 @@ +// Copyright (C) Tenacom and Contributors. Licensed under the MIT license. +// See the LICENSE file in the project root for full license information. + +using JetBrains.Annotations; + +namespace Buildvana.Runtime; + +/// +/// The resolved configuration of the GitLab host, from the hosts.gitlab section. +/// +[UsedImplicitly(ImplicitUseTargetFlags.WithMembers)] +public sealed record GitLabConfig +{ + /// + /// Gets the name of the environment variable holding the GitLab access token. + /// The token itself is read on demand through the GetToken extension method, never stored. + /// + public string TokenEnv { get; init; } = "GITLAB_TOKEN"; +} diff --git a/src/Buildvana.Runtime/GitLabConfigExtensions.cs b/src/Buildvana.Runtime/GitLabConfigExtensions.cs new file mode 100644 index 00000000..71fe58a4 --- /dev/null +++ b/src/Buildvana.Runtime/GitLabConfigExtensions.cs @@ -0,0 +1,28 @@ +// Copyright (C) Tenacom and Contributors. Licensed under the MIT license. +// See the LICENSE file in the project root for full license information. + +namespace Buildvana.Runtime; + +/// +/// Provides extension methods for `GitLabConfig` instances. +/// +#pragma warning disable CA1034 // Nested types should not be visible — false positive on C# 14 extension blocks; fixed in .NET 11, backport to .NET 10 requested in https://github.com/dotnet/sdk/issues/53984 +#pragma warning disable CA1708 // Identifiers should differ by more than case — false positive on classes with C# 14 extension blocks; fixed in .NET 11, https://github.com/dotnet/sdk/issues/51716 +public static class GitLabConfigExtensions +{ + extension(GitLabConfig @this) + { + /// + /// Gets the GitLab access token: the value of the environment variable named by + /// , read at the moment of the call. + /// + /// The token. + /// The environment variable is missing or empty. + /// + /// The model stores the variable's name, never its value: the model is serialized into hook args + /// files, and a resolved secret must never be written to disk. This accessor is a method rather than a + /// member, so the guarantee is structural — nothing here can be serialized. + /// + public string GetToken() => EnvironmentVariables.GetRequired(@this.TokenEnv); + } +} diff --git a/src/Buildvana.Runtime/HostKind.cs b/src/Buildvana.Runtime/HostKind.cs index 1d3668a5..4de90c42 100644 --- a/src/Buildvana.Runtime/HostKind.cs +++ b/src/Buildvana.Runtime/HostKind.cs @@ -14,7 +14,13 @@ public enum HostKind [JsonStringEnumMemberName("auto")] Auto, - /// The host is GitHub: github.com, or a GitHub Enterprise Server. + /// + /// The host is GitHub: github.com, GitHub Enterprise Cloud with data residency, or a GitHub Enterprise Server. + /// [JsonStringEnumMemberName("github")] GitHub, + + /// The host is GitLab: gitlab.com, or a self-managed GitLab instance. + [JsonStringEnumMemberName("gitlab")] + GitLab, } diff --git a/src/Buildvana.Runtime/HostsConfig.cs b/src/Buildvana.Runtime/HostsConfig.cs index 29a66d5c..97efa772 100644 --- a/src/Buildvana.Runtime/HostsConfig.cs +++ b/src/Buildvana.Runtime/HostsConfig.cs @@ -18,4 +18,8 @@ public sealed record HostsConfig /// Gets the resolved GitHub configuration. [JsonPropertyName("github")] public GitHubConfig GitHub { get; init; } = new(); + + /// Gets the resolved GitLab configuration. + [JsonPropertyName("gitlab")] + public GitLabConfig GitLab { get; init; } = new(); } diff --git a/src/Buildvana.Runtime/PublicAPI.Unshipped.txt b/src/Buildvana.Runtime/PublicAPI.Unshipped.txt index 0aa8441e..fa46a5f6 100644 --- a/src/Buildvana.Runtime/PublicAPI.Unshipped.txt +++ b/src/Buildvana.Runtime/PublicAPI.Unshipped.txt @@ -154,6 +154,15 @@ Buildvana.Runtime.GitIdentityConfig.Equals(Buildvana.Runtime.GitIdentityConfig? Buildvana.Runtime.GitIdentityConfig.GitIdentityConfig() -> void Buildvana.Runtime.GitIdentityConfig.Name.get -> string! Buildvana.Runtime.GitIdentityConfig.Name.init -> void +Buildvana.Runtime.GitLabConfig +Buildvana.Runtime.GitLabConfig.$() -> Buildvana.Runtime.GitLabConfig! +Buildvana.Runtime.GitLabConfig.Equals(Buildvana.Runtime.GitLabConfig? other) -> bool +Buildvana.Runtime.GitLabConfig.GitLabConfig() -> void +Buildvana.Runtime.GitLabConfig.TokenEnv.get -> string! +Buildvana.Runtime.GitLabConfig.TokenEnv.init -> void +Buildvana.Runtime.GitLabConfigExtensions +Buildvana.Runtime.GitLabConfigExtensions.extension(Buildvana.Runtime.GitLabConfig!) +Buildvana.Runtime.GitLabConfigExtensions.extension(Buildvana.Runtime.GitLabConfig!).GetToken() -> string! Buildvana.Runtime.HookArgs Buildvana.Runtime.HookArgs.HookArgs() -> void Buildvana.Runtime.HookArgs.HookArgs(Buildvana.Runtime.HookArgs! original) -> void @@ -162,11 +171,14 @@ Buildvana.Runtime.HookArgs.RuntimeInfo.init -> void Buildvana.Runtime.HostKind Buildvana.Runtime.HostKind.Auto = 0 -> Buildvana.Runtime.HostKind Buildvana.Runtime.HostKind.GitHub = 1 -> Buildvana.Runtime.HostKind +Buildvana.Runtime.HostKind.GitLab = 2 -> Buildvana.Runtime.HostKind Buildvana.Runtime.HostsConfig Buildvana.Runtime.HostsConfig.$() -> Buildvana.Runtime.HostsConfig! Buildvana.Runtime.HostsConfig.Equals(Buildvana.Runtime.HostsConfig? other) -> bool Buildvana.Runtime.HostsConfig.GitHub.get -> Buildvana.Runtime.GitHubConfig! Buildvana.Runtime.HostsConfig.GitHub.init -> void +Buildvana.Runtime.HostsConfig.GitLab.get -> Buildvana.Runtime.GitLabConfig! +Buildvana.Runtime.HostsConfig.GitLab.init -> void Buildvana.Runtime.HostsConfig.HostsConfig() -> void Buildvana.Runtime.IHookEvent Buildvana.Runtime.IHookEvent.Context.get -> string! @@ -350,6 +362,9 @@ override Buildvana.Runtime.GitHubConfig.ToString() -> string! override Buildvana.Runtime.GitIdentityConfig.Equals(object? obj) -> bool override Buildvana.Runtime.GitIdentityConfig.GetHashCode() -> int override Buildvana.Runtime.GitIdentityConfig.ToString() -> string! +override Buildvana.Runtime.GitLabConfig.Equals(object? obj) -> bool +override Buildvana.Runtime.GitLabConfig.GetHashCode() -> int +override Buildvana.Runtime.GitLabConfig.ToString() -> string! override Buildvana.Runtime.HookArgs.Equals(object? obj) -> bool override Buildvana.Runtime.HookArgs.GetHashCode() -> int override Buildvana.Runtime.HookArgs.ToString() -> string! @@ -420,6 +435,9 @@ static Buildvana.Runtime.GitHubConfig.operator ==(Buildvana.Runtime.GitHubConfig static Buildvana.Runtime.GitHubConfigExtensions.GetToken(this Buildvana.Runtime.GitHubConfig! this) -> string! static Buildvana.Runtime.GitIdentityConfig.operator !=(Buildvana.Runtime.GitIdentityConfig? left, Buildvana.Runtime.GitIdentityConfig? right) -> bool static Buildvana.Runtime.GitIdentityConfig.operator ==(Buildvana.Runtime.GitIdentityConfig? left, Buildvana.Runtime.GitIdentityConfig? right) -> bool +static Buildvana.Runtime.GitLabConfig.operator !=(Buildvana.Runtime.GitLabConfig? left, Buildvana.Runtime.GitLabConfig? right) -> bool +static Buildvana.Runtime.GitLabConfig.operator ==(Buildvana.Runtime.GitLabConfig? left, Buildvana.Runtime.GitLabConfig? right) -> bool +static Buildvana.Runtime.GitLabConfigExtensions.GetToken(this Buildvana.Runtime.GitLabConfig! this) -> string! static Buildvana.Runtime.HookArgs.Load(System.Text.Json.Serialization.Metadata.JsonTypeInfo! jsonTypeInfo, string? homeDirectory) -> TArgs! static Buildvana.Runtime.HookArgs.operator !=(Buildvana.Runtime.HookArgs? left, Buildvana.Runtime.HookArgs? right) -> bool static Buildvana.Runtime.HookArgs.operator ==(Buildvana.Runtime.HookArgs? left, Buildvana.Runtime.HookArgs? right) -> bool @@ -472,6 +490,7 @@ virtual Buildvana.Runtime.HookArgs.PrintMembers(System.Text.StringBuilder! build ~Buildvana.Runtime.BuildvanaJsonContext.GitConfig.get -> System.Text.Json.Serialization.Metadata.JsonTypeInfo ~Buildvana.Runtime.BuildvanaJsonContext.GitHubConfig.get -> System.Text.Json.Serialization.Metadata.JsonTypeInfo ~Buildvana.Runtime.BuildvanaJsonContext.GitIdentityConfig.get -> System.Text.Json.Serialization.Metadata.JsonTypeInfo +~Buildvana.Runtime.BuildvanaJsonContext.GitLabConfig.get -> System.Text.Json.Serialization.Metadata.JsonTypeInfo ~Buildvana.Runtime.BuildvanaJsonContext.HostKind.get -> System.Text.Json.Serialization.Metadata.JsonTypeInfo ~Buildvana.Runtime.BuildvanaJsonContext.HostsConfig.get -> System.Text.Json.Serialization.Metadata.JsonTypeInfo ~Buildvana.Runtime.BuildvanaJsonContext.IReadOnlyDictionaryStringString.get -> System.Text.Json.Serialization.Metadata.JsonTypeInfo> diff --git a/src/Buildvana.Tool/Services/Hosts/GitHost.cs b/src/Buildvana.Tool/Services/Hosts/GitHost.cs index 3da11e00..7549bcf3 100644 --- a/src/Buildvana.Tool/Services/Hosts/GitHost.cs +++ b/src/Buildvana.Tool/Services/Hosts/GitHost.cs @@ -9,6 +9,7 @@ using Buildvana.Runtime; using Buildvana.Tool.Services.Git; using Buildvana.Tool.Services.Hosts.GitHub; +using Buildvana.Tool.Services.Hosts.GitLab; using Buildvana.Tool.Services.Platforms; using CommunityToolkit.Diagnostics; using Microsoft.Extensions.DependencyInjection; @@ -90,6 +91,7 @@ public static GitHost Create(IServiceProvider services) var url = ParseRepositoryUrl(config.Repository); return DetectKind(config.Repository, url, services.GetRequiredService()) switch { HostKind.GitHub => new GitHubHost(services, url), + HostKind.GitLab => new GitLabHost(services, url), _ => new UnknownHost(url, services.GetRequiredService()), }; } @@ -119,7 +121,8 @@ public static GitUrlInfo ParseRepositoryUrl(RepositoryConfig repository) /// The parsed URL of the repository. /// The CI platform bv runs on. /// The first of these that applies: when it is not - /// ; the kind of a public host that names; the + /// ; the kind of a host that GitHub or GitLab runs, such as github.com, when + /// names it; the /// of a platform whose has the host /// name of ; otherwise, . public static HostKind? DetectKind(RepositoryConfig repository, GitUrlInfo url, CIPlatform platform) @@ -132,12 +135,21 @@ public static GitUrlInfo ParseRepositoryUrl(RepositoryConfig repository) return repository.Host; } - if (string.Equals(url.Host, "github.com", StringComparison.OrdinalIgnoreCase)) + // A host whose name ends in .ghe.com belongs to GitHub Enterprise Cloud with data residency. + var isGitHub = string.Equals(url.Host, "github.com", StringComparison.OrdinalIgnoreCase) + || url.Host.EndsWith(".ghe.com", StringComparison.OrdinalIgnoreCase); + if (isGitHub) { return HostKind.GitHub; } - // A GitHub Enterprise Server, or a self-hosted server of another kind, is recognized on its own runners. + if (string.Equals(url.Host, "gitlab.com", StringComparison.OrdinalIgnoreCase)) + { + return HostKind.GitLab; + } + + // A self-hosted server, such as a GitHub Enterprise Server or a self-managed GitLab instance, is recognized + // on its own runners. var isPlatformServer = string.Equals(platform.ServerUrl?.Host, url.Host, StringComparison.OrdinalIgnoreCase); return isPlatformServer ? platform.ServerKind : null; } diff --git a/src/Buildvana.Tool/Services/Hosts/GitHub/GitHubApiClient.cs b/src/Buildvana.Tool/Services/Hosts/GitHub/GitHubApiClient.cs index f0e52670..ba820290 100644 --- a/src/Buildvana.Tool/Services/Hosts/GitHub/GitHubApiClient.cs +++ b/src/Buildvana.Tool/Services/Hosts/GitHub/GitHubApiClient.cs @@ -8,9 +8,7 @@ using System.Net; using System.Net.Http; using System.Net.Http.Headers; -using System.Text; using System.Text.Json; -using System.Text.Json.Serialization.Metadata; using System.Threading; using System.Threading.Tasks; using Buildvana.Core; @@ -23,14 +21,16 @@ namespace Buildvana.Tool.Services.Hosts.GitHub; /// Performs the operations of the GitHub REST API that bv needs. /// /// -/// The API of github.com is at https://api.github.com. Any other host is taken for a -/// GitHub Enterprise Server, whose API is at https://<host>/api/v3. +/// The API of github.com is at https://api.github.com. A host whose name ends in +/// .ghe.com belongs to GitHub Enterprise Cloud with data residency, and its API is at +/// https://api.<host>. Any other host is taken for a GitHub Enterprise Server, whose API is at +/// https://<host>/api/v3. /// An asset upload goes to the address that the release states in its upload_url member, so this /// type derives no upload host. -/// An answer outside the 2xx range fails the build, with the status code and the message of the answer. -/// The one exception is a tag lookup, where a 404 means that the tag does not exist. +/// An answer outside the 2xx range fails the build, as says. The one exception +/// is a tag lookup, where a 404 means that the tag does not exist. /// -internal sealed class GitHubApiClient : IDisposable +internal sealed class GitHubApiClient : HostApiClient { /// /// The version of the REST API that every request asks for. @@ -44,9 +44,10 @@ internal sealed class GitHubApiClient : IDisposable public const string ApiVersion = "2022-11-28"; private const string GitHubComHostName = "github.com"; + private const string GheComHostSuffix = ".ghe.com"; private const string GitHubMediaType = "application/vnd.github+json"; - private readonly HttpClient _http; + private readonly AuthenticationHeaderValue _authorization; /// /// Initializes a new instance of the class. @@ -56,25 +57,17 @@ internal sealed class GitHubApiClient : IDisposable /// The handler that sends the requests. It is disposed together with this /// instance. public GitHubApiClient(string hostName, string token, HttpMessageHandler handler) + : base("GitHub", GetBaseUrl(hostName), handler) { - Guard.IsNotNullOrEmpty(hostName); Guard.IsNotNullOrEmpty(token); - Guard.IsNotNull(handler); - BaseUrl = GetBaseUrl(hostName); - _http = new HttpClient(handler); - var headers = _http.DefaultRequestHeaders; + _authorization = new AuthenticationHeaderValue("Bearer", token); + var headers = DefaultRequestHeaders; headers.Accept.Add(new MediaTypeWithQualityHeaderValue(GitHubMediaType)); headers.Add("X-GitHub-Api-Version", ApiVersion); - headers.Authorization = new AuthenticationHeaderValue("Bearer", token); headers.UserAgent.Add(new ProductInfoHeaderValue("bv", OwnVersion.Value.ToNormalizedString())); } - /// - /// Gets the base URL of the API, with a trailing slash. - /// - public Uri BaseUrl { get; } - /// /// Asynchronously gets a repository. /// @@ -86,7 +79,12 @@ public Task GetRepositoryAsync( string owner, string name, CancellationToken cancellationToken = default) - => SendAsync(HttpMethod.Get, ApiUrl(RepositoryPath(owner, name)), null, GitHubJsonContext.Default.GitHubRepository, cancellationToken); + => SendAsync( + HttpMethod.Get, + ApiUrl(RepositoryPath(owner, name)), + null, + GitHubJsonContext.Default.GitHubRepository, + cancellationToken); /// /// Asynchronously creates a release. @@ -194,7 +192,7 @@ public async Task TagExistsAsync( CancellationToken cancellationToken = default) { var url = ApiUrl(RepositoryPath(owner, name) + "/git/ref/tags/" + EscapeRefName(tagName)); - using var request = new HttpRequestMessage(HttpMethod.Get, url); + using var request = CreateRequest(HttpMethod.Get, url); using var response = await SendCoreAsync(request, url, cancellationToken).ConfigureAwait(false); if (response.StatusCode == HttpStatusCode.NotFound) { @@ -253,13 +251,46 @@ public Task UploadReleaseAssetAsync( return SendAsync(HttpMethod.Post, url, content, cancellationToken); } - /// - public void Dispose() => _http.Dispose(); + /// + protected override void Authenticate(HttpRequestMessage request) + { + Guard.IsNotNull(request); + request.Headers.Authorization = _authorization; + } + + /// + protected override async Task ReadErrorMessageAsync( + HttpResponseMessage response, + CancellationToken cancellationToken) + { + // A GitHub error answer is JSON with a "message" member. A proxy in front of a server may answer with + // anything else, and then the status code is the whole report. + Guard.IsNotNull(response); + GitHubErrorBody? body; + try + { + var stream = await response.Content.ReadAsStreamAsync(cancellationToken).ConfigureAwait(false); + await using (stream.ConfigureAwait(false)) + { + body = await JsonSerializer.DeserializeAsync(stream, GitHubJsonContext.Default.GitHubErrorBody, cancellationToken) + .ConfigureAwait(false); + } + } + catch (JsonException) + { + return null; + } + + return body?.Message is { Length: > 0 } message ? message : null; + } private static Uri GetBaseUrl(string hostName) - => string.Equals(hostName, GitHubComHostName, StringComparison.OrdinalIgnoreCase) - ? new Uri("https://api.github.com/") + { + Guard.IsNotNullOrEmpty(hostName); + return string.Equals(hostName, GitHubComHostName, StringComparison.OrdinalIgnoreCase) ? new Uri("https://api.github.com/") + : hostName.EndsWith(GheComHostSuffix, StringComparison.OrdinalIgnoreCase) ? new Uri($"https://api.{hostName}/") : new Uri($"https://{hostName}/api/v3/"); + } private static string ReleasePath(string owner, string name, long releaseId) => RepositoryPath(owner, name) + "/releases/" + releaseId.ToString(CultureInfo.InvariantCulture); @@ -298,131 +329,4 @@ private static Uri GetUploadUrl(GitHubRelease release, string fileName, string? return url; } - - private static StringContent JsonBody(T value, JsonTypeInfo typeInfo) - => new(JsonSerializer.Serialize(value, typeInfo), Encoding.UTF8, "application/json"); - - private static async Task EnsureSuccessAsync( - HttpMethod method, - Uri url, - HttpResponseMessage response, - CancellationToken cancellationToken) - { - if (response.IsSuccessStatusCode) - { - return; - } - - var message = await ReadErrorMessageAsync(response, cancellationToken).ConfigureAwait(false); - var status = string.Create(CultureInfo.InvariantCulture, $"{(int)response.StatusCode} {response.ReasonPhrase}"); - throw new BuildFailedException(message is null - ? $"The GitHub API answered {Describe(method, url)} with {status}." - : $"The GitHub API answered {Describe(method, url)} with {status}: {message}"); - } - - // A GitHub error answer is JSON with a "message" member. A proxy in front of a server may answer with - // anything else, and then the status code is the whole report. - private static async Task ReadErrorMessageAsync( - HttpResponseMessage response, - CancellationToken cancellationToken) - { - GitHubErrorBody? body; - try - { - var stream = await response.Content.ReadAsStreamAsync(cancellationToken).ConfigureAwait(false); - await using (stream.ConfigureAwait(false)) - { - body = await JsonSerializer.DeserializeAsync(stream, GitHubJsonContext.Default.GitHubErrorBody, cancellationToken) - .ConfigureAwait(false); - } - } - catch (JsonException) - { - return null; - } - - return body?.Message is { Length: > 0 } message ? message : null; - } - - // The client buffers every answer before SendAsync returns, so reading the body here meets no network. - private static async Task ReadAsync( - HttpMethod method, - Uri url, - HttpResponseMessage response, - JsonTypeInfo typeInfo, - CancellationToken cancellationToken) - where T : class - { - T? result; - try - { - var stream = await response.Content.ReadAsStreamAsync(cancellationToken).ConfigureAwait(false); - await using (stream.ConfigureAwait(false)) - { - result = await JsonSerializer.DeserializeAsync(stream, typeInfo, cancellationToken).ConfigureAwait(false); - } - } - catch (JsonException exception) - { - throw new BuildFailedException( - $"The GitHub API answered {Describe(method, url)} with something bv cannot read: {exception.Message}", - exception); - } - - return result ?? throw new BuildFailedException($"The GitHub API answered {Describe(method, url)} with null."); - } - - private static string Describe(HttpMethod method, Uri url) => $"{method} {url.AbsoluteUri}"; - - // Every request goes to an absolute URL that this type holds and passes along with the request, so that a - // diagnostic names the URL without reading back the nullable HttpRequestMessage.RequestUri. - private Uri ApiUrl(string path) => new(BaseUrl, path); - - private async Task SendAsync( - HttpMethod method, - Uri url, - HttpContent? content, - JsonTypeInfo typeInfo, - CancellationToken cancellationToken) - where T : class - { - using var request = new HttpRequestMessage(method, url); - request.Content = content; - using var response = await SendCoreAsync(request, url, cancellationToken).ConfigureAwait(false); - await EnsureSuccessAsync(method, url, response, cancellationToken).ConfigureAwait(false); - return await ReadAsync(method, url, response, typeInfo, cancellationToken).ConfigureAwait(false); - } - - private async Task SendAsync( - HttpMethod method, - Uri url, - HttpContent? content, - CancellationToken cancellationToken) - { - using var request = new HttpRequestMessage(method, url); - request.Content = content; - using var response = await SendCoreAsync(request, url, cancellationToken).ConfigureAwait(false); - await EnsureSuccessAsync(method, url, response, cancellationToken).ConfigureAwait(false); - } - - private async Task SendCoreAsync( - HttpRequestMessage request, - Uri url, - CancellationToken cancellationToken) - { - try - { - return await _http.SendAsync(request, cancellationToken).ConfigureAwait(false); - } - catch (HttpRequestException exception) - { - throw new BuildFailedException( - $"The GitHub API could not be reached for {Describe(request.Method, url)}: {exception.Message}", - exception); - } - catch (TaskCanceledException exception) when (!cancellationToken.IsCancellationRequested) - { - throw new BuildFailedException($"The GitHub API did not answer {Describe(request.Method, url)} in time.", exception); - } - } } diff --git a/src/Buildvana.Tool/Services/Hosts/GitHub/GitHubHost.cs b/src/Buildvana.Tool/Services/Hosts/GitHub/GitHubHost.cs index 62b5f294..e0f1e126 100644 --- a/src/Buildvana.Tool/Services/Hosts/GitHub/GitHubHost.cs +++ b/src/Buildvana.Tool/Services/Hosts/GitHub/GitHubHost.cs @@ -17,7 +17,7 @@ namespace Buildvana.Tool.Services.Hosts.GitHub; /// -/// The GitHub host: github.com, or a GitHub Enterprise Server. +/// The GitHub host: github.com, GitHub Enterprise Cloud with data residency, or a GitHub Enterprise Server. /// internal sealed class GitHubHost : GitHost, IDisposable { diff --git a/src/Buildvana.Tool/Services/Hosts/GitHub/GitHubReleasePublisher.cs b/src/Buildvana.Tool/Services/Hosts/GitHub/GitHubReleasePublisher.cs index 720fc3b3..3d195841 100644 --- a/src/Buildvana.Tool/Services/Hosts/GitHub/GitHubReleasePublisher.cs +++ b/src/Buildvana.Tool/Services/Hosts/GitHub/GitHubReleasePublisher.cs @@ -4,7 +4,9 @@ using System.Collections.Generic; using System.Globalization; using System.IO; +using System.Linq; using System.Threading.Tasks; +using Buildvana.Core; using Buildvana.Core.ConsoleOutput; using Buildvana.Core.IO; using Buildvana.Tool.Services.Git; @@ -46,6 +48,23 @@ public GitHubReleasePublisher(GitHubHost host, IReporter reporter, VersionServic _git = git; } + /// + /// Two assets have the same file name. + /// + /// GitHub names an asset after its file, and refuses two assets of a release with the same name. + /// + public override void CheckAssets(IReadOnlyList assets) + { + Guard.IsNotNull(assets); + var problems = Duplicates(assets.Select(static x => Path.GetFileName(x.Path))) + .Select(static x => $"More than one asset has the file name '{x}'.") + .ToList(); + + BuildFailedException.ThrowIf( + problems.Count > 0, + "GitHub refuses the release assets:" + string.Concat(problems.Select(static x => "\n- " + x))); + } + /// /// /// The draft is named after the version as it stands before the release commit, which may still move it. diff --git a/src/Buildvana.Tool/Services/Hosts/GitHub/GitHubRepositoryUrls.cs b/src/Buildvana.Tool/Services/Hosts/GitHub/GitHubRepositoryUrls.cs index c8cf0874..0287207f 100644 --- a/src/Buildvana.Tool/Services/Hosts/GitHub/GitHubRepositoryUrls.cs +++ b/src/Buildvana.Tool/Services/Hosts/GitHub/GitHubRepositoryUrls.cs @@ -2,7 +2,6 @@ // See the LICENSE file in the project root for full license information. using System; -using System.IO; using CommunityToolkit.Diagnostics; namespace Buildvana.Tool.Services.Hosts.GitHub; @@ -62,26 +61,11 @@ public Uri ReleaseTag(string version) /// The path to the file, relative to the repository root. /// The SHA or reference to which the file belongs. /// The URL of the file. + /// is rooted, or leaves the repository, as + /// says. public Uri File(string path, string commitish) { - Guard.IsNotNullOrEmpty(path); Guard.IsNotNullOrEmpty(commitish); - - // Normalize to forward slashes for the URL, then reject paths that are rooted or escape the repo. - // A rooted path is checked after normalization, so that a leading slash of either kind is rejected on - // every platform: Path.IsPathFullyQualified is true for "/docs/x.md" on Linux and false on Windows, - // where a fully qualified path needs a drive or a UNC share, and a backslash is a separator only there. - // Every ".." segment must go, not just a leading one: Uri collapses parent segments as it parses, - // so enough of them anywhere in the path walk out of the repository and even out of the owner - // (".../blob/main/docs/../../../../../../etc/passwd" parses to "https://github.com/etc/passwd"). - var remotePath = path.Replace('\\', '/'); - Guard.IsTrue(!Path.IsPathRooted(remotePath), nameof(path), "A path must be relative to be converted to a file URL."); - var hasParentSegment = remotePath == ".." - || remotePath.StartsWith("../", StringComparison.Ordinal) - || remotePath.EndsWith("/..", StringComparison.Ordinal) - || remotePath.Contains("/../", StringComparison.Ordinal); - Guard.IsTrue(!hasParentSegment, nameof(path), "Only a path to a file in the repository can be converted to a file URL."); - - return new Uri($"{_repository}/blob/{commitish}/{remotePath}"); + return new Uri($"{_repository}/blob/{commitish}/{RepositoryFilePath.ToUrlPath(path)}"); } } diff --git a/src/Buildvana.Tool/Services/Hosts/GitLab/GitLabApiClient.cs b/src/Buildvana.Tool/Services/Hosts/GitLab/GitLabApiClient.cs new file mode 100644 index 00000000..ee1a237b --- /dev/null +++ b/src/Buildvana.Tool/Services/Hosts/GitLab/GitLabApiClient.cs @@ -0,0 +1,409 @@ +// Copyright (C) Tenacom and Contributors. Licensed under the MIT license. +// See the LICENSE file in the project root for full license information. + +using System; +using System.Collections.Generic; +using System.Globalization; +using System.IO; +using System.Linq; +using System.Net; +using System.Net.Http; +using System.Net.Http.Headers; +using System.Text.Json; +using System.Text.Json.Serialization.Metadata; +using System.Text.RegularExpressions; +using System.Threading; +using System.Threading.Tasks; +using Buildvana.Core; +using Buildvana.Tool.Utilities; +using CommunityToolkit.Diagnostics; + +namespace Buildvana.Tool.Services.Hosts.GitLab; + +/// +/// Performs the operations of the GitLab REST API that bv needs. +/// +/// +/// The API of a host is at https://<host>/api/v4, on gitlab.com and on a self-managed +/// instance alike. +/// Every operation addresses a project by its path, such as group/subgroup/project. The client +/// escapes the path as one segment of the URL, as the API expects. +/// An answer outside the 2xx range fails the build, as says. The exceptions +/// are , where a 404 means that the project is not public, +/// , where a 404 means that the user is not a member, and +/// , where a 404 means that the release does not exist. +/// +internal sealed partial class GitLabApiClient : HostApiClient +{ + private const string AccessTokenHeader = "PRIVATE-TOKEN"; + private const string JobTokenHeader = "JOB-TOKEN"; + private const int DirectAssetPathMaxLength = 128; + + private readonly string _tokenHeader; + private readonly string _token; + + /// + /// Initializes a new instance of the class. + /// + /// The host name of the project, such as gitlab.com. + /// The token that authenticates every request. + /// The kind of . + /// The handler that sends the requests. It is disposed together with this + /// instance. + public GitLabApiClient(string hostName, string token, GitLabTokenKind tokenKind, HttpMessageHandler handler) + : base("GitLab", GetBaseUrl(hostName), handler) + { + Guard.IsNotNullOrEmpty(token); + + _tokenHeader = tokenKind switch { + GitLabTokenKind.AccessToken => AccessTokenHeader, + GitLabTokenKind.JobToken => JobTokenHeader, + _ => ThrowHelper.ThrowArgumentOutOfRangeException(nameof(tokenKind), tokenKind, "Unknown token kind."), + }; + + _token = token; + var headers = DefaultRequestHeaders; + headers.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json")); + headers.UserAgent.Add(new ProductInfoHeaderValue("bv", OwnVersion.Value.ToNormalizedString())); + } + + /// + /// Gets a value indicating whether a file name is valid in the generic package registry. + /// + /// The file name. + /// when holds only ASCII letters, digits, and + /// . _ - + ~ @, and neither starts nor ends with ~ or @; otherwise, + /// . + /// + /// The rule is generic_package_file_name_regex in lib/gitlab/regex/packages.rb of + /// GitLab. + /// + public static bool IsValidPackageFileName(string fileName) + { + Guard.IsNotNull(fileName); + return fileName is { Length: > 0 } + && fileName[0] is not ('~' or '@') + && fileName[^1] is not ('~' or '@') + && fileName.All(static c => char.IsAsciiLetterOrDigit(c) || c is '.' or '_' or '-' or '+' or '~' or '@'); + } + + /// + /// Gets a value indicating whether a path is valid as the direct_asset_path of a release link. + /// + /// The path, such as /Foo.1.0.0.nupkg. + /// when is at most 128 characters long, starts + /// with a single /, holds only ASCII letters, digits, and . _ - / after its second character, + /// holds no //, and ends with a letter or a digit; otherwise, . + /// + /// The rule is FILEPATH_REGEX and FILEPATH_MAX_LENGTH in + /// app/models/releases/link.rb of GitLab. + /// + public static bool IsValidDirectAssetPath(string path) + { + Guard.IsNotNull(path); + return path.Length <= DirectAssetPathMaxLength && GetDirectAssetPathRegex().IsMatch(path); + } + + /// + /// Asynchronously gets the user the token belongs to. + /// + /// A token that cancels the operation. + /// A whose result is the user. + /// + /// A job token cannot read a user. + /// + public Task GetCurrentUserAsync(CancellationToken cancellationToken = default) + => SendAsync(HttpMethod.Get, ApiUrl("user"), null, GitLabJsonContext.Default.GitLabUser, cancellationToken); + + /// + /// Asynchronously gets a project. + /// + /// The path of the project. + /// A token that cancels the operation. + /// A whose result is the project. + /// + /// A job token cannot read a project. Use with one. + /// + public Task GetProjectAsync(string projectPath, CancellationToken cancellationToken = default) + => SendAsync(HttpMethod.Get, ApiUrl(ProjectPath(projectPath)), null, GitLabJsonContext.Default.GitLabProject, cancellationToken); + + /// + /// Asynchronously gets a project with a request that carries no credentials. + /// + /// The path of the project. + /// A token that cancels the operation. + /// A whose result is the project, or when the API answers + /// 404. + /// + /// GitLab answers 404 for a project that an anonymous request cannot see, so a public project is the + /// only one this method returns. + /// + public async Task GetProjectWithoutCredentialsAsync( + string projectPath, + CancellationToken cancellationToken = default) + { + var url = ApiUrl(ProjectPath(projectPath)); + using var request = new HttpRequestMessage(HttpMethod.Get, url); + return await SendOrNullAsync(request, url, GitLabJsonContext.Default.GitLabProject, cancellationToken) + .ConfigureAwait(false); + } + + /// + /// Asynchronously gets the membership of a user in a project. + /// + /// The path of the project. + /// The identifier of the user. + /// A token that cancels the operation. + /// A whose result is the membership, or when the API answers + /// 404. + /// + /// The endpoint is members/all, which counts a direct membership, a membership of a parent group, + /// and a membership of a group the project or a parent group is shared with. The access level it states is + /// the one GitLab enforces on the project. + /// GitLab answers 404 for a user with no membership, such as an instance administrator, and for a + /// project the token cannot see. + /// A job token cannot read a membership. + /// + public async Task GetProjectMemberAsync( + string projectPath, + long userId, + CancellationToken cancellationToken = default) + { + var url = ApiUrl(ProjectPath(projectPath) + "/members/all/" + userId.ToString(CultureInfo.InvariantCulture)); + using var request = CreateRequest(HttpMethod.Get, url); + return await SendOrNullAsync(request, url, GitLabJsonContext.Default.GitLabMember, cancellationToken) + .ConfigureAwait(false); + } + + /// + /// Asynchronously gets the latest releases of a project. + /// + /// The path of the project. + /// The number of releases to get, from 1 to 100. + /// A token that cancels the operation. + /// A whose result is the releases, latest first. The list holds fewer than + /// releases when the project has fewer. + public Task> GetReleasesAsync( + string projectPath, + int count, + CancellationToken cancellationToken = default) + { + Guard.IsInRange(count, 1, 101); + var query = string.Create(CultureInfo.InvariantCulture, $"?per_page={count}"); + return SendAsync( + HttpMethod.Get, + ApiUrl(ProjectPath(projectPath) + "/releases" + query), + null, + GitLabJsonContext.Default.IReadOnlyListGitLabRelease, + cancellationToken); + } + + /// + /// Asynchronously creates a release. + /// + /// The path of the project. + /// The members of the new release. + /// A token that cancels the operation. + /// A whose result is the created release. + public Task CreateReleaseAsync( + string projectPath, + GitLabNewRelease release, + CancellationToken cancellationToken = default) + { + Guard.IsNotNull(release); + return SendAsync( + HttpMethod.Post, + ApiUrl(ProjectPath(projectPath) + "/releases"), + JsonBody(release, GitLabJsonContext.Default.GitLabNewRelease), + GitLabJsonContext.Default.GitLabRelease, + cancellationToken); + } + + /// + /// Asynchronously deletes a release. + /// + /// The path of the project. + /// The name of the tag of the release. + /// A token that cancels the operation. + /// A whose result is when the release was deleted, and + /// when the API answers 404. + /// + /// The tag of the release stays. + /// GitLab deletes a release together with its tag, so a release whose tag was deleted gets a 404. + /// + public async Task DeleteReleaseAsync( + string projectPath, + string tagName, + CancellationToken cancellationToken = default) + { + Guard.IsNotNullOrEmpty(tagName); + var url = ApiUrl(ProjectPath(projectPath) + "/releases/" + Uri.EscapeDataString(tagName)); + using var request = CreateRequest(HttpMethod.Delete, url); + using var response = await SendCoreAsync(request, url, cancellationToken).ConfigureAwait(false); + if (response.StatusCode == HttpStatusCode.NotFound) + { + return false; + } + + await EnsureSuccessAsync(HttpMethod.Delete, url, response, cancellationToken).ConfigureAwait(false); + return true; + } + + /// + /// Asynchronously uploads a file of a generic package, and creates the package when it does not exist. + /// + /// The path of the project. + /// The name of the package. + /// The version of the package. + /// The file name. + /// The contents of the file. The stream is disposed together with the request. + /// A token that cancels the operation. + /// A whose result is the uploaded file. + /// is not valid in the generic package + /// registry, as says. + public Task UploadPackageFileAsync( + string projectPath, + string packageName, + string packageVersion, + string fileName, + Stream contents, + CancellationToken cancellationToken = default) + { + Guard.IsNotNull(contents); + BuildFailedException.ThrowIfNot( + IsValidPackageFileName(fileName), + $"'{fileName}' is not a valid file name in the GitLab generic package registry."); + + // The answer of a plain upload has no body. With select=package_file, it holds the uploaded file. + var url = new Uri(GetPackageFileUrl(projectPath, packageName, packageVersion, fileName).AbsoluteUri + "?select=package_file"); + var content = new StreamContent(contents); + content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream"); + return SendAsync(HttpMethod.Put, url, content, GitLabJsonContext.Default.GitLabPackageFile, cancellationToken); + } + + /// + /// Gets the URL of a file of a generic package, which is also the URL a file is uploaded to. + /// + /// The path of the project. + /// The name of the package. + /// The version of the package. + /// The file name. + /// The URL of the file. + public Uri GetPackageFileUrl(string projectPath, string packageName, string packageVersion, string fileName) + { + Guard.IsNotNullOrEmpty(packageName); + Guard.IsNotNullOrEmpty(packageVersion); + Guard.IsNotNullOrEmpty(fileName); + var path = $"{ProjectPath(projectPath)}/packages/generic/{Uri.EscapeDataString(packageName)}/" + + $"{Uri.EscapeDataString(packageVersion)}/{Uri.EscapeDataString(fileName)}"; + return ApiUrl(path); + } + + /// + /// Asynchronously deletes a package, with all its files. + /// + /// The path of the project. + /// The identifier of the package. + /// A token that cancels the operation. + /// A representing the ongoing operation. + public Task DeletePackageAsync(string projectPath, long packageId, CancellationToken cancellationToken = default) + => SendAsync( + HttpMethod.Delete, + ApiUrl(ProjectPath(projectPath) + "/packages/" + packageId.ToString(CultureInfo.InvariantCulture)), + null, + cancellationToken); + + /// + protected override void Authenticate(HttpRequestMessage request) + { + Guard.IsNotNull(request); + request.Headers.Add(_tokenHeader, _token); + } + + /// + protected override async Task ReadErrorMessageAsync( + HttpResponseMessage response, + CancellationToken cancellationToken) + { + // A GitLab error answer is JSON. Its "message" member is a string, an array of strings, or an object + // that maps each member of the request to its errors. Some answers state an "error" member instead. + // A proxy in front of a server may answer with anything else, and then the status code is the whole + // report. + Guard.IsNotNull(response); + JsonDocument document; + try + { + var stream = await response.Content.ReadAsStreamAsync(cancellationToken).ConfigureAwait(false); + await using (stream.ConfigureAwait(false)) + { + document = await JsonDocument.ParseAsync(stream, default, cancellationToken).ConfigureAwait(false); + } + } + catch (JsonException) + { + return null; + } + + using (document) + { + var root = document.RootElement; + if (root.ValueKind != JsonValueKind.Object) + { + return null; + } + + return Quote(root, "message") ?? Quote(root, "error"); + } + } + + // FILEPATH_REGEX of GitLab, with the ASCII class Ruby gives \w spelled out. + [GeneratedRegex(@"\A/[^/](?!.*//)[-.A-Za-z0-9_/]+[0-9A-Za-z]+\z", RegexOptions.CultureInvariant)] + private static partial Regex GetDirectAssetPathRegex(); + + private static Uri GetBaseUrl(string hostName) + { + Guard.IsNotNullOrEmpty(hostName); + return new Uri($"https://{hostName}/api/v4/"); + } + + private static string ProjectPath(string projectPath) + { + Guard.IsNotNullOrEmpty(projectPath); + return "projects/" + Uri.EscapeDataString(projectPath); + } + + // A string is quoted as it is. An array or an object is quoted as JSON, because its shape varies with the + // endpoint, and the raw text is what a reader can match against the GitLab documentation. + private static string? Quote(JsonElement root, string memberName) + { + if (!root.TryGetProperty(memberName, out var value)) + { + return null; + } + + var text = value.ValueKind switch { + JsonValueKind.String => value.GetString(), + JsonValueKind.Null => null, + _ => value.GetRawText(), + }; + + return text is { Length: > 0 } ? text : null; + } + + private async Task SendOrNullAsync( + HttpRequestMessage request, + Uri url, + JsonTypeInfo typeInfo, + CancellationToken cancellationToken) + where T : class + { + using var response = await SendCoreAsync(request, url, cancellationToken).ConfigureAwait(false); + if (response.StatusCode == HttpStatusCode.NotFound) + { + return null; + } + + await EnsureSuccessAsync(request.Method, url, response, cancellationToken).ConfigureAwait(false); + return await ReadAsync(request.Method, url, response, typeInfo, cancellationToken).ConfigureAwait(false); + } +} diff --git a/src/Buildvana.Tool/Services/Hosts/GitLab/GitLabHost.cs b/src/Buildvana.Tool/Services/Hosts/GitLab/GitLabHost.cs new file mode 100644 index 00000000..f95ec4e8 --- /dev/null +++ b/src/Buildvana.Tool/Services/Hosts/GitLab/GitLabHost.cs @@ -0,0 +1,148 @@ +// Copyright (C) Tenacom and Contributors. Licensed under the MIT license. +// See the LICENSE file in the project root for full license information. + +using System; +using System.Net; +using System.Net.Http; +using System.Threading.Tasks; +using Buildvana.Core; +using Buildvana.Core.ConsoleOutput; +using Buildvana.Runtime; +using Buildvana.Tool.Services.Git; +using Buildvana.Tool.Utilities; +using CommunityToolkit.Diagnostics; +using Microsoft.Extensions.DependencyInjection; + +namespace Buildvana.Tool.Services.Hosts.GitLab; + +/// +/// The GitLab host: gitlab.com, or a self-managed GitLab instance. +/// +/// +/// The token is the job token of GitLab CI when is CI_JOB_TOKEN, and an +/// access token otherwise. GitLab exposes a job token through that variable alone, so its name decides. +/// +internal sealed class GitLabHost : GitHost, IDisposable +{ + /// + /// The name of the variable that holds the job token of GitLab CI. + /// + public const string JobTokenVariable = "CI_JOB_TOKEN"; + + private readonly IServiceProvider _services; + private readonly GitLabConfig _config; + private readonly GitLabRepositoryUrls _urls; + private readonly HttpMessageHandler? _handler; + + private GitLabApiClient? _client; + + /// + /// Initializes a new instance of the class. + /// + /// The service provider. + /// The parsed URL of the repository. + /// The handler that sends the requests of the API client, or + /// for one that reaches the network. It is disposed together with this instance once the client exists. + public GitLabHost(IServiceProvider services, GitUrlInfo url, HttpMessageHandler? handler = null) + : base(url) + { + Guard.IsNotNull(services); + _services = services; + _config = services.GetRequiredService().Hosts.GitLab; + ProjectPath = $"{Owner}/{Repository}"; + _urls = new GitLabRepositoryUrls(HostName, ProjectPath); + _handler = handler; + } + + /// + public override string Name => "GitLab"; + + /// + /// Always . + public override HostKind? Kind => HostKind.GitLab; + + /// + public override Uri RepositoryUrl => _urls.Repository; + + /// + public override string TokenEnv => _config.TokenEnv; + + /// + /// gitlab-ci-token for a job token, oauth2 for an access token. + public override string PushUsername => TokenKind == GitLabTokenKind.JobToken ? "gitlab-ci-token" : "oauth2"; + + /// + /// Gets the path of the project, such as group/subgroup/project. + /// + public string ProjectPath { get; } + + /// + /// Gets the kind of the token names. + /// + public GitLabTokenKind TokenKind + => string.Equals(TokenEnv, JobTokenVariable, StringComparison.Ordinal) ? GitLabTokenKind.JobToken : GitLabTokenKind.AccessToken; + + /// + /// Gets the client of the REST API of the host, created on first use. + /// + /// The variable names is not set. + /// + /// The client is created on first use, so that a host whose token is missing can still answer what + /// needs no API call, such as the URLs and the push username. + /// + internal GitLabApiClient Client => _client ??= new GitLabApiClient( + HostName, + RuntimeAccess.Translate(_config.GetToken), + TokenKind, + _handler ?? new HttpClientHandler { AutomaticDecompression = DecompressionMethods.All }); + + /// + public override Uri GetReleaseUrl(string tag) => _urls.Release(tag); + + /// + /// + /// A GitLab file URL takes the name of a tag, a branch, or a commit alike, so the kind of + /// plays no part. + /// + public override Uri GetFileUrl(string path, GitRef gitRef) + { + Guard.IsNotNull(gitRef); + return _urls.File(path, gitRef.Name); + } + + /// + /// + /// A project of internal visibility counts as private, because only the users of the instance + /// can see it. + /// A job token cannot read a project, so with one the project is read without credentials. GitLab + /// answers that request only for a public project. + /// + public override async Task IsPrivateRepositoryAsync() + { + _services.GetRequiredService().Info("Fetching project information..."); + var project = TokenKind == GitLabTokenKind.JobToken + ? await Client.GetProjectWithoutCredentialsAsync(ProjectPath).ConfigureAwait(false) + : await Client.GetProjectAsync(ProjectPath).ConfigureAwait(false); + return project is not { Visibility: "public" }; + } + + /// + public override ReleasePublisher CreateReleasePublisher() + => new GitLabReleasePublisher( + this, + _services.GetRequiredService(), + _services.GetRequiredService()); + + /// + public void Dispose() + { + if (_client is not null) + { + _client.Dispose(); + } + else + { + _handler?.Dispose(); + } + } +} diff --git a/src/Buildvana.Tool/Services/Hosts/GitLab/GitLabJsonContext.cs b/src/Buildvana.Tool/Services/Hosts/GitLab/GitLabJsonContext.cs new file mode 100644 index 00000000..d34b290b --- /dev/null +++ b/src/Buildvana.Tool/Services/Hosts/GitLab/GitLabJsonContext.cs @@ -0,0 +1,26 @@ +// Copyright (C) Tenacom and Contributors. Licensed under the MIT license. +// See the LICENSE file in the project root for full license information. + +using System.Collections.Generic; +using System.Text.Json.Serialization; + +namespace Buildvana.Tool.Services.Hosts.GitLab; + +/// +/// The source-generated serializer context for the requests and answers of the GitLab REST API. +/// +/// +/// Reading ignores unknown members, because every GitLab answer carries many more members than +/// bv reads. A required member that is missing fails the read. +/// +[JsonSourceGenerationOptions( + PropertyNamingPolicy = JsonKnownNamingPolicy.SnakeCaseLower, + RespectNullableAnnotations = true)] +[JsonSerializable(typeof(GitLabUser))] +[JsonSerializable(typeof(GitLabProject))] +[JsonSerializable(typeof(GitLabMember))] +[JsonSerializable(typeof(IReadOnlyList))] +[JsonSerializable(typeof(GitLabRelease))] +[JsonSerializable(typeof(GitLabNewRelease))] +[JsonSerializable(typeof(GitLabPackageFile))] +internal sealed partial class GitLabJsonContext : JsonSerializerContext; diff --git a/src/Buildvana.Tool/Services/Hosts/GitLab/GitLabMember.cs b/src/Buildvana.Tool/Services/Hosts/GitLab/GitLabMember.cs new file mode 100644 index 00000000..470ad0e9 --- /dev/null +++ b/src/Buildvana.Tool/Services/Hosts/GitLab/GitLabMember.cs @@ -0,0 +1,13 @@ +// Copyright (C) Tenacom and Contributors. Licensed under the MIT license. +// See the LICENSE file in the project root for full license information. + +namespace Buildvana.Tool.Services.Hosts.GitLab; + +/// +/// The members of a GitLab project member that bv reads. +/// +internal sealed record GitLabMember +{ + /// Gets the access level of the member, such as 30 for Developer and 40 for Maintainer. + public required int AccessLevel { get; init; } +} diff --git a/src/Buildvana.Tool/Services/Hosts/GitLab/GitLabNewRelease.cs b/src/Buildvana.Tool/Services/Hosts/GitLab/GitLabNewRelease.cs new file mode 100644 index 00000000..2e901055 --- /dev/null +++ b/src/Buildvana.Tool/Services/Hosts/GitLab/GitLabNewRelease.cs @@ -0,0 +1,22 @@ +// Copyright (C) Tenacom and Contributors. Licensed under the MIT license. +// See the LICENSE file in the project root for full license information. + +namespace Buildvana.Tool.Services.Hosts.GitLab; + +/// +/// The members bv sends to create a GitLab release. +/// +internal sealed record GitLabNewRelease +{ + /// Gets the name of the tag of the release. The tag must exist. + public required string TagName { get; init; } + + /// Gets the name of the release. + public required string Name { get; init; } + + /// Gets the description of the release, in Markdown. + public required string Description { get; init; } + + /// Gets the assets of the release. + public required GitLabReleaseAssets Assets { get; init; } +} diff --git a/src/Buildvana.Tool/Services/Hosts/GitLab/GitLabPackageFile.cs b/src/Buildvana.Tool/Services/Hosts/GitLab/GitLabPackageFile.cs new file mode 100644 index 00000000..c16b2a96 --- /dev/null +++ b/src/Buildvana.Tool/Services/Hosts/GitLab/GitLabPackageFile.cs @@ -0,0 +1,13 @@ +// Copyright (C) Tenacom and Contributors. Licensed under the MIT license. +// See the LICENSE file in the project root for full license information. + +namespace Buildvana.Tool.Services.Hosts.GitLab; + +/// +/// The members of a GitLab package file that bv reads. +/// +internal sealed record GitLabPackageFile +{ + /// Gets the identifier of the package the file belongs to. + public required long PackageId { get; init; } +} diff --git a/src/Buildvana.Tool/Services/Hosts/GitLab/GitLabProject.cs b/src/Buildvana.Tool/Services/Hosts/GitLab/GitLabProject.cs new file mode 100644 index 00000000..a160a924 --- /dev/null +++ b/src/Buildvana.Tool/Services/Hosts/GitLab/GitLabProject.cs @@ -0,0 +1,13 @@ +// Copyright (C) Tenacom and Contributors. Licensed under the MIT license. +// See the LICENSE file in the project root for full license information. + +namespace Buildvana.Tool.Services.Hosts.GitLab; + +/// +/// The members of a GitLab project that bv reads. +/// +internal sealed record GitLabProject +{ + /// Gets the visibility of the project: public, internal, or private. + public required string Visibility { get; init; } +} diff --git a/src/Buildvana.Tool/Services/Hosts/GitLab/GitLabRelease.cs b/src/Buildvana.Tool/Services/Hosts/GitLab/GitLabRelease.cs new file mode 100644 index 00000000..f5c3a22f --- /dev/null +++ b/src/Buildvana.Tool/Services/Hosts/GitLab/GitLabRelease.cs @@ -0,0 +1,13 @@ +// Copyright (C) Tenacom and Contributors. Licensed under the MIT license. +// See the LICENSE file in the project root for full license information. + +namespace Buildvana.Tool.Services.Hosts.GitLab; + +/// +/// The members of a GitLab release that bv reads. +/// +internal sealed record GitLabRelease +{ + /// Gets the name of the tag of the release, which identifies the release in the API. + public required string TagName { get; init; } +} diff --git a/src/Buildvana.Tool/Services/Hosts/GitLab/GitLabReleaseAssets.cs b/src/Buildvana.Tool/Services/Hosts/GitLab/GitLabReleaseAssets.cs new file mode 100644 index 00000000..bdde7c86 --- /dev/null +++ b/src/Buildvana.Tool/Services/Hosts/GitLab/GitLabReleaseAssets.cs @@ -0,0 +1,15 @@ +// Copyright (C) Tenacom and Contributors. Licensed under the MIT license. +// See the LICENSE file in the project root for full license information. + +using System.Collections.Generic; + +namespace Buildvana.Tool.Services.Hosts.GitLab; + +/// +/// The assets bv sends with a new GitLab release. +/// +internal sealed record GitLabReleaseAssets +{ + /// Gets the links of the release. + public required IReadOnlyList Links { get; init; } +} diff --git a/src/Buildvana.Tool/Services/Hosts/GitLab/GitLabReleaseLink.cs b/src/Buildvana.Tool/Services/Hosts/GitLab/GitLabReleaseLink.cs new file mode 100644 index 00000000..e0404266 --- /dev/null +++ b/src/Buildvana.Tool/Services/Hosts/GitLab/GitLabReleaseLink.cs @@ -0,0 +1,25 @@ +// Copyright (C) Tenacom and Contributors. Licensed under the MIT license. +// See the LICENSE file in the project root for full license information. + +namespace Buildvana.Tool.Services.Hosts.GitLab; + +/// +/// The members bv sends for a link of a new GitLab release. +/// +internal sealed record GitLabReleaseLink +{ + /// Gets the name of the link, as the release page shows it. + public required string Name { get; init; } + + /// Gets the URL the link points at. + public required string Url { get; init; } + + /// + /// Gets the path of the permanent link of the asset, relative to + /// https://<host>/<project>/-/releases/<tag>/downloads, such as /Foo.1.0.0.nupkg. + /// + public required string DirectAssetPath { get; init; } + + /// Gets the type of the link: other, runbook, image, or package. + public required string LinkType { get; init; } +} diff --git a/src/Buildvana.Tool/Services/Hosts/GitLab/GitLabReleasePublisher.cs b/src/Buildvana.Tool/Services/Hosts/GitLab/GitLabReleasePublisher.cs new file mode 100644 index 00000000..a2e2157a --- /dev/null +++ b/src/Buildvana.Tool/Services/Hosts/GitLab/GitLabReleasePublisher.cs @@ -0,0 +1,241 @@ +// Copyright (C) Tenacom and Contributors. Licensed under the MIT license. +// See the LICENSE file in the project root for full license information. + +using System; +using System.Collections.Generic; +using System.Globalization; +using System.IO; +using System.Linq; +using System.Threading.Tasks; +using Buildvana.Core; +using Buildvana.Core.ConsoleOutput; +using Buildvana.Core.IO; +using Buildvana.Tool.Services.Git; +using Buildvana.Tool.Services.Release; +using CommunityToolkit.Diagnostics; +using Louis.Diagnostics; + +namespace Buildvana.Tool.Services.Hosts.GitLab; + +/// +/// The release publisher of a : the assets uploaded to a generic package, then a release +/// that links them. +/// +/// +/// GitLab has no draft releases, so nothing exists on the host before . +/// The generic package is named after the repository, and its version is the release tag. +/// +internal sealed class GitLabReleasePublisher : ReleasePublisher +{ + // The Maintainer role. Deleting a package needs it, and a rollback deletes the package. + private const int MaintainerAccessLevel = 40; + + private readonly GitLabHost _host; + private readonly IReporter _reporter; + private readonly IGitService _git; + + private string? _releaseTag; + private long? _packageId; + private string? _packageVersion; + + /// + /// Initializes a new instance of the class. + /// + /// The host to publish on. + /// The reporter to report to. + /// The Git service. + public GitLabReleasePublisher(GitLabHost host, IReporter reporter, IGitService git) + { + Guard.IsNotNull(host); + Guard.IsNotNull(reporter); + Guard.IsNotNull(git); + _host = host; + _reporter = reporter; + _git = git; + } + + /// + /// A file name is not valid in the generic package registry, or as the + /// direct asset path of a release link; or two assets have the same file name, or the same description. + /// + /// A release link takes its name from the description of the asset, and GitLab refuses two links of a + /// release with the same name or the same direct asset path. + /// + public override void CheckAssets(IReadOnlyList assets) + { + Guard.IsNotNull(assets); + var problems = new List(); + var fileNames = assets.Select(static x => Path.GetFileName(x.Path)).ToArray(); + foreach (var fileName in fileNames.Distinct(StringComparer.Ordinal)) + { + if (!GitLabApiClient.IsValidPackageFileName(fileName)) + { + problems.Add($"'{fileName}' is not a valid file name in the generic package registry."); + } + else if (!GitLabApiClient.IsValidDirectAssetPath("/" + fileName)) + { + problems.Add($"'{fileName}' is not a valid file name for a release link."); + } + } + + foreach (var fileName in Duplicates(fileNames)) + { + problems.Add($"More than one asset has the file name '{fileName}'."); + } + + foreach (var description in Duplicates(assets.Select(static x => x.Description))) + { + problems.Add($"More than one asset has the description '{description}'."); + } + + BuildFailedException.ThrowIf( + problems.Count > 0, + "GitLab refuses the release assets:" + string.Concat(problems.Select(static x => "\n- " + x))); + } + + /// + /// + /// This method creates nothing. It reads the releases of the project, which fails for a token that cannot + /// reach them. With an access token, it also checks that the user of the token has the Maintainer role. + /// + public override async Task BeginAsync() + { + var project = _host.ProjectPath; + _reporter.Info($"Checking access to the releases of {project}..."); + _ = await _host.Client.GetReleasesAsync(project, 1).ConfigureAwait(false); + if (_host.TokenKind == GitLabTokenKind.JobToken) + { + // A job token has the access of the user who started the job, and cannot read a membership. + _reporter.Info("Role check skipped: GitLab states no role for a job token."); + return; + } + + _reporter.Info("Checking the role of the token..."); + var user = await _host.Client.GetCurrentUserAsync().ConfigureAwait(false); + var member = await _host.Client.GetProjectMemberAsync(project, user.Id).ConfigureAwait(false); + + // The releases of the project are readable, so a missing membership is not a missing project. A user + // with no membership is an instance administrator or an organization owner, who can delete a package, + // or a user without push access, whose push of the release commit fails before anything is published. + if (member is null) + { + _reporter.Info("Role check skipped: the user of the token is not a member of the project."); + return; + } + + if (member.AccessLevel < MaintainerAccessLevel) + { + var level = member.AccessLevel.ToString(CultureInfo.InvariantCulture); + throw new BuildFailedException( + $"The GitLab token has access level {level} on {project}, below the Maintainer role. " + + "A release needs the Maintainer role, because a rollback deletes the generic package."); + } + } + + /// + public override async Task PublishAsync(string tag, string sha, IReadOnlyList assets) + { + Guard.IsNotNullOrEmpty(tag); + Guard.IsNotNullOrEmpty(sha); + Guard.IsNotNull(assets); + var assetCount = assets.Count; + if (assetCount == 0) + { + _reporter.Info("Asset upload skipped: no release assets defined."); + } + + var links = new List(assetCount); + for (var i = 0; i < assetCount; i++) + { + var asset = assets[i]; + _reporter.Info(string.Create( + CultureInfo.InvariantCulture, + $"Uploading asset {i + 1} of {assetCount}: {Path.GetFileName(asset.Path)} ({asset.Description})...")); + links.Add(await UploadAssetAsync(tag, asset).ConfigureAwait(false)); + } + + var changelogUrl = _host.GetFileUrl("CHANGELOG.md", GitRef.Branch(_git.CurrentBranch)); + _reporter.Info($"Creating release {tag}..."); + var release = new GitLabNewRelease { + TagName = tag, + Name = tag, + Description = $"We have a [human-curated changelog]({changelogUrl}).", + Assets = new GitLabReleaseAssets { Links = links }, + }; + + _ = await _host.Client.CreateReleaseAsync(_host.ProjectPath, release).ConfigureAwait(false); + _releaseTag = tag; + } + + /// + /// + /// The release goes first, because its links point at the files of the package. + /// A release deletion that fails is a warning, and the package deletion goes on. A rollback deletes the + /// release tag before this method runs, and GitLab deletes the release with it, so a missing release is no + /// failure. + /// + public override async Task UndoAsync() + { + if (_releaseTag is { } tag) + { + await DeleteReleaseAsync(tag).ConfigureAwait(false); + } + + if (_packageId is { } packageId) + { + _reporter.Info("Deleting the previously created generic package..."); + await _host.Client.DeletePackageAsync(_host.ProjectPath, packageId).ConfigureAwait(false); + _reporter.Notice($"Deleted generic package {_host.Repository} {_packageVersion}."); + _packageId = null; + } + } + + private async Task UploadAssetAsync(string tag, ReleaseAsset asset) + { + _reporter.Detail($"Uploading asset {asset.Path}..."); + var project = _host.ProjectPath; + var packageName = _host.Repository; + var fileName = Path.GetFileName(asset.Path); + var contents = UserFile.OpenRead(asset.Path); + await using (contents.ConfigureAwait(false)) + { + var file = await _host.Client.UploadPackageFileAsync(project, packageName, tag, fileName, contents) + .ConfigureAwait(false); + _packageId ??= file.PackageId; + _packageVersion = tag; + } + + return new GitLabReleaseLink { + Name = asset.Description, + Url = _host.Client.GetPackageFileUrl(project, packageName, tag, fileName).AbsoluteUri, + DirectAssetPath = "/" + fileName, + LinkType = "package", + }; + } + + private async Task DeleteReleaseAsync(string tag) + { + _reporter.Info("Deleting the previously created release..."); + bool deleted; + try + { + deleted = await _host.Client.DeleteReleaseAsync(_host.ProjectPath, tag).ConfigureAwait(false); + } + catch (Exception ex) when (!ex.IsCriticalError()) + { + _reporter.Warning($"{ex.GetType().Name} while deleting release {tag}: {ex.Message}"); + return; + } + + if (deleted) + { + _reporter.Notice($"Deleted release {tag}."); + } + else + { + _reporter.Info($"Release {tag} not found: GitLab deleted it with its tag."); + } + + _releaseTag = null; + } +} diff --git a/src/Buildvana.Tool/Services/Hosts/GitLab/GitLabRepositoryUrls.cs b/src/Buildvana.Tool/Services/Hosts/GitLab/GitLabRepositoryUrls.cs new file mode 100644 index 00000000..ec9f35c4 --- /dev/null +++ b/src/Buildvana.Tool/Services/Hosts/GitLab/GitLabRepositoryUrls.cs @@ -0,0 +1,65 @@ +// Copyright (C) Tenacom and Contributors. Licensed under the MIT license. +// See the LICENSE file in the project root for full license information. + +using System; +using CommunityToolkit.Diagnostics; + +namespace Buildvana.Tool.Services.Hosts.GitLab; + +/// +/// Builds the URLs of a GitLab project and of the resources it contains. +/// +/// +/// The URL of a resource puts a - segment between the path of the project and the kind of the +/// resource, because a project path can hold any number of group segments. +/// The project URL carries no trailing slash, as says of +/// a GitHub repository URL. +/// +internal sealed class GitLabRepositoryUrls +{ + private readonly string _repository; + + /// + /// Initializes a new instance of the class. + /// + /// The host name of the GitLab instance, such as gitlab.com. + /// The path of the project, such as group/subgroup/project. + public GitLabRepositoryUrls(string hostName, string projectPath) + { + Guard.IsNotNullOrEmpty(hostName); + Guard.IsNotNullOrEmpty(projectPath); + + _repository = $"https://{hostName}/{projectPath}"; + Repository = new Uri(_repository); + } + + /// + /// Gets the URL of the project, without a trailing slash. + /// + public Uri Repository { get; } + + /// + /// Builds the URL of the release of a tag. + /// + /// The name of the tag. + /// The URL of the release. + public Uri Release(string tag) + { + Guard.IsNotNullOrEmpty(tag); + return new Uri($"{_repository}/-/releases/{tag}"); + } + + /// + /// Builds the URL of a file of the project, as of a tag, a branch, or a commit. + /// + /// The path of the file, relative to the root of the repository. + /// The name of the tag or the branch, or the SHA of the commit. + /// The URL of the file. + /// is rooted, or leaves the repository, as + /// says. + public Uri File(string path, string commitish) + { + Guard.IsNotNullOrEmpty(commitish); + return new Uri($"{_repository}/-/blob/{commitish}/{RepositoryFilePath.ToUrlPath(path)}"); + } +} diff --git a/src/Buildvana.Tool/Services/Hosts/GitLab/GitLabTokenKind.cs b/src/Buildvana.Tool/Services/Hosts/GitLab/GitLabTokenKind.cs new file mode 100644 index 00000000..3ea84cfc --- /dev/null +++ b/src/Buildvana.Tool/Services/Hosts/GitLab/GitLabTokenKind.cs @@ -0,0 +1,20 @@ +// Copyright (C) Tenacom and Contributors. Licensed under the MIT license. +// See the LICENSE file in the project root for full license information. + +namespace Buildvana.Tool.Services.Hosts.GitLab; + +/// +/// Specifies the kind of a GitLab token, which decides the header that carries it. +/// +internal enum GitLabTokenKind +{ + /// + /// A personal, project, or group access token, carried by the PRIVATE-TOKEN header. + /// + AccessToken, + + /// + /// The job token of GitLab CI, carried by the JOB-TOKEN header. + /// + JobToken, +} diff --git a/src/Buildvana.Tool/Services/Hosts/GitLab/GitLabUser.cs b/src/Buildvana.Tool/Services/Hosts/GitLab/GitLabUser.cs new file mode 100644 index 00000000..62aa21b5 --- /dev/null +++ b/src/Buildvana.Tool/Services/Hosts/GitLab/GitLabUser.cs @@ -0,0 +1,13 @@ +// Copyright (C) Tenacom and Contributors. Licensed under the MIT license. +// See the LICENSE file in the project root for full license information. + +namespace Buildvana.Tool.Services.Hosts.GitLab; + +/// +/// The members of a GitLab user that bv reads. +/// +internal sealed record GitLabUser +{ + /// Gets the identifier of the user. + public required long Id { get; init; } +} diff --git a/src/Buildvana.Tool/Services/Hosts/HostApiClient.cs b/src/Buildvana.Tool/Services/Hosts/HostApiClient.cs new file mode 100644 index 00000000..8bb06484 --- /dev/null +++ b/src/Buildvana.Tool/Services/Hosts/HostApiClient.cs @@ -0,0 +1,287 @@ +// Copyright (C) Tenacom and Contributors. Licensed under the MIT license. +// See the LICENSE file in the project root for full license information. + +using System; +using System.Globalization; +using System.Net.Http; +using System.Net.Http.Headers; +using System.Text; +using System.Text.Json; +using System.Text.Json.Serialization.Metadata; +using System.Threading; +using System.Threading.Tasks; +using Buildvana.Core; +using CommunityToolkit.Diagnostics; + +namespace Buildvana.Tool.Services.Hosts; + +/// +/// Base class for the clients of the REST API of a Git host. +/// +/// +/// An answer outside the 2xx range fails the build, with the status code and the message of the answer. +/// A derived class that expects another status code sends the request with , and +/// checks the code before it calls . +/// Every diagnostic names the API after , with the method and the URL of the +/// request. +/// A request carries the credentials of the client when creates it. A request +/// that a derived class creates otherwise carries none. +/// +internal abstract class HostApiClient : IDisposable +{ + private readonly HttpClient _http; + + /// + /// Initializes a new instance of the class. + /// + /// The name of the API in diagnostics, such as GitHub. + /// The base URL of the API, with a trailing slash. + /// The handler that sends the requests. It is disposed together with this + /// instance. + protected HostApiClient(string apiName, Uri baseUrl, HttpMessageHandler handler) + { + Guard.IsNotNullOrEmpty(apiName); + Guard.IsNotNull(baseUrl); + Guard.IsNotNull(handler); + + ApiName = apiName; + BaseUrl = baseUrl; + _http = new HttpClient(handler); + } + + /// + /// Gets the base URL of the API, with a trailing slash. + /// + public Uri BaseUrl { get; } + + /// + /// Gets the name of the API in diagnostics. + /// + protected string ApiName { get; } + + /// + /// Gets the headers that every request carries. + /// + protected HttpRequestHeaders DefaultRequestHeaders => _http.DefaultRequestHeaders; + + /// + public void Dispose() + { + Dispose(true); + GC.SuppressFinalize(this); + } + + /// + /// Serializes a value as the JSON body of a request. + /// + /// The type of the value. + /// The value. + /// The serialization metadata of . + /// A newly-created . + protected static StringContent JsonBody(T value, JsonTypeInfo typeInfo) + => new(JsonSerializer.Serialize(value, typeInfo), Encoding.UTF8, "application/json"); + + /// + /// Builds the absolute URL of an endpoint. + /// + /// The path of the endpoint, relative to . + /// The URL of the endpoint. + /// + /// Every request goes to an absolute URL that the client holds and passes along with the request, + /// so that a diagnostic names the URL without reading back the nullable + /// . + /// + protected Uri ApiUrl(string path) => new(BaseUrl, path); + + /// + /// Asynchronously sends a request, and reads the JSON answer. + /// + /// The type of the answer. + /// The method of the request. + /// The URL of the request. + /// The body of the request, or for a request without one. + /// The serialization metadata of . + /// A token that cancels the operation. + /// A whose result is the answer. + protected async Task SendAsync( + HttpMethod method, + Uri url, + HttpContent? content, + JsonTypeInfo typeInfo, + CancellationToken cancellationToken) + where T : class + { + using var request = CreateRequest(method, url); + request.Content = content; + using var response = await SendCoreAsync(request, url, cancellationToken).ConfigureAwait(false); + await EnsureSuccessAsync(method, url, response, cancellationToken).ConfigureAwait(false); + return await ReadAsync(method, url, response, typeInfo, cancellationToken).ConfigureAwait(false); + } + + /// + /// Asynchronously sends a request whose answer has no body to read. + /// + /// The method of the request. + /// The URL of the request. + /// The body of the request, or for a request without one. + /// A token that cancels the operation. + /// A representing the ongoing operation. + protected async Task SendAsync( + HttpMethod method, + Uri url, + HttpContent? content, + CancellationToken cancellationToken) + { + using var request = CreateRequest(method, url); + request.Content = content; + using var response = await SendCoreAsync(request, url, cancellationToken).ConfigureAwait(false); + await EnsureSuccessAsync(method, url, response, cancellationToken).ConfigureAwait(false); + } + + /// + /// Creates a request that carries the credentials of the client. + /// + /// The method of the request. + /// The URL of the request. + /// A newly-created . + protected HttpRequestMessage CreateRequest(HttpMethod method, Uri url) + { + var request = new HttpRequestMessage(method, url); + Authenticate(request); + return request; + } + + /// + /// Adds the credentials of the client to a request. + /// + /// The request. + protected abstract void Authenticate(HttpRequestMessage request); + + /// + /// Asynchronously sends a request, and returns the answer whatever its status code. + /// + /// The request. + /// The URL of the request. + /// A token that cancels the operation. + /// A whose result is the answer. + /// The API cannot be reached, or does not answer in time. + protected async Task SendCoreAsync( + HttpRequestMessage request, + Uri url, + CancellationToken cancellationToken) + { + Guard.IsNotNull(request); + try + { + return await _http.SendAsync(request, cancellationToken).ConfigureAwait(false); + } + catch (HttpRequestException exception) + { + throw new BuildFailedException( + $"The {ApiName} API could not be reached for {Describe(request.Method, url)}: {exception.Message}", + exception); + } + catch (TaskCanceledException exception) when (!cancellationToken.IsCancellationRequested) + { + throw new BuildFailedException($"The {ApiName} API did not answer {Describe(request.Method, url)} in time.", exception); + } + } + + /// + /// Asynchronously fails the build when an answer is outside the 2xx range. + /// + /// The method of the request. + /// The URL of the request. + /// The answer. + /// A token that cancels the operation. + /// A representing the ongoing operation. + /// The status code of is outside the + /// 2xx range. + protected async Task EnsureSuccessAsync( + HttpMethod method, + Uri url, + HttpResponseMessage response, + CancellationToken cancellationToken) + { + Guard.IsNotNull(response); + if (response.IsSuccessStatusCode) + { + return; + } + + var message = await ReadErrorMessageAsync(response, cancellationToken).ConfigureAwait(false); + var status = string.Create(CultureInfo.InvariantCulture, $"{(int)response.StatusCode} {response.ReasonPhrase}"); + throw new BuildFailedException(message is null + ? $"The {ApiName} API answered {Describe(method, url)} with {status}." + : $"The {ApiName} API answered {Describe(method, url)} with {status}: {message}"); + } + + /// + /// Asynchronously reads the message of an error answer. + /// + /// The answer, whose status code is outside the 2xx range. + /// A token that cancels the operation. + /// A whose result is the message, or when the answer + /// states none that the client can read. + protected abstract Task ReadErrorMessageAsync( + HttpResponseMessage response, + CancellationToken cancellationToken); + + /// + /// Asynchronously reads the JSON answer of a request. + /// + /// The type of the answer. + /// The method of the request. + /// The URL of the request. + /// The answer. + /// The serialization metadata of . + /// A token that cancels the operation. + /// A whose result is the answer. + /// The answer is not JSON of type , or is + /// null. + /// + /// The client buffers every answer before returns, so reading the body + /// meets no network. + /// + protected async Task ReadAsync( + HttpMethod method, + Uri url, + HttpResponseMessage response, + JsonTypeInfo typeInfo, + CancellationToken cancellationToken) + where T : class + { + Guard.IsNotNull(response); + T? result; + try + { + var stream = await response.Content.ReadAsStreamAsync(cancellationToken).ConfigureAwait(false); + await using (stream.ConfigureAwait(false)) + { + result = await JsonSerializer.DeserializeAsync(stream, typeInfo, cancellationToken).ConfigureAwait(false); + } + } + catch (JsonException exception) + { + throw new BuildFailedException( + $"The {ApiName} API answered {Describe(method, url)} with something bv cannot read: {exception.Message}", + exception); + } + + return result ?? throw new BuildFailedException($"The {ApiName} API answered {Describe(method, url)} with null."); + } + + /// + /// Releases the resources of this instance. + /// + /// when called from . + protected virtual void Dispose(bool disposing) + { + if (disposing) + { + _http.Dispose(); + } + } + + private static string Describe(HttpMethod method, Uri url) => $"{method} {url.AbsoluteUri}"; +} diff --git a/src/Buildvana.Tool/Services/Hosts/ReleasePublisher.cs b/src/Buildvana.Tool/Services/Hosts/ReleasePublisher.cs index 1a009cfd..82df10e6 100644 --- a/src/Buildvana.Tool/Services/Hosts/ReleasePublisher.cs +++ b/src/Buildvana.Tool/Services/Hosts/ReleasePublisher.cs @@ -1,7 +1,9 @@ // Copyright (C) Tenacom and Contributors. Licensed under the MIT license. // See the LICENSE file in the project root for full license information. +using System; using System.Collections.Generic; +using System.Linq; using System.Threading.Tasks; using Buildvana.Tool.Services.Release; @@ -29,6 +31,18 @@ private protected ReleasePublisher() /// public abstract Task BeginAsync(); + /// + /// Checks that the host accepts a list of release assets, and fails the build when it does not. + /// + /// The files to attach to the release. + /// + /// calls this before it pushes the commits and the tag, so that a refusal + /// fails the release while nothing has left the clone. This implementation accepts every list. + /// + public virtual void CheckAssets(IReadOnlyList assets) + { + } + /// /// Asynchronously publishes the release on a tag that already exists on the host. /// @@ -43,4 +57,12 @@ private protected ReleasePublisher() /// /// A representing the ongoing operation. public abstract Task UndoAsync(); + + /// + /// Finds the values that occur more than once in a sequence, compared ordinally. + /// + /// The values to search. + /// Each value that occurs more than once, once, in the order of its first occurrence. + private protected static IEnumerable Duplicates(IEnumerable values) + => values.CountBy(static x => x, StringComparer.Ordinal).Where(static x => x.Value > 1).Select(static x => x.Key); } diff --git a/src/Buildvana.Tool/Services/Hosts/RepositoryFilePath.cs b/src/Buildvana.Tool/Services/Hosts/RepositoryFilePath.cs new file mode 100644 index 00000000..1e26bccb --- /dev/null +++ b/src/Buildvana.Tool/Services/Hosts/RepositoryFilePath.cs @@ -0,0 +1,42 @@ +// Copyright (C) Tenacom and Contributors. Licensed under the MIT license. +// See the LICENSE file in the project root for full license information. + +using System; +using System.IO; +using CommunityToolkit.Diagnostics; + +namespace Buildvana.Tool.Services.Hosts; + +/// +/// Converts the path of a file of a repository to the path a file URL of a Git host ends with. +/// +internal static class RepositoryFilePath +{ + /// + /// Converts the path of a file of a repository to the path a file URL ends with. + /// + /// The path of the file, relative to the root of the repository. + /// , with forward slashes as separators. + /// is empty, rooted, or holds a .. + /// segment. + public static string ToUrlPath(string path) + { + Guard.IsNotNullOrEmpty(path); + + // Normalize to forward slashes for the URL, then reject paths that are rooted or escape the repo. + // A rooted path is checked after normalization, so that a leading slash of either kind is rejected on + // every platform: Path.IsPathFullyQualified is true for "/docs/x.md" on Linux and false on Windows, + // where a fully qualified path needs a drive or a UNC share, and a backslash is a separator only there. + // Every ".." segment must go, not just a leading one: Uri collapses parent segments as it parses, + // so enough of them anywhere in the path walk out of the repository and even out of the owner + // (".../blob/main/docs/../../../../../../etc/passwd" parses to "https://github.com/etc/passwd"). + var remotePath = path.Replace('\\', '/'); + Guard.IsTrue(!Path.IsPathRooted(remotePath), nameof(path), "A path must be relative to be converted to a file URL."); + var hasParentSegment = remotePath == ".." + || remotePath.StartsWith("../", StringComparison.Ordinal) + || remotePath.EndsWith("/..", StringComparison.Ordinal) + || remotePath.Contains("/../", StringComparison.Ordinal); + Guard.IsTrue(!hasParentSegment, nameof(path), "Only a path to a file in the repository can be converted to a file URL."); + return remotePath; + } +} diff --git a/src/Buildvana.Tool/Services/Platforms/GitLabCIPlatform.cs b/src/Buildvana.Tool/Services/Platforms/GitLabCIPlatform.cs index e415a553..8ab4df13 100644 --- a/src/Buildvana.Tool/Services/Platforms/GitLabCIPlatform.cs +++ b/src/Buildvana.Tool/Services/Platforms/GitLabCIPlatform.cs @@ -2,6 +2,7 @@ // See the LICENSE file in the project root for full license information. using System; +using Buildvana.Runtime; using Buildvana.Tool.Services.Git; using CommunityToolkit.Diagnostics; @@ -32,6 +33,10 @@ private GitLabCIPlatform(string? serverHost, Uri? serverUrl) /// The value of CI_SERVER_URL. public override Uri? ServerUrl { get; } + /// + /// Always . + public override HostKind? ServerKind => HostKind.GitLab; + /// /// Creates and returns an instance of if bv runs in a GitLab CI runner. /// diff --git a/src/Buildvana.Tool/Services/Release/ServerRelease.cs b/src/Buildvana.Tool/Services/Release/ServerRelease.cs index 9084443f..ad4f2363 100644 --- a/src/Buildvana.Tool/Services/Release/ServerRelease.cs +++ b/src/Buildvana.Tool/Services/Release/ServerRelease.cs @@ -253,6 +253,8 @@ public void AddPostReleaseCommit(string message, params string[] files) /// /// The tag is created here, and not with the release commit, because the release commit is amended /// until the changelog is finalized after the build. + /// The publisher checks the assets first, so that an asset the host refuses fails the release before + /// the push, and before the caller pushes the packages to a feed that nothing can delete them from. /// public void PushUpdates() { @@ -269,6 +271,7 @@ public void PushUpdates() ThrowHelper.ThrowInvalidOperationException("Internal error: updates have already been pushed."); } + _publisher.CheckAssets(_assets); var tag = _version.CurrentStr; _git.CreateTag(tag, ReleaseCommitSha); OnRollback(() => { @@ -292,11 +295,27 @@ public void PushUpdates() _tag = tag; } + /// + /// Adds a file to attach to the release. + /// + /// The path of the file. + /// The description of the asset, or for the file name. + /// The MIME type of the asset, or for + /// application/octet-stream. + /// + /// An asset is added before , which has the publisher check every asset. + /// public void AddAsset(string path, string? description = null, string? mimeType = null) { EnsurePending(); Guard.IsNotNullOrEmpty(path); + if (_updatesPushed) + { + ThrowHelper.ThrowInvalidOperationException( + "Internal error: cannot add an asset when updates have already been pushed."); + } + if (string.IsNullOrEmpty(description)) { description = Path.GetFileName(path); diff --git a/src/Buildvana.Tool/Subcommands/ReleaseCommand.cs b/src/Buildvana.Tool/Subcommands/ReleaseCommand.cs index 1ee639d2..c2217eb8 100644 --- a/src/Buildvana.Tool/Subcommands/ReleaseCommand.cs +++ b/src/Buildvana.Tool/Subcommands/ReleaseCommand.cs @@ -335,12 +335,8 @@ public async Task ExecuteAsync(CancellationToken cancellationToken) release.AddPostReleaseCommit($"Post-release updates for {version.CurrentStr} [skip ci]", postReleaseUpdates); } - release.PushUpdates(); - - // Publish NuGet packages - await dotnet.NuGetPushAllAsync(artifactsPath, cancellationToken).ConfigureAwait(false); - - // Gather build assets from Buildvana.Sdk release asset lists + // Gather build assets from Buildvana.Sdk release asset lists. + // Must happen before the push, which has the host check the assets while nothing has left the clone. reporter.Info("Reading release asset lists..."); foreach (var path in UserDirectory.EnumerateFiles(artifactsPath, "*.assets.txt")) { @@ -381,6 +377,11 @@ public async Task ExecuteAsync(CancellationToken cancellationToken) } } + release.PushUpdates(); + + // Publish NuGet packages + await dotnet.NuGetPushAllAsync(artifactsPath, cancellationToken).ConfigureAwait(false); + // Last but not least, publish the release. await release.PublishAsync().ConfigureAwait(false); } diff --git a/tests/Buildvana.Core.Configuration.Tests/BuildvanaConfigFactoryTests.cs b/tests/Buildvana.Core.Configuration.Tests/BuildvanaConfigFactoryTests.cs index e3b396bd..f0495809 100644 --- a/tests/Buildvana.Core.Configuration.Tests/BuildvanaConfigFactoryTests.cs +++ b/tests/Buildvana.Core.Configuration.Tests/BuildvanaConfigFactoryTests.cs @@ -30,6 +30,7 @@ public async Task Create_NoSources_YieldsDomainDefaults() await Assert.That(config.NuGet.Feeds.Prerelease).IsNull(); await Assert.That(config.NuGet.Feeds.Release).IsNull(); await Assert.That(config.Hosts.GitHub.TokenEnv).IsEqualTo("GITHUB_TOKEN"); + await Assert.That(config.Hosts.GitLab.TokenEnv).IsEqualTo("GITLAB_TOKEN"); await Assert.That(config.Git.Identity).IsNull(); await Assert.That(config.Dependencies.Scopes.NetSdk).IsEqualTo("major"); await Assert.That(config.Dependencies.Scopes.Sdks).IsEqualTo("minor"); @@ -58,7 +59,10 @@ public async Task Create_FileValues_OverrideDefaults() AssemblyVersionPrecision = AssemblyVersionPrecision.Build, }, DotNet = new() { Configuration = "Debug" }, - Hosts = new() { GitHub = new() { TokenEnv = "MY_TOKEN" } }, + Hosts = new() { + GitHub = new() { TokenEnv = "MY_TOKEN" }, + GitLab = new() { TokenEnv = "CI_JOB_TOKEN" }, + }, }; var config = BuildvanaConfigFactory.Create(json, null); @@ -76,6 +80,7 @@ public async Task Create_FileValues_OverrideDefaults() await Assert.That(config.Versioning.AssemblyVersionPrecision).IsEqualTo(AssemblyVersionPrecision.Build); await Assert.That(config.DotNet.Configuration).IsEqualTo("Debug"); await Assert.That(config.Hosts.GitHub.TokenEnv).IsEqualTo("MY_TOKEN"); + await Assert.That(config.Hosts.GitLab.TokenEnv).IsEqualTo("CI_JOB_TOKEN"); } [Test] @@ -159,6 +164,9 @@ public async Task Create_TokenEnv_BlankFallsBackToDefault() var whitespace = new BuildvanaJsonConfig { Hosts = new() { GitHub = new() { TokenEnv = " \t " } } }; await Assert.That(BuildvanaConfigFactory.Create(whitespace, null).Hosts.GitHub.TokenEnv).IsEqualTo("GITHUB_TOKEN"); + + var gitLab = new BuildvanaJsonConfig { Hosts = new() { GitLab = new() { TokenEnv = " " } } }; + await Assert.That(BuildvanaConfigFactory.Create(gitLab, null).Hosts.GitLab.TokenEnv).IsEqualTo("GITLAB_TOKEN"); } // A blank URL names no repository, so it counts as not stated, and bv release refuses to run. diff --git a/tests/Buildvana.Runtime.Tests/HookArgsSecretsTests.cs b/tests/Buildvana.Runtime.Tests/HookArgsSecretsTests.cs index d3cfe42e..5b22dd24 100644 --- a/tests/Buildvana.Runtime.Tests/HookArgsSecretsTests.cs +++ b/tests/Buildvana.Runtime.Tests/HookArgsSecretsTests.cs @@ -39,7 +39,10 @@ public async Task Serialize_WritesCredentialVariableNames_NeverTheirValues() ScratchDirectory = "/repo/.buildvana-temp", ConfigFile = null, Configuration = new() { - Hosts = new() { GitHub = new() { TokenEnv = credentialEnvVar } }, + Hosts = new() { + GitHub = new() { TokenEnv = credentialEnvVar }, + GitLab = new() { TokenEnv = credentialEnvVar }, + }, NuGet = new() { Feeds = new() { Release = new() { Source = "https://nuget.example/v3/index.json", ApiKeyEnv = credentialEnvVar }, diff --git a/tests/Buildvana.Tool.Tests/CIPlatformTests.cs b/tests/Buildvana.Tool.Tests/CIPlatformTests.cs index 911a47ee..85b9953f 100644 --- a/tests/Buildvana.Tool.Tests/CIPlatformTests.cs +++ b/tests/Buildvana.Tool.Tests/CIPlatformTests.cs @@ -151,7 +151,7 @@ public async Task Detect_OnGitLabCI_TakesTheServerFromCIServerUrl() var platform = Detect(("GITLAB_CI", "true"), ("CI_SERVER_URL", "https://gitlab.example.invalid")); await Assert.That(platform.ServerUrl!.Host).IsEqualTo("gitlab.example.invalid"); - await Assert.That(platform.ServerKind).IsNull(); + await Assert.That(platform.ServerKind).IsEqualTo(HostKind.GitLab); } // A value that is not an absolute URL names no server, and host detection then has no hint to go on. diff --git a/tests/Buildvana.Tool.Tests/GitHostTests.cs b/tests/Buildvana.Tool.Tests/GitHostTests.cs index 008155e7..7a5d28ec 100644 --- a/tests/Buildvana.Tool.Tests/GitHostTests.cs +++ b/tests/Buildvana.Tool.Tests/GitHostTests.cs @@ -8,6 +8,7 @@ using Buildvana.Tool.Services.Git; using Buildvana.Tool.Services.Hosts; using Buildvana.Tool.Services.Hosts.GitHub; +using Buildvana.Tool.Services.Hosts.GitLab; using Buildvana.Tool.Services.Platforms; using Microsoft.Extensions.DependencyInjection; @@ -22,6 +23,25 @@ public async Task DetectKind_ForGitHubCom_IsGitHub(string url) await Assert.That(GitHost.DetectKind(Repository(url), Parse(url), new FakeCIPlatform())).IsEqualTo(HostKind.GitHub); } + // GitHub Enterprise Cloud with data residency is recognized on any CI platform, as github.com is. + [Test] + [Arguments("https://octocorp.ghe.com/Tenacom/Buildvana")] + [Arguments("https://OctoCorp.GHE.com/Tenacom/Buildvana.git")] + [Arguments("git@octocorp.ghe.com:Tenacom/Buildvana.git")] + public async Task DetectKind_ForAGheComHost_IsGitHub(string url) + { + await Assert.That(GitHost.DetectKind(Repository(url), Parse(url), new FakeCIPlatform())).IsEqualTo(HostKind.GitHub); + } + + [Test] + [Arguments("https://gitlab.com/tenacom/tools/buildvana")] + [Arguments("https://GitLab.com/tenacom/buildvana.git")] + [Arguments("git@gitlab.com:tenacom/buildvana.git")] + public async Task DetectKind_ForGitLabCom_IsGitLab(string url) + { + await Assert.That(GitHost.DetectKind(Repository(url), Parse(url), new FakeCIPlatform())).IsEqualTo(HostKind.GitLab); + } + [Test] public async Task DetectKind_WhenTheConfigurationStatesTheHost_IsThatHost() { @@ -31,15 +51,20 @@ public async Task DetectKind_WhenTheConfigurationStatesTheHost_IsThatHost() await Assert.That(GitHost.DetectKind(repository, Parse(url), new FakeCIPlatform())).IsEqualTo(HostKind.GitHub); } - // A GitHub Enterprise Server is recognized on its own runners, which name it in GITHUB_SERVER_URL. + // A GitHub Enterprise Server is recognized on its own runners, which name it in GITHUB_SERVER_URL, and a + // self-managed GitLab instance on its own, which name it in CI_SERVER_URL. [Test] - [Arguments("https://git.example.invalid", HostKind.GitHub)] - [Arguments("https://GIT.example.invalid/", HostKind.GitHub)] - [Arguments("https://elsewhere.example.invalid", null)] - public async Task DetectKind_OnAPlatformWithAServer_IsThePlatformKind_WhenTheHostNamesMatch(string serverUrl, HostKind? expected) + [Arguments("https://git.example.invalid", HostKind.GitHub, HostKind.GitHub)] + [Arguments("https://GIT.example.invalid/", HostKind.GitHub, HostKind.GitHub)] + [Arguments("https://git.example.invalid", HostKind.GitLab, HostKind.GitLab)] + [Arguments("https://elsewhere.example.invalid", HostKind.GitHub, null)] + public async Task DetectKind_OnAPlatformWithAServer_IsThePlatformKind_WhenTheHostNamesMatch( + string serverUrl, + HostKind serverKind, + HostKind? expected) { const string url = "https://git.example.invalid/tenacom/test-repo"; - var platform = new FakeCIPlatform { Server = new Uri(serverUrl), Kind = HostKind.GitHub }; + var platform = new FakeCIPlatform { Server = new Uri(serverUrl), Kind = serverKind }; await Assert.That(GitHost.DetectKind(Repository(url), Parse(url), platform)).IsEqualTo(expected); } @@ -78,6 +103,16 @@ public async Task Create_ForGitHubCom_CreatesAGitHubHost() } } + [Test] + public async Task Create_ForGitLabCom_CreatesAGitLabHost() + { + var services = Services("https://gitlab.com/tenacom/tools/buildvana"); + await using (services.ConfigureAwait(false)) + { + await Assert.That(GitHost.Create(services)).IsTypeOf(); + } + } + [Test] public async Task Create_ForAnotherHost_CreatesAnUnknownHost() { diff --git a/tests/Buildvana.Tool.Tests/GitHubApiClientTests.cs b/tests/Buildvana.Tool.Tests/GitHubApiClientTests.cs index 9e551260..e1a044dc 100644 --- a/tests/Buildvana.Tool.Tests/GitHubApiClientTests.cs +++ b/tests/Buildvana.Tool.Tests/GitHubApiClientTests.cs @@ -44,6 +44,18 @@ public async Task BaseUrl_ForGitHubCom_IsApiGitHubCom() await Assert.That(client.BaseUrl.AbsoluteUri).IsEqualTo("https://api.github.com/"); } + [Test] + [Arguments("octocorp.ghe.com")] + [Arguments("OctoCorp.GHE.com")] + public async Task BaseUrl_ForAGheComHost_IsTheApiSubdomainOfTheHost(string hostName) + { + var handler = Answering((HttpMethod.Get, "https://api.octocorp.ghe.com/repos/Tenacom/Buildvana"), """{"private":false}"""); + using var client = new GitHubApiClient(hostName, Token, handler); + _ = await client.GetRepositoryAsync(Owner, Repo).ConfigureAwait(false); + await Assert.That(client.BaseUrl.AbsoluteUri).IsEqualTo("https://api.octocorp.ghe.com/"); + await Assert.That(handler.Requests[0].Url).IsEqualTo("https://api.octocorp.ghe.com/repos/Tenacom/Buildvana"); + } + [Test] public async Task BaseUrl_ForAnotherHost_IsApiV3OfTheHost() { diff --git a/tests/Buildvana.Tool.Tests/GitLabApiClientTests.cs b/tests/Buildvana.Tool.Tests/GitLabApiClientTests.cs new file mode 100644 index 00000000..a843121b --- /dev/null +++ b/tests/Buildvana.Tool.Tests/GitLabApiClientTests.cs @@ -0,0 +1,414 @@ +// Copyright (C) Tenacom and Contributors. Licensed under the MIT license. +// See the LICENSE file in the project root for full license information. + +using System.Net; +using Buildvana.Core; +using Buildvana.Tool.Services.Hosts.GitLab; +using Buildvana.Tool.Utilities; + +// The answers are canned here: what the client is judged on is what it sends and what it makes of the shape +// GitLab documents, not whether the network is up. +internal sealed class GitLabApiClientTests +{ + private const string Project = "tenacom/tools/buildvana"; + private const string Token = "secret-token"; + private const string ApiProjectUrl = "https://gitlab.com/api/v4/projects/tenacom%2Ftools%2Fbuildvana"; + private const string PackageFileUrl = ApiProjectUrl + "/packages/generic/buildvana/1.0.0-preview/Foo.1.0.0.nupkg"; + + private static readonly GitLabNewRelease NewRelease = new() { + TagName = "1.0.0-preview", + Name = "1.0.0-preview", + Description = "Text", + Assets = new GitLabReleaseAssets { + Links = [ + new GitLabReleaseLink { + Name = "Foo package", + Url = PackageFileUrl, + DirectAssetPath = "/Foo.1.0.0.nupkg", + LinkType = "package", + }, + ], + }, + }; + + [Test] + public async Task BaseUrl_IsApiV4OfTheHost() + { + using var client = new GitLabApiClient("gitlab.example.com", Token, GitLabTokenKind.AccessToken, Answering()); + await Assert.That(client.BaseUrl.AbsoluteUri).IsEqualTo("https://gitlab.example.com/api/v4/"); + } + + [Test] + [Arguments("", Token)] + [Arguments("gitlab.com", "")] + public async Task Constructor_RejectsEmptyArguments(string hostName, string token) + { + await Assert.That(() => new GitLabApiClient(hostName, token, GitLabTokenKind.AccessToken, Answering())) + .Throws(); + } + + [Test] + public async Task Constructor_RejectsAnUnknownTokenKind() + { + await Assert.That(() => new GitLabApiClient("gitlab.com", Token, (GitLabTokenKind)42, Answering())) + .Throws(); + } + + [Test] + [Arguments(GitLabTokenKind.AccessToken, "PRIVATE-TOKEN")] + [Arguments(GitLabTokenKind.JobToken, "JOB-TOKEN")] + public async Task EveryRequest_CarriesTheApiHeaders(GitLabTokenKind tokenKind, string tokenHeader) + { + var handler = Answering((HttpMethod.Get, ApiProjectUrl), """{"visibility":"public"}"""); + using var client = new GitLabApiClient("gitlab.com", Token, tokenKind, handler); + _ = await client.GetProjectAsync(Project).ConfigureAwait(false); + var headers = handler.Requests[0].Headers; + await Assert.That(headers[tokenHeader]).IsEqualTo(Token); + await Assert.That(headers.Keys).DoesNotContain(tokenKind == GitLabTokenKind.JobToken ? "PRIVATE-TOKEN" : "JOB-TOKEN"); + await Assert.That(headers.Keys).DoesNotContain("Authorization"); + await Assert.That(headers["Accept"]).IsEqualTo("application/json"); + await Assert.That(headers["User-Agent"]).IsEqualTo($"bv/{OwnVersion.Value.ToNormalizedString()}"); + } + + [Test] + public async Task GetCurrentUserAsync_ReadsTheId() + { + var handler = Answering((HttpMethod.Get, "https://gitlab.com/api/v4/user"), """{"id":42,"username":"ric"}"""); + using var client = NewClient(handler); + var user = await client.GetCurrentUserAsync().ConfigureAwait(false); + await Assert.That(user.Id).IsEqualTo(42); + await Assert.That(handler.Requests[0].Headers["PRIVATE-TOKEN"]).IsEqualTo(Token); + } + + [Test] + [Arguments("public")] + [Arguments("internal")] + [Arguments("private")] + public async Task GetProjectAsync_ReadsTheVisibility(string visibility) + { + var handler = Answering((HttpMethod.Get, ApiProjectUrl), $$"""{"id":7,"visibility":"{{visibility}}"}"""); + using var client = NewClient(handler); + var project = await client.GetProjectAsync(Project).ConfigureAwait(false); + await Assert.That(project.Visibility).IsEqualTo(visibility); + await Assert.That(handler.Requests[0].Body).IsNull(); + } + + [Test] + public async Task GetProjectAsync_WhenTheAnswerLacksTheVisibility_Fails() + { + using var client = NewClient(Answering((HttpMethod.Get, ApiProjectUrl), """{"id":7}""")); + + // ReSharper disable once AccessToDisposedClosure // the assertion invokes the delegate before returning + await Assert.That(async () => await client.GetProjectAsync(Project).ConfigureAwait(false)) + .Throws(); + } + + [Test] + public async Task GetProjectWithoutCredentialsAsync_SendsNoToken_AndReadsTheProject() + { + var handler = Answering((HttpMethod.Get, ApiProjectUrl), """{"visibility":"public"}"""); + using var client = new GitLabApiClient("gitlab.com", Token, GitLabTokenKind.JobToken, handler); + var project = await client.GetProjectWithoutCredentialsAsync(Project).ConfigureAwait(false); + await Assert.That(project?.Visibility).IsEqualTo("public"); + var headers = handler.Requests[0].Headers; + await Assert.That(headers.Keys).DoesNotContain("JOB-TOKEN"); + await Assert.That(headers.Keys).DoesNotContain("PRIVATE-TOKEN"); + await Assert.That(headers["User-Agent"]).IsEqualTo($"bv/{OwnVersion.Value.ToNormalizedString()}"); + } + + [Test] + public async Task GetProjectWithoutCredentialsAsync_WhenTheApiAnswersNotFound_ReturnsNull() + { + using var client = NewClient(Answering()); + await Assert.That(await client.GetProjectWithoutCredentialsAsync(Project).ConfigureAwait(false)).IsNull(); + } + + [Test] + public async Task GetProjectMemberAsync_ReadsTheAccessLevelOfEveryMembership() + { + var handler = Answering( + (HttpMethod.Get, ApiProjectUrl + "/members/all/42"), + """{"id":42,"username":"ric","access_level":40}"""); + using var client = NewClient(handler); + var member = await client.GetProjectMemberAsync(Project, 42).ConfigureAwait(false); + await Assert.That(member).IsEqualTo(new GitLabMember { AccessLevel = 40 }); + await Assert.That(handler.Requests[0].Headers["PRIVATE-TOKEN"]).IsEqualTo(Token); + } + + [Test] + public async Task GetProjectMemberAsync_WhenTheApiAnswersNotFound_ReturnsNull() + { + using var client = NewClient(Answering()); + await Assert.That(await client.GetProjectMemberAsync(Project, 42).ConfigureAwait(false)).IsNull(); + } + + [Test] + public async Task GetReleasesAsync_AsksForOnePage_AndReadsTheReleases() + { + var handler = Answering( + (HttpMethod.Get, ApiProjectUrl + "/releases?per_page=1"), + """[{"tag_name":"0.9.0","name":"0.9.0"}]"""); + using var client = NewClient(handler); + var releases = await client.GetReleasesAsync(Project, 1).ConfigureAwait(false); + await Assert.That(releases).IsEquivalentTo([new GitLabRelease { TagName = "0.9.0" }]); + } + + [Test] + public async Task GetReleasesAsync_WhenTheProjectHasNoReleases_ReturnsAnEmptyList() + { + using var client = NewClient(Answering((HttpMethod.Get, ApiProjectUrl + "/releases?per_page=1"), "[]")); + await Assert.That(await client.GetReleasesAsync(Project, 1).ConfigureAwait(false)).IsEmpty(); + } + + [Test] + [Arguments(0)] + [Arguments(101)] + public async Task GetReleasesAsync_RejectsACountOutOfRange(int count) + { + var handler = Answering(); + using var client = NewClient(handler); + + // ReSharper disable once AccessToDisposedClosure // the assertion invokes the delegate before returning + await Assert.That(async () => await client.GetReleasesAsync(Project, count).ConfigureAwait(false)) + .Throws(); + await Assert.That(handler.Requests).IsEmpty(); + } + + [Test] + public async Task CreateReleaseAsync_PostsTheNewRelease_AndReadsTheRelease() + { + var handler = Answering( + (HttpMethod.Post, ApiProjectUrl + "/releases"), + """{"tag_name":"1.0.0-preview","name":"1.0.0-preview","description":"Text"}""", + HttpStatusCode.Created); + using var client = NewClient(handler); + var release = await client.CreateReleaseAsync(Project, NewRelease).ConfigureAwait(false); + await Assert.That(release).IsEqualTo(new GitLabRelease { TagName = "1.0.0-preview" }); + var request = handler.Requests[0]; + await Assert.That(request.Headers["Content-Type"]).IsEqualTo("application/json; charset=utf-8"); + await Assert.That(request.Body).IsEqualTo( + """{"tag_name":"1.0.0-preview","name":"1.0.0-preview","description":"Text","assets":{"links":[""" + + $$$"""{"name":"Foo package","url":"{{{PackageFileUrl}}}","direct_asset_path":"/Foo.1.0.0.nupkg","link_type":"package"}]}}"""); + } + + [Test] + public async Task DeleteReleaseAsync_DeletesTheReleaseOfTheTag() + { + var handler = Answering((HttpMethod.Delete, ApiProjectUrl + "/releases/release%2F1.0"), """{"tag_name":"release/1.0"}"""); + using var client = NewClient(handler); + var deleted = await client.DeleteReleaseAsync(Project, "release/1.0").ConfigureAwait(false); + await Assert.That(deleted).IsTrue(); + await Assert.That(handler.Requests).Count().IsEqualTo(1); + } + + [Test] + public async Task DeleteReleaseAsync_WhenTheReleaseDoesNotExist_ReturnsFalse() + { + var handler = Answering( + (HttpMethod.Delete, ApiProjectUrl + "/releases/1.0"), + """{"message":"404 Not found"}""", + HttpStatusCode.NotFound); + using var client = NewClient(handler); + var deleted = await client.DeleteReleaseAsync(Project, "1.0").ConfigureAwait(false); + await Assert.That(deleted).IsFalse(); + } + + [Test] + public async Task UploadPackageFileAsync_PutsTheContents_AndReadsThePackageId() + { + var handler = Answering( + (HttpMethod.Put, PackageFileUrl + "?select=package_file"), + """{"id":11,"package_id":5,"file_name":"Foo.1.0.0.nupkg","size":14}"""); + using var client = NewClient(handler); + var file = await UploadAsync(client, "Foo.1.0.0.nupkg").ConfigureAwait(false); + await Assert.That(file.PackageId).IsEqualTo(5); + var request = handler.Requests[0]; + await Assert.That(request.Headers["Content-Type"]).IsEqualTo("application/octet-stream"); + await Assert.That(request.Body).IsEqualTo("asset contents"); + } + + [Test] + public async Task UploadPackageFileAsync_WhenTheFileNameIsInvalid_Fails() + { + var handler = Answering(); + using var client = NewClient(handler); + + // ReSharper disable once AccessToDisposedClosure // the assertion invokes the delegate before returning + await Assert.That(async () => await UploadAsync(client, "Foo 1.0.0.nupkg").ConfigureAwait(false)) + .Throws(); + await Assert.That(handler.Requests).IsEmpty(); + } + + [Test] + public async Task GetPackageFileUrl_EscapesEverySegment() + { + using var client = NewClient(Answering()); + var url = client.GetPackageFileUrl(Project, "buildvana", "1.0.0+build.7", "Foo@1.0.0.zip"); + await Assert.That(url.AbsoluteUri) + .IsEqualTo(ApiProjectUrl + "/packages/generic/buildvana/1.0.0%2Bbuild.7/Foo%401.0.0.zip"); + } + + [Test] + public async Task DeletePackageAsync_DeletesThePackage() + { + var handler = Answering((HttpMethod.Delete, ApiProjectUrl + "/packages/5"), null, HttpStatusCode.NoContent); + using var client = NewClient(handler); + await client.DeletePackageAsync(Project, 5).ConfigureAwait(false); + await Assert.That(handler.Requests).Count().IsEqualTo(1); + } + + [Test] + [Arguments("Foo.1.0.0.nupkg", true)] + [Arguments("Foo_Bar-1.0.0+build~7@x.zip", true)] + [Arguments("a", true)] + [Arguments("", false)] + [Arguments("Foo 1.0.0.nupkg", false)] + [Arguments("dir/Foo.nupkg", false)] + [Arguments("Föö.nupkg", false)] + [Arguments("~Foo.nupkg", false)] + [Arguments("Foo.nupkg~", false)] + [Arguments("@Foo.nupkg", false)] + [Arguments("Foo.nupkg@", false)] + public async Task IsValidPackageFileName_FollowsTheRuleOfTheRegistry(string fileName, bool expected) + { + await Assert.That(GitLabApiClient.IsValidPackageFileName(fileName)).IsEqualTo(expected); + } + + // The rule wants a character after the slash, then at least one more of the class, then a letter or a digit, + // so the shortest valid path holds three characters after the slash. + [Test] + [Arguments("/Foo.1.0.0.nupkg", true)] + [Arguments("/bin/Foo_Bar-1.0.zip", true)] + [Arguments("/abc", true)] + [Arguments("/ab", false)] + [Arguments("Foo.nupkg", false)] + [Arguments("//Foo.nupkg", false)] + [Arguments("/bin//Foo.nupkg", false)] + [Arguments("/Foo+build.nupkg", false)] + [Arguments("/Foo.nupkg.", false)] + [Arguments("/Föö.nupkg", false)] + public async Task IsValidDirectAssetPath_FollowsTheRuleOfReleaseLinks(string path, bool expected) + { + await Assert.That(GitLabApiClient.IsValidDirectAssetPath(path)).IsEqualTo(expected); + } + + [Test] + [Arguments(127, true)] + [Arguments(128, false)] + public async Task IsValidDirectAssetPath_AcceptsAtMost128Characters(int nameLength, bool expected) + { + await Assert.That(GitLabApiClient.IsValidDirectAssetPath("/" + new string('a', nameLength))).IsEqualTo(expected); + } + + [Test] + [Arguments("GetCurrentUser")] + [Arguments("GetProject")] + [Arguments("GetProjectWithoutCredentials")] + [Arguments("GetProjectMember")] + [Arguments("GetReleases")] + [Arguments("CreateRelease")] + [Arguments("DeleteRelease")] + [Arguments("UploadPackageFile")] + [Arguments("DeletePackage")] + public async Task Operation_WhenTheApiRefuses_FailsWithTheStatusAndTheMessage(string operation) + { + using var client = NewClient(AnsweringEverything( + new StubHttpAnswer(HttpStatusCode.Forbidden, """{"message":"403 Forbidden"}"""))); + + // ReSharper disable once AccessToDisposedClosure // the assertion invokes the delegate before returning + var exception = await Assert.That(async () => await InvokeAsync(client, operation).ConfigureAwait(false)) + .Throws(); + await Assert.That(exception!.Message).Contains("403 Forbidden: 403 Forbidden"); + } + + [Test] + [Arguments("""{"message":"404 Project Not Found"}""", ": 404 Project Not Found")] + [Arguments("""{"message":["Tag name is missing"]}""", """: ["Tag name is missing"]""")] + [Arguments("""{"message":{"name":["has already been taken"]}}""", """: {"name":["has already been taken"]}""")] + [Arguments("""{"error":"file_name is invalid"}""", ": file_name is invalid")] + [Arguments("""{"message":"","error":"tag_name is missing"}""", ": tag_name is missing")] + public async Task Operation_WhenTheApiRefuses_QuotesTheMessageWhateverItsShape(string body, string expected) + { + using var client = NewClient(AnsweringEverything(new StubHttpAnswer(HttpStatusCode.BadRequest, body))); + + // ReSharper disable once AccessToDisposedClosure // the assertion invokes the delegate before returning + var exception = await Assert.That(async () => await client.GetProjectAsync(Project).ConfigureAwait(false)) + .Throws(); + await Assert.That(exception!.Message).EndsWith("with 400 Bad Request" + expected); + } + + [Test] + [Arguments("Bad gateway")] + [Arguments("")] + [Arguments("{}")] + [Arguments("null")] + [Arguments("""["Bad gateway"]""")] + [Arguments("""{"message":null}""")] + public async Task Operation_WhenTheErrorAnswerStatesNoMessage_FailsWithTheStatus(string body) + { + using var client = NewClient(AnsweringEverything(new StubHttpAnswer(HttpStatusCode.BadGateway, body))); + + // ReSharper disable once AccessToDisposedClosure // the assertion invokes the delegate before returning + var exception = await Assert.That(async () => await client.GetProjectAsync(Project).ConfigureAwait(false)) + .Throws(); + await Assert.That(exception!.Message).EndsWith("with 502 Bad Gateway."); + } + + [Test] + [Arguments("not json")] + [Arguments("null")] + public async Task GetProjectWithoutCredentialsAsync_WhenTheAnswerCannotBeRead_Fails(string body) + { + using var client = NewClient(Answering((HttpMethod.Get, ApiProjectUrl), body)); + + // ReSharper disable once AccessToDisposedClosure // the assertion invokes the delegate before returning + await Assert.That(async () => await client.GetProjectWithoutCredentialsAsync(Project).ConfigureAwait(false)) + .Throws(); + } + + [Test] + public async Task GetProjectAsync_WhenTheHostCannotBeReached_Fails() + { + using var client = NewClient(AnsweringEverything(StubHttpAnswer.Throwing(new HttpRequestException("No such host is known.")))); + + // ReSharper disable once AccessToDisposedClosure // the assertion invokes the delegate before returning + await Assert.That(async () => await client.GetProjectAsync(Project).ConfigureAwait(false)) + .Throws(); + } + + private static GitLabApiClient NewClient(StubHttpMessageHandler handler) + => new("gitlab.com", Token, GitLabTokenKind.AccessToken, handler); + + private static StubHttpMessageHandler Answering() => new(new Dictionary<(HttpMethod Method, string Url), StubHttpAnswer>()); + + private static StubHttpMessageHandler AnsweringEverything(StubHttpAnswer answer) + => new(new Dictionary<(HttpMethod Method, string Url), StubHttpAnswer>(), answer); + + private static StubHttpMessageHandler Answering( + (HttpMethod Method, string Url) request, + string? body, + HttpStatusCode status = HttpStatusCode.OK) + => new(new Dictionary<(HttpMethod Method, string Url), StubHttpAnswer> { [request] = new(status, body) }); + + private static Task InvokeAsync(GitLabApiClient client, string operation) + => operation switch { + "GetCurrentUser" => client.GetCurrentUserAsync(), + "GetProject" => client.GetProjectAsync(Project), + "GetProjectWithoutCredentials" => client.GetProjectWithoutCredentialsAsync(Project), + "GetProjectMember" => client.GetProjectMemberAsync(Project, 42), + "GetReleases" => client.GetReleasesAsync(Project, 1), + "CreateRelease" => client.CreateReleaseAsync(Project, NewRelease), + "DeleteRelease" => client.DeleteReleaseAsync(Project, "1.0.0-preview"), + "UploadPackageFile" => UploadAsync(client, "Foo.1.0.0.nupkg"), + "DeletePackage" => client.DeletePackageAsync(Project, 5), + _ => throw new ArgumentOutOfRangeException(nameof(operation), operation, "Unknown operation."), + }; + + private static Task UploadAsync(GitLabApiClient client, string fileName) + => client.UploadPackageFileAsync( + Project, + "buildvana", + "1.0.0-preview", + fileName, + new MemoryStream("asset contents"u8.ToArray())); +} diff --git a/tests/Buildvana.Tool.Tests/GitLabHostTests.cs b/tests/Buildvana.Tool.Tests/GitLabHostTests.cs new file mode 100644 index 00000000..b1aa03c3 --- /dev/null +++ b/tests/Buildvana.Tool.Tests/GitLabHostTests.cs @@ -0,0 +1,182 @@ +// Copyright (C) Tenacom and Contributors. Licensed under the MIT license. +// See the LICENSE file in the project root for full license information. + +using System.Net; +using Buildvana.Core; +using Buildvana.Core.ConsoleOutput; +using Buildvana.Runtime; +using Buildvana.Tool.Services.Git; +using Buildvana.Tool.Services.Hosts; +using Buildvana.Tool.Services.Hosts.GitLab; +using Microsoft.Extensions.DependencyInjection; + +internal sealed class GitLabHostTests +{ + private const string TokenVariable = "BV_TEST_GITLAB_HOST_TOKEN"; + private const string ApiProjectUrl = "https://gitlab.com/api/v4/projects/tenacom%2Ftools%2Fbuildvana"; + + [Test] + public async Task Members_DescribeTheProject() + { + var services = Services(TokenVariable); + await using (services.ConfigureAwait(false)) + { + using var host = new GitLabHost(services, Parse("git@gitlab.com:tenacom/tools/buildvana.git")); + + await Assert.That(host.Name).IsEqualTo("GitLab"); + await Assert.That(host.Kind).IsEqualTo(HostKind.GitLab); + await Assert.That(host.ProjectPath).IsEqualTo("tenacom/tools/buildvana"); + await Assert.That(host.RepositoryUrl.AbsoluteUri).IsEqualTo("https://gitlab.com/tenacom/tools/buildvana"); + await Assert.That(host.TokenEnv).IsEqualTo(TokenVariable); + await Assert.That(host.GetReleaseUrl("1.0.0").AbsoluteUri) + .IsEqualTo("https://gitlab.com/tenacom/tools/buildvana/-/releases/1.0.0"); + } + } + + // The name of the variable is the one way to tell a job token from an access token. + [Test] + [Arguments(TokenVariable, GitLabTokenKind.AccessToken, "oauth2")] + [Arguments("GITLAB_TOKEN", GitLabTokenKind.AccessToken, "oauth2")] + [Arguments("CI_JOB_TOKEN", GitLabTokenKind.JobToken, "gitlab-ci-token")] + [Arguments("ci_job_token", GitLabTokenKind.AccessToken, "oauth2")] + public async Task TokenKind_AndPushUsername_FollowTheNameOfTheVariable( + string tokenEnv, + GitLabTokenKind expectedKind, + string expectedUsername) + { + var services = Services(tokenEnv); + await using (services.ConfigureAwait(false)) + { + using var host = new GitLabHost(services, Parse("https://gitlab.com/tenacom/buildvana")); + + await Assert.That(host.TokenKind).IsEqualTo(expectedKind); + await Assert.That(host.PushUsername).IsEqualTo(expectedUsername); + } + } + + // GitLab puts a tag, a branch, and a commit in the same place of a file URL. + [Test] + [Arguments(GitRefKind.Tag)] + [Arguments(GitRefKind.Branch)] + [Arguments(GitRefKind.Commit)] + public async Task GetFileUrl_TakesTheNameOfAnyKindOfReference(GitRefKind kind) + { + var services = Services(TokenVariable); + await using (services.ConfigureAwait(false)) + { + using var host = new GitLabHost(services, Parse("https://gitlab.com/tenacom/tools/buildvana")); + var gitRef = kind switch { + GitRefKind.Tag => GitRef.Tag("v1"), + GitRefKind.Branch => GitRef.Branch("v1"), + _ => GitRef.Commit("v1"), + }; + + var url = host.GetFileUrl(@"docs\README.md", gitRef); + + await Assert.That(url.AbsoluteUri).IsEqualTo("https://gitlab.com/tenacom/tools/buildvana/-/blob/v1/docs/README.md"); + } + } + + [Test] + [Arguments("../outside.md")] + [Arguments("/docs/README.md")] + public async Task GetFileUrl_ForAPathOutsideTheRepository_Throws(string path) + { + var services = Services(TokenVariable); + await using (services.ConfigureAwait(false)) + { + using var host = new GitLabHost(services, Parse("https://gitlab.com/tenacom/buildvana")); + + // ReSharper disable once AccessToDisposedClosure // False positive: the assertion invokes the lambda before host is disposed + await Assert.That(() => host.GetFileUrl(path, GitRef.Branch("main"))).Throws(); + } + } + + // Only users of the instance can see an internal project, so it counts as private. + [Test] + [NotInParallel] + [Arguments("private", true)] + [Arguments("internal", true)] + [Arguments("public", false)] + public async Task IsPrivateRepositoryAsync_WithAnAccessToken_ReadsTheVisibility(string visibility, bool expected) + { + var services = Services(TokenVariable); + await using (services.ConfigureAwait(false)) + { + var api = new StubHttpMessageHandler(new Dictionary { + [ApiProjectUrl] = $$"""{"visibility":"{{visibility}}"}""", + }); + using var host = new GitLabHost(services, Parse("https://gitlab.com/tenacom/tools/buildvana"), api); + Environment.SetEnvironmentVariable(TokenVariable, "secret"); + try + { + await Assert.That(await host.IsPrivateRepositoryAsync().ConfigureAwait(false)).IsEqualTo(expected); + await Assert.That(api.Requests[0].Headers["PRIVATE-TOKEN"]).IsEqualTo("secret"); + } + finally + { + Environment.SetEnvironmentVariable(TokenVariable, null); + } + } + } + + // A job token cannot read a project, so the request carries no credentials, and GitLab answers 404 for + // every project that is not public. + [Test] + [NotInParallel] + [Arguments(HttpStatusCode.OK, false)] + [Arguments(HttpStatusCode.NotFound, true)] + public async Task IsPrivateRepositoryAsync_WithAJobToken_ReadsTheProjectWithoutCredentials( + HttpStatusCode status, + bool expected) + { + var services = Services(GitLabHost.JobTokenVariable); + await using (services.ConfigureAwait(false)) + { + var api = new StubHttpMessageHandler(new Dictionary<(HttpMethod Method, string Url), StubHttpAnswer> { + [(HttpMethod.Get, ApiProjectUrl)] = new(status, """{"visibility":"public"}"""), + }); + using var host = new GitLabHost(services, Parse("https://gitlab.com/tenacom/tools/buildvana"), api); + var originalToken = Environment.GetEnvironmentVariable(GitLabHost.JobTokenVariable); + Environment.SetEnvironmentVariable(GitLabHost.JobTokenVariable, "job-secret"); + try + { + await Assert.That(await host.IsPrivateRepositoryAsync().ConfigureAwait(false)).IsEqualTo(expected); + var headers = api.Requests[0].Headers; + await Assert.That(headers.Keys).DoesNotContain("JOB-TOKEN"); + await Assert.That(headers.Keys).DoesNotContain("PRIVATE-TOKEN"); + } + finally + { + Environment.SetEnvironmentVariable(GitLabHost.JobTokenVariable, originalToken); + } + } + } + + // The client is created on first use, so a missing token fails the call that needs the API, and nothing before it. + [Test] + [NotInParallel] + public async Task IsPrivateRepositoryAsync_WithoutAToken_Throws() + { + var services = Services(TokenVariable); + await using (services.ConfigureAwait(false)) + { + using var api = new StubHttpMessageHandler(new Dictionary<(HttpMethod Method, string Url), StubHttpAnswer>()); + using var host = new GitLabHost(services, Parse("https://gitlab.com/tenacom/buildvana"), api); + Environment.SetEnvironmentVariable(TokenVariable, null); + + // ReSharper disable once AccessToDisposedClosure // False positive: the assertion invokes the lambda before host is disposed + var exception = await Assert.That(host.IsPrivateRepositoryAsync).Throws(); + await Assert.That(exception!.Message).Contains(TokenVariable); + await Assert.That(api.Requests).IsEmpty(); + } + } + + private static GitUrlInfo Parse(string url) => GitUrlInfo.TryCreate(url, out var info) ? info : throw new InvalidOperationException(url); + + private static ServiceProvider Services(string tokenEnv) + => new ServiceCollection() + .AddSingleton(new BuildvanaConfig { Hosts = new() { GitLab = new() { TokenEnv = tokenEnv } } }) + .AddSingleton(NullReporter.Instance) + .BuildServiceProvider(); +} diff --git a/tests/Buildvana.Tool.Tests/GitLabReleasePublisherTests.cs b/tests/Buildvana.Tool.Tests/GitLabReleasePublisherTests.cs new file mode 100644 index 00000000..b93ad526 --- /dev/null +++ b/tests/Buildvana.Tool.Tests/GitLabReleasePublisherTests.cs @@ -0,0 +1,322 @@ +// Copyright (C) Tenacom and Contributors. Licensed under the MIT license. +// See the LICENSE file in the project root for full license information. + +using System.Net; +using Buildvana.Core; +using Buildvana.Core.ConsoleOutput; +using Buildvana.Core.HomeDirectory; +using Buildvana.Core.Testing; +using Buildvana.Runtime; +using Buildvana.Tool.Services.Git; +using Buildvana.Tool.Services.Hosts; +using Buildvana.Tool.Services.Hosts.GitLab; +using Buildvana.Tool.Services.Release; +using Microsoft.Extensions.DependencyInjection; +using TUnit.Assertions.Enums; + +// The publisher runs over a real GitLabHost, whose API answers from a table, and a real Git repository, whose +// current branch names the changelog the release description links. +[NotInParallel] +internal sealed class GitLabReleasePublisherTests +{ + private const string TokenVariable = "BV_TEST_GITLAB_PUBLISHER_TOKEN"; + private const string Tag = "1.0.0"; + private const string ApiUrl = "https://gitlab.com/api/v4"; + private const string ApiProjectUrl = ApiUrl + "/projects/tenacom%2Fbuildvana"; + private const string ReleasesUrl = ApiProjectUrl + "/releases"; + private const string PackageUrl = ApiProjectUrl + "/packages/generic/buildvana/" + Tag; + private const string MemberUrl = ApiProjectUrl + "/members/all/42"; + + [Test] + public async Task BeginAsync_WithAnAccessToken_ReadsTheReleases_ThenTheRoleOfTheUser() + { + var api = Answering(Member(40)); + + await RunAsync(api, static (publisher, _) => publisher.BeginAsync()).ConfigureAwait(false); + + string[] expected = [ReleasesUrl + "?per_page=1", ApiUrl + "/user", MemberUrl]; + await Assert.That(api.Requests.Select(static x => x.Url)).IsEquivalentTo(expected, CollectionOrdering.Matching); + await Assert.That(api.Requests.All(static x => x.Headers["PRIVATE-TOKEN"] == "secret")).IsTrue(); + } + + [Test] + public async Task BeginAsync_BelowTheMaintainerRole_Throws() + { + using var api = Answering(Member(30)); + + // ReSharper disable once AccessToDisposedClosure // False positive: the assertion invokes the lambda before api is disposed + var exception = await Assert.That(() => RunAsync(api, static (publisher, _) => publisher.BeginAsync())) + .Throws(); + + await Assert.That(exception!.Message).Contains("access level 30 on tenacom/buildvana, below the Maintainer role"); + } + + // An instance administrator, or an organization owner, has no membership and can still delete a package. + [Test] + public async Task BeginAsync_WhenTheUserIsNotAMember_Proceeds() + { + var api = Answering(); + var reporter = new CaptureReporter(); + + await RunAsync(api, static (publisher, _) => publisher.BeginAsync(), reporter).ConfigureAwait(false); + + await Assert.That(api.Requests[^1].Url).IsEqualTo(MemberUrl); + await Assert.That(reporter.Messages.Select(static x => x.Message)) + .Contains("Role check skipped: the user of the token is not a member of the project."); + } + + [Test] + public async Task BeginAsync_WithAJobToken_ReadsTheReleasesAlone() + { + var api = Answering(); + + await RunAsync(api, static (publisher, _) => publisher.BeginAsync(), tokenEnv: GitLabHost.JobTokenVariable) + .ConfigureAwait(false); + + await Assert.That(api.Requests).Count().IsEqualTo(1); + await Assert.That(api.Requests[0].Url).IsEqualTo(ReleasesUrl + "?per_page=1"); + await Assert.That(api.Requests[0].Headers["JOB-TOKEN"]).IsEqualTo("secret"); + } + + [Test] + public async Task BeginAsync_WhenTheReleasesCannotBeRead_Throws_AndReadsNothingElse() + { + using var api = new StubHttpMessageHandler(new Dictionary<(HttpMethod Method, string Url), StubHttpAnswer> { + [(HttpMethod.Get, ReleasesUrl + "?per_page=1")] = new(HttpStatusCode.Forbidden, """{"message":"403 Forbidden"}"""), + }); + + // ReSharper disable once AccessToDisposedClosure // False positive: the assertion invokes the lambda before api is disposed + var exception = await Assert.That(() => RunAsync(api, static (publisher, _) => publisher.BeginAsync())) + .Throws(); + + await Assert.That(exception!.Message).Contains("403 Forbidden"); + await Assert.That(api.Requests).Count().IsEqualTo(1); + } + + [Test] + public async Task PublishAsync_UploadsEachAsset_ThenCreatesAReleaseThatLinksThem() + { + var api = Answering(); + + await RunAsync(api, static (publisher, assets) => publisher.PublishAsync(Tag, "abc123", assets)).ConfigureAwait(false); + + var requests = api.Requests; + await Assert.That(requests).Count().IsEqualTo(3); + await Assert.That(requests[0].Method).IsEqualTo(HttpMethod.Put); + await Assert.That(requests[0].Url).IsEqualTo(PackageUrl + "/Foo.1.0.0.nupkg?select=package_file"); + await Assert.That(requests[0].Body).IsEqualTo("fake package"); + await Assert.That(requests[1].Url).IsEqualTo(PackageUrl + "/setup.exe?select=package_file"); + var release = requests[2]; + await Assert.That(release.Method).IsEqualTo(HttpMethod.Post); + await Assert.That(release.Url).IsEqualTo(ReleasesUrl); + await Assert.That(release.Body).Contains($"\"tag_name\":\"{Tag}\",\"name\":\"{Tag}\""); + await Assert.That(release.Body).Contains("(https://gitlab.com/tenacom/buildvana/-/blob/release/CHANGELOG.md)"); + const string packageUrl = PackageUrl + "/Foo.1.0.0.nupkg"; + const string setupUrl = PackageUrl + "/setup.exe"; + await Assert.That(release.Body).Contains( + $$"""{"name":"Foo.1.0.0.nupkg","url":"{{packageUrl}}","direct_asset_path":"/Foo.1.0.0.nupkg","link_type":"package"}"""); + await Assert.That(release.Body).Contains( + $$"""{"name":"Setup program","url":"{{setupUrl}}","direct_asset_path":"/setup.exe","link_type":"package"}"""); + } + + // The release goes first, because its links point at the files of the package. + [Test] + public async Task UndoAsync_AfterPublishAsync_DeletesTheRelease_ThenThePackage() + { + var api = Answering(); + var reporter = new CaptureReporter(); + + await RunAsync( + api, + static async (publisher, assets) => { + await publisher.PublishAsync(Tag, "abc123", assets).ConfigureAwait(false); + await publisher.UndoAsync().ConfigureAwait(false); + await publisher.UndoAsync().ConfigureAwait(false); + }, + reporter).ConfigureAwait(false); + + var deletions = api.Requests.Where(static x => x.Method == HttpMethod.Delete).Select(static x => x.Url); + string[] expected = [ReleasesUrl + "/" + Tag, ApiProjectUrl + "/packages/7"]; + await Assert.That(deletions).IsEquivalentTo(expected, CollectionOrdering.Matching); + var notices = reporter.Messages.Where(static x => x.Level == MessageLevel.Notice).Select(static x => x.Message); + string[] expectedNotices = [$"Deleted release {Tag}.", $"Deleted generic package buildvana {Tag}."]; + await Assert.That(notices).IsEquivalentTo(expectedNotices, CollectionOrdering.Matching); + } + + // A rollback deletes the release tag first, and GitLab deletes the release with it. + [Test] + public async Task UndoAsync_WhenTheReleaseIsGone_DeletesThePackage() + { + var api = Answering(releaseDeletion: new(HttpStatusCode.NotFound, """{"message":"404 Not found"}""")); + var reporter = new CaptureReporter(); + + await RunAsync(api, PublishThenUndoAsync, reporter).ConfigureAwait(false); + + await Assert.That(api.Requests[^1].Url).IsEqualTo(ApiProjectUrl + "/packages/7"); + var notices = reporter.Messages.Where(static x => x.Level == MessageLevel.Notice).Select(static x => x.Message); + string[] expectedNotices = [$"Deleted generic package buildvana {Tag}."]; + await Assert.That(notices).IsEquivalentTo(expectedNotices, CollectionOrdering.Matching); + await Assert.That(reporter.Messages.Any(static x => x.Level == MessageLevel.Warning)).IsFalse(); + } + + [Test] + public async Task UndoAsync_WhenTheReleaseDeletionFails_WarnsAndDeletesThePackage() + { + var api = Answering(releaseDeletion: new(HttpStatusCode.InternalServerError, """{"message":"500 Internal Server Error"}""")); + var reporter = new CaptureReporter(); + + await RunAsync(api, PublishThenUndoAsync, reporter).ConfigureAwait(false); + + await Assert.That(api.Requests[^1].Url).IsEqualTo(ApiProjectUrl + "/packages/7"); + var warnings = reporter.Messages.Where(static x => x.Level == MessageLevel.Warning).Select(static x => x.Message).ToArray(); + await Assert.That(warnings.Length).IsEqualTo(1); + await Assert.That(warnings[0]).StartsWith($"BuildFailedException while deleting release {Tag}:"); + } + + [Test] + public async Task UndoAsync_WhenTheReleaseWasNotCreated_DeletesThePackage() + { + var api = Answering(release: new(HttpStatusCode.BadRequest, """{"message":{"links":["is invalid"]}}""")); + + await RunAsync( + api, + static async (publisher, assets) => { + await Assert.That(() => publisher.PublishAsync(Tag, "abc123", assets)).Throws(); + await publisher.UndoAsync().ConfigureAwait(false); + }).ConfigureAwait(false); + + var deletions = api.Requests.Where(static x => x.Method == HttpMethod.Delete).Select(static x => x.Url); + string[] expected = [ApiProjectUrl + "/packages/7"]; + await Assert.That(deletions).IsEquivalentTo(expected, CollectionOrdering.Matching); + } + + [Test] + public async Task UndoAsync_WithNothingPublished_SendsNothing() + { + var api = Answering(); + + await RunAsync(api, static (publisher, _) => publisher.UndoAsync()).ConfigureAwait(false); + + await Assert.That(api.Requests).IsEmpty(); + } + + [Test] + public async Task CheckAssets_ReportsEveryAssetGitLabRefuses() + { + ReleaseAsset[] assets = [ + new("out/Foo 1.zip", "Foo", "application/zip"), + new("out/a+b.zip", "Plus", "application/zip"), + new("one/Bar.zip", "Bar", "application/zip"), + new("two/Bar.zip", "Bar", "application/zip"), + ]; + + using var api = Answering(); + + // ReSharper disable once AccessToDisposedClosure // False positive: the assertion invokes the lambda before api is disposed + var exception = await Assert.That(() => RunAsync(api, (publisher, _) => CheckAssetsAsync(publisher, assets))) + .Throws(); + + await Assert.That(api.Requests).IsEmpty(); + await Assert.That(exception!.Message).IsEqualTo( + "GitLab refuses the release assets:" + + "\n- 'Foo 1.zip' is not a valid file name in the generic package registry." + + "\n- 'a+b.zip' is not a valid file name for a release link." + + "\n- More than one asset has the file name 'Bar.zip'." + + "\n- More than one asset has the description 'Bar'."); + } + + [Test] + public async Task CheckAssets_AcceptsAssetsGitLabAccepts() + { + ReleaseAsset[] assets = [ + new("out/Foo.1.0.0.nupkg", "Foo.1.0.0.nupkg", "application/octet-stream"), + new("out/setup.exe", "Setup program", "application/vnd.microsoft.portable-executable"), + ]; + + using var api = Answering(); + + // ReSharper disable once AccessToDisposedClosure // False positive: the assertion invokes the lambda before api is disposed + await Assert.That(() => RunAsync(api, (publisher, _) => CheckAssetsAsync(publisher, assets))).ThrowsNothing(); + } + + private static Task CheckAssetsAsync(ReleasePublisher publisher, IReadOnlyList assets) + { + publisher.CheckAssets(assets); + return Task.CompletedTask; + } + + private static async Task PublishThenUndoAsync(ReleasePublisher publisher, IReadOnlyList assets) + { + await publisher.PublishAsync(Tag, "abc123", assets).ConfigureAwait(false); + await publisher.UndoAsync().ConfigureAwait(false); + } + + private static StubHttpAnswer Member(int accessLevel) + => new(HttpStatusCode.OK, $$"""{"id":42,"access_level":{{accessLevel}}}"""); + + // The releases, the user, every upload, and the deletions have answers. The membership has one when + // member is not null. The release creation and the release deletion succeed unless release and + // releaseDeletion name other answers. + private static StubHttpMessageHandler Answering( + StubHttpAnswer? member = null, + StubHttpAnswer? release = null, + StubHttpAnswer? releaseDeletion = null) + { + var answers = new Dictionary<(HttpMethod Method, string Url), StubHttpAnswer> { + [(HttpMethod.Get, ReleasesUrl + "?per_page=1")] = new(HttpStatusCode.OK, "[]"), + [(HttpMethod.Get, ApiUrl + "/user")] = new(HttpStatusCode.OK, """{"id":42}"""), + [(HttpMethod.Put, PackageUrl + "/Foo.1.0.0.nupkg?select=package_file")] = new(HttpStatusCode.OK, """{"package_id":7}"""), + [(HttpMethod.Put, PackageUrl + "/setup.exe?select=package_file")] = new(HttpStatusCode.OK, """{"package_id":7}"""), + [(HttpMethod.Post, ReleasesUrl)] = release ?? new(HttpStatusCode.Created, $$"""{"tag_name":"{{Tag}}"}"""), + [(HttpMethod.Delete, ReleasesUrl + "/" + Tag)] = releaseDeletion ?? new(HttpStatusCode.OK, $$"""{"tag_name":"{{Tag}}"}"""), + [(HttpMethod.Delete, ApiProjectUrl + "/packages/7")] = new(HttpStatusCode.NoContent), + }; + if (member is not null) + { + answers[(HttpMethod.Get, MemberUrl)] = member; + } + + return new StubHttpMessageHandler(answers); + } + + private static async Task RunAsync( + StubHttpMessageHandler api, + Func, Task> act, + CaptureReporter? reporter = null, + string tokenEnv = TokenVariable) + { + using var repo = new TempGitRepo(); + repo.WriteFile("Foo.1.0.0.nupkg", "fake package"); + repo.WriteFile("setup.exe", "fake setup"); + repo.CommitAll(); + repo.CheckoutNewBranch("release"); + ReleaseAsset[] assets = [ + new(Path.Combine(repo.RootPath, "Foo.1.0.0.nupkg"), "Foo.1.0.0.nupkg", "application/octet-stream"), + new(Path.Combine(repo.RootPath, "setup.exe"), "Setup program", "application/vnd.microsoft.portable-executable"), + ]; + + reporter ??= new CaptureReporter(); + using var git = new GitService(reporter, new FixedHomeDirectoryProvider(repo.RootPath)); + var services = new ServiceCollection() + .AddSingleton(new BuildvanaConfig { Hosts = new() { GitLab = new() { TokenEnv = tokenEnv } } }) + .AddSingleton(reporter) + .AddSingleton(git) + .BuildServiceProvider(); + await using (services.ConfigureAwait(false)) + { + using var host = new GitLabHost(services, Parse("https://gitlab.com/tenacom/buildvana"), api); + var originalToken = Environment.GetEnvironmentVariable(tokenEnv); + Environment.SetEnvironmentVariable(tokenEnv, "secret"); + try + { + await act(host.CreateReleasePublisher(), assets).ConfigureAwait(false); + } + finally + { + Environment.SetEnvironmentVariable(tokenEnv, originalToken); + } + } + } + + private static GitUrlInfo Parse(string url) => GitUrlInfo.TryCreate(url, out var info) ? info : throw new InvalidOperationException(url); +} diff --git a/tests/Buildvana.Tool.Tests/RecordingReleasePublisher.cs b/tests/Buildvana.Tool.Tests/RecordingReleasePublisher.cs index 25010fdb..c90d29b3 100644 --- a/tests/Buildvana.Tool.Tests/RecordingReleasePublisher.cs +++ b/tests/Buildvana.Tool.Tests/RecordingReleasePublisher.cs @@ -43,6 +43,11 @@ internal sealed class RecordingReleasePublisher : ReleasePublisher /// public IReadOnlyList PublishedAssets => _publishedAssets; + /// + /// Gets the assets the release had checked, or until they are checked. + /// + public IReadOnlyList? CheckedAssets { get; private set; } + /// /// Gets a callback invoked while publishing, after the assets have been recorded, simulating the /// host's behavior. Exceptions thrown by the callback propagate to the caller, simulating a failed @@ -57,6 +62,9 @@ public override Task BeginAsync() return Task.CompletedTask; } + /// + public override void CheckAssets(IReadOnlyList assets) => CheckedAssets = [.. assets]; + /// public override Task PublishAsync(string tag, string sha, IReadOnlyList assets) { diff --git a/tests/Buildvana.Tool.Tests/ReleaseCommandGitHubTests.cs b/tests/Buildvana.Tool.Tests/ReleaseCommandGitHubTests.cs index 6ad9a4c2..8a38e5da 100644 --- a/tests/Buildvana.Tool.Tests/ReleaseCommandGitHubTests.cs +++ b/tests/Buildvana.Tool.Tests/ReleaseCommandGitHubTests.cs @@ -49,6 +49,36 @@ public async Task Release_CreatesADraft_UploadsTheAssets_AndPublishesOnTheTag() await Assert.That(harness.Repo.GetRemoteTagSha(ReleasedVersion)).IsEqualTo(releaseCommitSha); } + // The assets are checked before the push, so a refused asset reaches neither the remote nor the feed. + [Test] + public async Task Release_WithTwoAssetsOfTheSameFileName_FailsBeforeThePush() + { + var api = Answering(new StubHttpAnswer(HttpStatusCode.Created, """{"id":7}""")); + using var harness = new ReleaseHarness(new() { GitHubApi = api, WithHook = true }); + var remoteTip = harness.Repo.GetRemoteTipSha(); + var artifactsPath = harness.ArtifactsPath; + + harness.HookBehavior = () => { + var directory = Directory.CreateDirectory(Path.Combine(artifactsPath, "other")).FullName; + var path = Path.Combine(directory, "setup.exe"); + File.WriteAllText(path, "fake setup"); + File.WriteAllText( + Path.Combine(artifactsPath, "Other.assets.txt"), + $"{path}\tapplication/vnd.microsoft.portable-executable\tOther setup program\n"); + }; + + // ReSharper disable once AccessToDisposedClosure // False positive: the assertion invokes Act before the harness is disposed + Task Act() => harness.RunAsync(); + + var exception = await Assert.That(Act).Throws(); + await Assert.That(exception!.Message).IsEqualTo( + "GitHub refuses the release assets:\n- More than one asset has the file name 'setup.exe'."); + await Assert.That(harness.Events.Select(static x => x.Name)).DoesNotContain("nuget-push"); + await Assert.That(api.Requests.Any(static x => x.Url.StartsWith(UploadUrl, StringComparison.Ordinal))).IsFalse(); + await Assert.That(harness.Repo.GetRemoteTipSha()).IsEqualTo(remoteTip); + await Assert.That(harness.Repo.GetRemoteTagSha(ReleasedVersion)).IsNull(); + } + // The uploads get the fallback answer, which refuses them, so the release fails after the tag is pushed. [Test] public async Task Release_WhenAnUploadFails_DeletesTheDraft_AndTheTag() diff --git a/tests/Buildvana.Tool.Tests/ReleaseCommandGitLabTests.cs b/tests/Buildvana.Tool.Tests/ReleaseCommandGitLabTests.cs new file mode 100644 index 00000000..9cec0823 --- /dev/null +++ b/tests/Buildvana.Tool.Tests/ReleaseCommandGitLabTests.cs @@ -0,0 +1,145 @@ +// Copyright (C) Tenacom and Contributors. Licensed under the MIT license. +// See the LICENSE file in the project root for full license information. + +using System.Net; +using Buildvana.Core; +using Buildvana.Tool.Services.Hosts.GitLab; + +/// +/// End-to-end tests of the release command on a real GitLab host, whose API answers from a table +/// (see ). +/// +/// +/// The repository URL of the harness is not gitlab.com, so the host stands for a self-managed +/// instance, with its API under /api/v4. +/// +[NotInParallel] +internal sealed class ReleaseCommandGitLabTests +{ + private const string ReleasedVersion = "2.3.2-preview"; + private const string ApiUrl = "https://git.example.invalid/api/v4"; + private const string ApiProjectUrl = ApiUrl + "/projects/tenacom%2Ftest-repo"; + private const string ReleasesUrl = ApiProjectUrl + "/releases"; + private const string PackageUrl = ApiProjectUrl + "/packages/generic/test-repo/" + ReleasedVersion; + private const string MemberUrl = ApiProjectUrl + "/members/all/42"; + + [Test] + public async Task Release_ChecksTheRole_UploadsTheAssets_AndCreatesTheReleaseOnTheTag() + { + var api = Answering(accessLevel: 40); + using var harness = new ReleaseHarness(new() { GitLabApi = api }); + + var exitCode = await harness.RunAsync().ConfigureAwait(false); + + await Assert.That(exitCode).IsEqualTo(0); + var requests = api.Requests; + await Assert.That(requests[0].Url).IsEqualTo(ReleasesUrl + "?per_page=1"); + await Assert.That(requests[1].Url).IsEqualTo(ApiUrl + "/user"); + await Assert.That(requests[2].Url).IsEqualTo(MemberUrl); + + // One upload per asset, then the release that links them. + var uploads = requests.Where(static x => x.Method == HttpMethod.Put).ToArray(); + await Assert.That(uploads.Length).IsEqualTo(5); + await Assert.That(uploads.All(static x => x.Url.StartsWith(PackageUrl + "/", StringComparison.Ordinal))).IsTrue(); + var release = requests[^1]; + await Assert.That(release.Method).IsEqualTo(HttpMethod.Post); + await Assert.That(release.Url).IsEqualTo(ReleasesUrl); + await Assert.That(release.Body).Contains($"\"tag_name\":\"{ReleasedVersion}\""); + await Assert.That(release.Body).Contains("https://git.example.invalid/tenacom/test-repo/-/blob/release/CHANGELOG.md"); + await Assert.That(release.Body).Contains("\"direct_asset_path\":\"/setup.exe\""); + var releaseCommitSha = harness.Repo.GetCommits(2)[1].Sha; + await Assert.That(harness.Repo.GetRemoteTagSha(ReleasedVersion)).IsEqualTo(releaseCommitSha); + } + + // The check runs before the release commit, so a refusal leaves nothing to roll back. + [Test] + public async Task Release_BelowTheMaintainerRole_FailsBeforeTheReleaseCommit() + { + var api = Answering(accessLevel: 30); + using var harness = new ReleaseHarness(new() { GitLabApi = api }); + var remoteTip = harness.Repo.GetRemoteTipSha(); + + // ReSharper disable once AccessToDisposedClosure // False positive: the assertion invokes Act before the harness is disposed + Task Act() => harness.RunAsync(); + + var exception = await Assert.That(Act).Throws(); + await Assert.That(exception!.Message).Contains("below the Maintainer role"); + await Assert.That(api.Requests.Count).IsEqualTo(3); + await Assert.That(harness.Repo.GetRemoteTipSha()).IsEqualTo(remoteTip); + await Assert.That(harness.Repo.GetRemoteTagSha(ReleasedVersion)).IsNull(); + } + + // A job token has no role to check, and pushes as gitlab-ci-token. + [Test] + public async Task Release_WithAJobToken_ChecksNoRole_AndPushesAsGitLabCIToken() + { + var api = Answering(accessLevel: 40); + using var harness = new ReleaseHarness(new() { GitLabApi = api, HostTokenEnv = GitLabHost.JobTokenVariable }); + + var exitCode = await harness.RunAsync().ConfigureAwait(false); + + await Assert.That(exitCode).IsEqualTo(0); + await Assert.That(api.Requests.Select(static x => x.Url)).DoesNotContain(ApiUrl + "/user"); + await Assert.That(api.Requests.All(static x => x.Headers["JOB-TOKEN"] == "test-token")).IsTrue(); + await Assert.That(harness.Reporter.Messages.Select(static x => x.Message)) + .Contains("Fallback Git credentials provided by GitLab (protocol username: 'gitlab-ci-token')."); + } + + // The assets are checked before the push, so a refused asset reaches neither the remote nor the feed. + [Test] + public async Task Release_WithAnAssetGitLabRefuses_FailsBeforeThePush() + { + var api = Answering(accessLevel: 40); + using var harness = new ReleaseHarness(new() { GitLabApi = api, WithHook = true }); + var remoteTip = harness.Repo.GetRemoteTipSha(); + var artifactsPath = harness.ArtifactsPath; + + harness.HookBehavior = () => { + var path = Path.Combine(artifactsPath, "Foo 1.zip"); + File.WriteAllText(path, "fake zip"); + File.WriteAllText(Path.Combine(artifactsPath, "Bad.assets.txt"), $"{path}\tapplication/zip\tBad zip\n"); + }; + + // ReSharper disable once AccessToDisposedClosure // False positive: the assertion invokes Act before the harness is disposed + Task Act() => harness.RunAsync(); + + var exception = await Assert.That(Act).Throws(); + await Assert.That(exception!.Message).Contains("'Foo 1.zip' is not a valid file name in the generic package registry."); + await Assert.That(harness.Events.Select(static x => x.Name)).DoesNotContain("nuget-push"); + await Assert.That(api.Requests.Any(static x => x.Method == HttpMethod.Put)).IsFalse(); + await Assert.That(harness.Repo.GetRemoteTipSha()).IsEqualTo(remoteTip); + await Assert.That(harness.Repo.GetRemoteTagSha(ReleasedVersion)).IsNull(); + } + + // The uploads succeed and the release creation fails, so the release fails after the tag is pushed. + [Test] + public async Task Release_WhenTheReleaseCannotBeCreated_DeletesThePackage_AndTheTag() + { + var api = Answering(accessLevel: 40, release: new(HttpStatusCode.BadRequest, """{"message":"Validation failed"}""")); + using var harness = new ReleaseHarness(new() { GitLabApi = api }); + + // ReSharper disable once AccessToDisposedClosure // False positive: the assertion invokes Act before the harness is disposed + Task Act() => harness.RunAsync(); + + var exception = await Assert.That(Act).Throws(); + await Assert.That(exception!.Message).Contains("Validation failed"); + var delete = api.Requests[^1]; + await Assert.That(delete.Method).IsEqualTo(HttpMethod.Delete); + await Assert.That(delete.Url).IsEqualTo(ApiProjectUrl + "/packages/7"); + await Assert.That(harness.Notices).Contains($"Deleted generic package test-repo {ReleasedVersion}."); + await Assert.That(harness.Repo.GetRemoteTagSha(ReleasedVersion)).IsNull(); + } + + // The releases, the user, the membership, the release, and the deletion of the package have answers. Every + // other request, the uploads included, gets the fallback answer. + private static StubHttpMessageHandler Answering(int accessLevel, StubHttpAnswer? release = null) + => new( + new Dictionary<(HttpMethod Method, string Url), StubHttpAnswer> { + [(HttpMethod.Get, ReleasesUrl + "?per_page=1")] = new(HttpStatusCode.OK, "[]"), + [(HttpMethod.Get, ApiUrl + "/user")] = new(HttpStatusCode.OK, """{"id":42}"""), + [(HttpMethod.Get, MemberUrl)] = new(HttpStatusCode.OK, $$"""{"id":42,"access_level":{{accessLevel}}}"""), + [(HttpMethod.Post, ReleasesUrl)] = release ?? new(HttpStatusCode.Created, $$"""{"tag_name":"{{ReleasedVersion}}"}"""), + [(HttpMethod.Delete, ApiProjectUrl + "/packages/7")] = new(HttpStatusCode.NoContent), + }, + new StubHttpAnswer(HttpStatusCode.OK, """{"package_id":7}""")); +} diff --git a/tests/Buildvana.Tool.Tests/ReleaseHarness.cs b/tests/Buildvana.Tool.Tests/ReleaseHarness.cs index 12fb0350..7faecf11 100644 --- a/tests/Buildvana.Tool.Tests/ReleaseHarness.cs +++ b/tests/Buildvana.Tool.Tests/ReleaseHarness.cs @@ -17,6 +17,7 @@ using Buildvana.Tool.Services.Git; using Buildvana.Tool.Services.Hosts; using Buildvana.Tool.Services.Hosts.GitHub; +using Buildvana.Tool.Services.Hosts.GitLab; using Buildvana.Tool.Services.Platforms; using Buildvana.Tool.Services.Release; using Buildvana.Tool.Services.Versioning; @@ -54,7 +55,6 @@ internal sealed class ReleaseHarness : IDisposable public const string ApiFilesDirectory = "src/Test.Lib"; private const string ApiKeyEnvVar = "BV_TEST_NUGET_API_KEY"; - private const string TokenEnvVar = "BV_TEST_HOST_TOKEN"; private const string RemoteName = "origin"; private const string BranchName = "release"; private const string Configuration = "Release"; @@ -68,7 +68,7 @@ internal sealed class ReleaseHarness : IDisposable private readonly ServiceProvider _services; private readonly string _originalCurrentDirectory = Environment.CurrentDirectory; private readonly string? _originalApiKey = Environment.GetEnvironmentVariable(ApiKeyEnvVar); - private readonly string? _originalToken = Environment.GetEnvironmentVariable(TokenEnvVar); + private readonly string? _originalToken; private IReadOnlyList _commandOptions = []; @@ -87,7 +87,8 @@ public ReleaseHarness(ReleaseHarnessOptions? options = null) }; Environment.SetEnvironmentVariable(ApiKeyEnvVar, "test-api-key"); - Environment.SetEnvironmentVariable(TokenEnvVar, _options.PushSecret); + _originalToken = Environment.GetEnvironmentVariable(_options.HostTokenEnv); + Environment.SetEnvironmentVariable(_options.HostTokenEnv, _options.PushSecret); Repo = new TempGitRepo(); PopulateRepository(); Repo.CommitAll("Initial commit"); @@ -242,7 +243,7 @@ public void Dispose() // The current directory has to leave the repository before the repository can be deleted. Environment.CurrentDirectory = _originalCurrentDirectory; Environment.SetEnvironmentVariable(ApiKeyEnvVar, _originalApiKey); - Environment.SetEnvironmentVariable(TokenEnvVar, _originalToken); + Environment.SetEnvironmentVariable(_options.HostTokenEnv, _originalToken); Repo.Dispose(); } @@ -285,8 +286,13 @@ private GitHost CreateHost(IServiceProvider services) return new GitHubHost(services, ParseRepositoryUrl(services), gitHubApi); } + if (_options.GitLabApi is { } gitLabApi) + { + return new GitLabHost(services, ParseRepositoryUrl(services), gitLabApi); + } + return new RecordingHost { - TokenVariable = TokenEnvVar, + TokenVariable = _options.HostTokenEnv, PushUser = _options.PushUser, OnPublishing = () => { Record("publish"); @@ -460,7 +466,10 @@ private string BuildConfiguration() ["versioning"] = new JsonObject { ["prereleaseTag"] = "preview" }, ["dotnet"] = new JsonObject { ["configuration"] = Configuration }, ["nuget"] = new JsonObject { ["feeds"] = new JsonObject { ["release"] = feed } }, - ["hosts"] = new JsonObject { ["github"] = new JsonObject { ["tokenEnv"] = TokenEnvVar } }, + ["hosts"] = new JsonObject { + ["github"] = new JsonObject { ["tokenEnv"] = _options.HostTokenEnv }, + ["gitlab"] = new JsonObject { ["tokenEnv"] = _options.HostTokenEnv }, + }, }; if (_options.RepositoryUrl is { } repositoryUrl) { diff --git a/tests/Buildvana.Tool.Tests/ReleaseHarnessOptions.cs b/tests/Buildvana.Tool.Tests/ReleaseHarnessOptions.cs index c3b0b56c..de8462d6 100644 --- a/tests/Buildvana.Tool.Tests/ReleaseHarnessOptions.cs +++ b/tests/Buildvana.Tool.Tests/ReleaseHarnessOptions.cs @@ -107,6 +107,19 @@ internal sealed record ReleaseHarnessOptions /// public StubHttpMessageHandler? GitHubApi { get; init; } + /// + /// Gets the handler that answers the requests of a real , + /// or for a . Ignored when is + /// , or is not . + /// + public StubHttpMessageHandler? GitLabApi { get; init; } + + /// + /// Gets the name of the variable that holds the token of the host: the tokenEnv of every section + /// under hosts. The harness sets it to for the duration of the test. + /// + public string HostTokenEnv { get; init; } = "BV_TEST_HOST_TOKEN"; + /// /// Gets the value of release.allowUnknownHost. /// diff --git a/tests/Buildvana.Tool.Tests/ServerReleaseTests.cs b/tests/Buildvana.Tool.Tests/ServerReleaseTests.cs index 835c0541..92e518e9 100644 --- a/tests/Buildvana.Tool.Tests/ServerReleaseTests.cs +++ b/tests/Buildvana.Tool.Tests/ServerReleaseTests.cs @@ -145,6 +145,41 @@ public async Task PushUpdates_TagsTheReleaseCommit_AndPushesTheTag() } } + // The publisher checks the assets before the push, so an asset added after it would go unchecked. + [Test] + public async Task AddAsset_AfterPushingUpdates_Throws() + { + using var harness = new ReleaseHarness(); + var release = await StartReleaseAsync(harness).ConfigureAwait(false); + await using (release.ConfigureAwait(false)) + { + release.EnsureReleaseCommit(); + release.PushUpdates(); + + void Act() => release.AddAsset("artifacts/late.bin"); + + var exception = await Assert.That(Act).Throws(); + await Assert.That(exception!.Message).Contains("already been pushed"); + } + } + + [Test] + public async Task PushUpdates_HasThePublisherCheckTheAssets() + { + using var harness = new ReleaseHarness(); + var release = await StartReleaseAsync(harness).ConfigureAwait(false); + await using (release.ConfigureAwait(false)) + { + release.EnsureReleaseCommit(); + release.AddAsset("artifacts/setup.exe", "Setup program", "application/vnd.microsoft.portable-executable"); + + release.PushUpdates(); + + ReleaseAsset[] expected = [new("artifacts/setup.exe", "Setup program", "application/vnd.microsoft.portable-executable")]; + await Assert.That(harness.Publisher!.CheckedAssets).IsEquivalentTo(expected); + } + } + [Test] public async Task PublishAsync_BeforePushingUpdates_Throws() { @@ -195,12 +230,12 @@ public async Task PublishAsync_RecordsTheNumberOfAssets(int assetCount, string e await using (release.ConfigureAwait(false)) { release.EnsureReleaseCommit(); - release.PushUpdates(); for (var i = 0; i < assetCount; i++) { release.AddAsset(string.Create(CultureInfo.InvariantCulture, $"artifacts/asset-{i}.bin")); } + release.PushUpdates(); await release.PublishAsync().ConfigureAwait(false); }