Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion DOCUMENTATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -1254,7 +1254,7 @@ jobs:

The last step runs `testsprite ci run --project <id> --env <name> --target-url <preview>`. In a pull request, the run uses a temporary copy of that environment pointed at the preview (named `github-pr-<N>`, like GitHub App runs, and released when the run finishes), so the Portal shows which pull request it tested.

What the generated workflow fixes in place: it installs `@testsprite/testsprite-cli@latest` (results are decided by the backend, which is not versioned with the CLI, so a pinned client would only freeze old behaviour); it requests `contents: read` and `pull-requests: read`; it skips fork pull requests (they run without repository secrets) and draft pull requests (it runs again on `ready_for_review`); it passes `--fail-on-skipped`, so a test that never ran fails the gate instead of warning; under shape B a new push to the pull request cancels the previous job; under shape A the concurrency group is **per commit** (Vercel sets the deployment's `ref` to the commit SHA, so there is no branch to key on) and a new push does not cancel the previous commit's job — either way, cancelling only stops that job's CLI; the TestSprite run it started keeps executing and reports in the Portal, and the new push starts its own run. The group is declared on the job, so deployment events the `if:` drops (`pending`, `inactive`, another project's deployment of the same commit) never cancel a running gate; and it uploads `ci run`'s summary file even when the job fails.
What the generated workflow fixes in place: it installs `@testsprite/testsprite-cli@latest` (results are decided by the backend, which is not versioned with the CLI, so a pinned client would only freeze old behaviour); it requests `contents: read` and `pull-requests: read`; it skips fork pull requests (they run without repository secrets) and, under shape B, draft pull requests (it runs again on `ready_for_review`; under shape A no event follows `ready_for_review`, so a draft's preview is gated rather than skipped); it passes `--fail-on-skipped`, so a test that never ran fails the gate instead of warning; under shape B a new push to the pull request cancels the previous job; under shape A the concurrency group is **per commit** (Vercel sets the deployment's `ref` to the commit SHA, so there is no branch to key on) and a new push does not cancel the previous commit's job — either way, cancelling only stops that job's CLI; the TestSprite run it started keeps executing and reports in the Portal, and the new push starts its own run. The group is declared on the job, so deployment events the `if:` drops (`pending`, `inactive`, another project's deployment of the same commit) never cancel a running gate; and it uploads `ci run`'s summary file even when the job fails.

> **Forks and shape A.** A `deployment_status` event runs the workflow file **at the deployed commit**, with the repository's secrets, and the event does not say whether that commit came from a fork. The `if:` can skip fork pull requests on `pull_request` events, not on deployments. If your hosting platform deploys pull requests from forks, turn that off on the platform (Vercel: fork protection / require authorization for fork deployments), or a fork's edit of the workflow runs with your `TESTSPRITE_API_KEY`.

Expand Down
16 changes: 16 additions & 0 deletions src/commands/ci-run.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -753,6 +753,22 @@ describe('ci run — pull request context and single-execution admission', () =>
expect(existsSync(h.deps.env.GITHUB_STEP_SUMMARY)).toBe(false);
});

it('deployment_status: a draft pull request is gated, not skipped (no ready_for_review re-run follows)', async () => {
const h = harness(undefined, 'passed', {
pulls: [{ number: 7, state: 'open', draft: true, head: { sha: SHA, ref: 'feat/y' } }],
});
await runCiRun(
{ ...h.opts, environment: 'staging', targetUrl: PREVIEW, skipPreflight: true },
githubEnv(h, 'deployment_status', { deployment: { sha: SHA } }, { GITHUB_TOKEN: 'ghs_x' }),
);
expect(h.dispatches()).toHaveLength(1);
expect(h.dispatches()[0]!.body).toMatchObject({
previewName: 'github-pr-7',
githubTrigger: { prNumber: 7, commitSha: SHA },
});
expect(h.stderr.some(s => s.includes('Skipping draft'))).toBe(false);
});

it('deployment_status: finds the PR by commit with GITHUB_TOKEN; none → notice, nothing to gate', async () => {
const found = harness(undefined, 'passed', {
pulls: [{ number: 7, state: 'open', draft: false, head: { sha: SHA, ref: 'feat/y' } }],
Expand Down
7 changes: 6 additions & 1 deletion src/commands/ci.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1331,7 +1331,12 @@ export async function runCiRun(opts: CiRunOptions, deps: TestDeps = {}): Promise
let previewName: string | undefined;
let githubTrigger: CiGithubTrigger | undefined;
if (github.kind === 'pull_request') {
if (github.draft) {
// Only a pull_request event comes back for this PR on `ready_for_review`. A
// deployment_status event is not repeated when a draft is marked ready, so
// skipping it here would leave a green check on the head commit that no
// later event replaces. Its preview already exists: gate it.
const event = env.GITHUB_EVENT_NAME;
if (github.draft && (event === 'pull_request' || event === 'pull_request_target')) {
recordTelemetryExtras({ ciRunOutcome: 'skipped_draft' });
skipCiRun(
`Skipping draft pull request #${github.trigger.prNumber}; the gate runs once it is ready for review.`,
Expand Down
Loading