Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions DOCUMENTATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -1145,6 +1145,8 @@ testsprite ci init github --set-secret --repo owner/name
- **A — your platform deploys every PR** (for example the Vercel or Netlify git integration): change the trigger to `on: deployment_status`, delete the placeholder step, and keep the rest. A repository that deploys **several** projects gets one deployment per project for the same commit (Vercel names them `Preview – <project>`); pin the one to gate with `&& github.event.deployment.environment == 'Preview – <project>'` in the `if:`, or the gate tests whichever project's URL arrives. The run step reads the URL from `github.event.deployment_status.environment_url`, and `ci run` finds the pull request behind the deployment's commit with `GITHUB_TOKEN`; a deployment that belongs to no open pull request is skipped with a notice. The job only runs for successful, non-production deployments (it checks both `production_environment` and the environment name, because Vercel reports `production_environment: false` on its production deployments). This shape needs the repository to actually receive deployment events from your platform; confirm it with one real pull request.
- **B — you deploy in the workflow**: keep `on: pull_request`, replace the placeholder's commands with your deploy, and end it with `echo "url=<preview address>" >> "$GITHUB_OUTPUT"`. Your platform token goes in its own repo secret.

Concurrency groups include the workflow filename from `--path` so separate generated workflows do not cancel each other. The filename keeps its extension and URI-escapes special characters rather than discarding them; because GitHub compares group names case-insensitively, uppercase letters are escaped too, so `TestSprite.yml` and `testsprite.yml` stay separate. Regenerating the same file keeps its group; the PR number or deployment SHA still identifies runs within that workflow. Both examples below use `.github/workflows/testsprite.yml`.

The two finished shapes, with the template's comments left out (the generated file carries them):

```yaml
Expand All @@ -1158,7 +1160,7 @@ permissions:
jobs:
gate:
concurrency:
group: testsprite-${{ github.event.pull_request.number || github.event.deployment.sha }}
group: testsprite-testsprite.yml-${{ github.event.pull_request.number || github.event.deployment.sha }}
cancel-in-progress: true
if: >-
github.event_name == 'deployment_status' &&
Expand Down Expand Up @@ -1207,7 +1209,7 @@ permissions:
jobs:
gate:
concurrency:
group: testsprite-${{ github.event.pull_request.number || github.event.deployment.sha }}
group: testsprite-testsprite.yml-${{ github.event.pull_request.number || github.event.deployment.sha }}
cancel-in-progress: true
if: >-
github.event_name == 'pull_request' &&
Expand Down
8 changes: 6 additions & 2 deletions src/commands/__snapshots__/ci.spec.ts.snap
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ permissions:

jobs:
testsprite-tests:
# Include this workflow's filename: concurrency groups are repository-wide,
# so another project's workflow must not cancel this gate.
# B (pull_request): a new push to the pull request cancels the previous job.
# A (deployment_status): one group per COMMIT — Vercel sets deployment.ref to
# the commit sha, so a new push does not cancel the previous commit's job.
Expand All @@ -27,7 +29,7 @@ jobs:
# Cancelling only stops this job's CLI: the TestSprite run it started keeps
# executing and reports in the Portal; the new push starts its own run.
concurrency:
group: testsprite-\${{ github.event.pull_request.number || github.event.deployment.sha }}
group: testsprite-testsprite.yml-\${{ github.event.pull_request.number || github.event.deployment.sha }}
cancel-in-progress: true
# pull_request: skip fork PRs (they get no repository secrets) and draft PRs.
# deployment_status: only successful, non-production deployments. Vercel leaves
Expand Down Expand Up @@ -117,6 +119,8 @@ permissions:

jobs:
testsprite-tests:
# Include this workflow's filename: concurrency groups are repository-wide,
# so another project's workflow must not cancel this gate.
# B (pull_request): a new push to the pull request cancels the previous job.
# A (deployment_status): one group per COMMIT — Vercel sets deployment.ref to
# the commit sha, so a new push does not cancel the previous commit's job.
Expand All @@ -125,7 +129,7 @@ jobs:
# Cancelling only stops this job's CLI: the TestSprite run it started keeps
# executing and reports in the Portal; the new push starts its own run.
concurrency:
group: testsprite-\${{ github.event.pull_request.number || github.event.deployment.sha }}
group: testsprite-testsprite.yml-\${{ github.event.pull_request.number || github.event.deployment.sha }}
cancel-in-progress: true
# pull_request: skip fork PRs (they get no repository secrets) and draft PRs.
# deployment_status: only successful, non-production deployments. Vercel leaves
Expand Down
78 changes: 77 additions & 1 deletion src/commands/ci.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -186,6 +186,82 @@ const WF_BASE = {
};

describe('buildGithubWorkflow', () => {
const concurrency = (workflow: string) => {
const match = workflow.match(/ {4}concurrency:\n {6}group: (.+)\n {6}cancel-in-progress: (.+)/);
if (!match?.[1] || !match[2]) throw new Error('Missing workflow concurrency configuration');
return { group: match[1], cancelInProgress: match[2] };
};

it('isolates different workflow paths even for the same project and environment', () => {
const web = concurrency(
buildGithubWorkflow({ ...WF_BASE, workflowPath: '.github/workflows/testsprite-web.yml' }),
);
const api = concurrency(
buildGithubWorkflow({ ...WF_BASE, workflowPath: '.github/workflows/testsprite-api.yml' }),
);
expect(web.group).not.toBe(api.group);
});

it('keeps the same workflow group when regenerated with different project settings', () => {
const first = concurrency(buildGithubWorkflow(WF_BASE));
const regenerated = concurrency(buildGithubWorkflow({ ...WF_BASE }));
const reconfigured = concurrency(
buildGithubWorkflow({ ...WF_BASE, projectId: 'proj-2', environment: 'qa' }),
);
expect(first.group).toContain('testsprite.yml');
expect(regenerated).toEqual(first);
expect(reconfigured).toEqual(first);
});

it.each([
['.github/workflows/testsprite.yml', '.github/workflows/testsprite.yaml'],
['.github/workflows/testsprite.api.yml', '.github/workflows/testsprite-api.yml'],
['.github/workflows/testsprite api.yml', '.github/workflows/testsprite%20api.yml'],
['.github/workflows/TestSprite.yml', '.github/workflows/testsprite.yml'],
['.github/workflows/A.yml', '.github/workflows/%41.yml'],
['.github/workflows/test:sprite.yml', '.github/workflows/test%3Asprite.yml'],
])('keeps distinct filenames independent: %s versus %s', (first, second) => {
// GitHub compares concurrency group names case-insensitively.
const groupOf = (workflowPath: string) =>
concurrency(buildGithubWorkflow({ ...WF_BASE, workflowPath })).group.toLowerCase();
expect(groupOf(first)).not.toBe(groupOf(second));
});

it('escapes uppercase letters without altering existing escapes', () => {
const group = concurrency(
buildGithubWorkflow({ ...WF_BASE, workflowPath: '.github/workflows/Test Sprite:CI.yml' }),
).group;
expect(group).toContain('testsprite-%54est%20%53prite%3A%43%49.yml-');
});

it.each(['pull_request', 'deployment_status'] as const)(
'retains cancellation within one workflow for %s',
event => {
const generated = buildGithubWorkflow(WF_BASE);
// The documented deployment shape changes only the trigger and URL step.
const workflow =
event === 'deployment_status'
? generated.replace(
' pull_request:\n types: [opened, synchronize, reopened, ready_for_review]',
' deployment_status:',
)
: generated;
expect(workflow).toContain(`\n ${event}:`);
const { group, cancelInProgress } = concurrency(workflow);
expect(group).toContain('testsprite.yml-');
expect(group).toContain(
'${{ github.event.pull_request.number || github.event.deployment.sha }}',
);
expect(cancelInProgress).toBe('true');
const eventKey = '${{ github.event.pull_request.number || github.event.deployment.sha }}';
const key = event === 'pull_request' ? '42' : 'a'.repeat(40);
expect(group.replace(eventKey, key)).toBe(
concurrency(buildGithubWorkflow({ ...WF_BASE })).group.replace(eventKey, key),
);
expect(group.replace(eventKey, key)).not.toBe(group.replace(eventKey, 'another-event'));
},
);

it('matches the reviewed template (default flags)', () => {
expect(buildGithubWorkflow(WF_BASE)).toMatchSnapshot();
});
Expand Down Expand Up @@ -234,7 +310,7 @@ describe('buildGithubWorkflow', () => {
// Vercel writes production_environment: false on production deployments.
expect(yaml).toContain("github.event.deployment.environment != 'production'");
expect(yaml).toContain(
'group: testsprite-${{ github.event.pull_request.number || github.event.deployment.sha }}',
'group: testsprite-testsprite.yml-${{ github.event.pull_request.number || github.event.deployment.sha }}',
);
});

Expand Down
14 changes: 13 additions & 1 deletion src/commands/ci.ts
Original file line number Diff line number Diff line change
Expand Up @@ -278,6 +278,16 @@ export function buildGithubWorkflow(input: {
assertSafeWorkflowValue('project', input.projectId);
assertSafeWorkflowValue('env', input.environment);
if (input.endpointUrl) assertSafeWorkflowValue('endpoint-url', input.endpointUrl);
// All generated workflows live directly under .github/workflows/. Keep the
// readable filename, including its extension; escape rather than discard
// punctuation so distinct filenames cannot collapse to the same slug.
// GitHub compares concurrency group names case-insensitively, so uppercase
// letters are escaped too (keeping existing %XX escapes intact): otherwise
// TestSprite.yml and testsprite.yml would share a group.
const workflowSlug = encodeURIComponent(path.posix.basename(input.workflowPath)).replace(
/%[0-9A-F]{2}|[A-Z]/g,
match => (match.startsWith('%') ? match : `%${match.charCodeAt(0).toString(16).toUpperCase()}`),
);

const envLines = [
` ${API_KEY_SECRET_NAME}: \${{ secrets.${API_KEY_SECRET_NAME} }}`,
Expand Down Expand Up @@ -320,6 +330,8 @@ permissions:

jobs:
testsprite-tests:
# Include this workflow's filename: concurrency groups are repository-wide,
# so another project's workflow must not cancel this gate.
# B (pull_request): a new push to the pull request cancels the previous job.
# A (deployment_status): one group per COMMIT — Vercel sets deployment.ref to
# the commit sha, so a new push does not cancel the previous commit's job.
Expand All @@ -328,7 +340,7 @@ jobs:
# Cancelling only stops this job's CLI: the TestSprite run it started keeps
# executing and reports in the Portal; the new push starts its own run.
concurrency:
group: testsprite-\${{ github.event.pull_request.number || github.event.deployment.sha }}
group: testsprite-${workflowSlug}-\${{ github.event.pull_request.number || github.event.deployment.sha }}
cancel-in-progress: true
# pull_request: skip fork PRs (they get no repository secrets) and draft PRs.
# deployment_status: only successful, non-production deployments. Vercel leaves
Expand Down
Loading