Skip to content

fix(ci): gate a draft pull request on deployment_status instead of skipping it green - #704

Open
neromtoobad wants to merge 1 commit into
TestSprite:mainfrom
neromtoobad:fix/ci-deployment-status-draft
Open

neromtoobad wants to merge 1 commit into
TestSprite:mainfrom
neromtoobad:fix/ci-deployment-status-draft

Conversation

@neromtoobad

@neromtoobad neromtoobad commented Oct 29, 2026 •

Copy link
Copy Markdown

What does this PR do?

ci run skipped every draft PR with exit 0 and a notice saying the gate runs once the PR is ready for review. That only holds for the pull_request trigger, whose types include ready_for_review. Under the deployment_status shape (Vercel and other platforms that deploy every PR), marking a draft ready triggers no new deployment. So the skip left a green testsprite-tests check on the head commit that no later event replaced, and the PR could be merged with no tests run.

Drafts are now skipped only on pull_request / pull_request_target. On deployment_status, the draft's preview, which already exists, is gated like any other PR. The DOCUMENTATION.md sentence that promised a re-run for both shapes now says this.

Related issue

Closes #701

Season 4 CLI Improvement Bonus

  • I'm entering this PR for the Season 4 CLI Improvement Bonus

Discord username: neromtoobad

Type of change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that changes existing behavior)
  • Documentation only
  • Build / CI / chore

Checklist

  • PR targets the main branch.
  • Commits follow Conventional Commits
    (feat(...), fix(...), docs(...), …).
  • npm run lint and npm run format:check pass.
  • npm run typecheck passes.
  • npm test passes and coverage stays at or above the 80% gate.
  • New behavior is covered by unit tests (mock-based; no network or
    credentials required).
  • No secrets, API keys, internal endpoints, or personal data are included.
  • User-facing changes are reflected in README.md / DOCUMENTATION.md where
    relevant.

Notes for reviewers

  • src/commands/ci.ts: the draft skip checks GITHUB_EVENT_NAME. The GithubRunContext type is unchanged.
  • New unit test in ci-run.spec.ts. It fails on main, where the draft is skipped and nothing is dispatched, and passes with the fix: the draft is dispatched with previewName: github-pr-7. The existing pull_request draft-skip test still passes.
  • Local results: ci-run.spec.ts + ci.spec.ts 159/159, full suite green, tsc / Prettier clean.
  • Left out on purpose (mentioned in [Hackathon] fix(ci): a draft PR on the deployment_status trigger gets a green check that no later event replaces #701): the no-PR-yet deployment_status skip has the same lasting-green shape. Happy to follow up on whichever behaviour you prefer.

Summary by CodeRabbit

  • Bug Fixes
    • CI runs triggered by deployment status events now proceed for draft pull requests instead of skipping them. Draft pull requests triggered by pull request events continue to be skipped until they are ready for review.
  • Documentation
    • Clarified how draft pull requests are handled by the two CI workflow event types.

…ipping it green

ci run skipped every draft with exit 0, saying the gate runs once the PR is
ready for review. That is only true for the pull_request trigger, whose
types include ready_for_review. Under the deployment_status shape (Vercel and
other deploy-every-PR platforms) marking a draft ready triggers no new
deployment, so the skip left a passing check on the head commit that no
later event replaced, and the PR could merge with no tests run. Drafts are
now skipped only on pull_request/pull_request_target; on deployment_status
the draft's preview, which already exists, is gated.
@coderabbitai

coderabbitai Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between 5281202 and 12f517b.

Files selected for processing: 3
  • DOCUMENTATION.md
  • src/commands/ci-run.spec.ts
  • src/commands/ci.ts

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.

Walkthrough

Draft pull requests from deployment_status events now proceed through the CI run instead of being skipped. Drafts from pull_request and pull_request_target events are still skipped. The documentation and regression test reflect this distinction.

Changes

Draft pull request gating

Layer / File(s) Summary
Event-specific draft handling
src/commands/ci.ts, src/commands/ci-run.spec.ts, DOCUMENTATION.md
The CI run skips drafts only for pull_request and pull_request_target events. A regression test checks dispatch for a draft found through a deployment_status event. The documentation describes the different draft handling for the two workflow shapes.

Priority: Normal

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix · Severity of issue fixed: Medium

Possibly related issues

  • TestSprite/testsprite-cli#650 — The issue describes the deployment_status draft behavior addressed by this change.
  • TestSprite/testsprite-cli#671 — The change retains the pull_request draft skip path and does not address the reported missing JSON output.

Suggested labels: s4-cli-bonus, needs-issue

Suggested reviewers: zeshi-du

Merge Risk: Minimal · up to 12f51

The changes correctly gate drafts by event type and are ready to merge.

Pre-merge checks: Passed 5
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title accurately describes the main change: deployment_status now gates draft pull requests instead of skipping them. The wording is slightly awkward but remains specific and understandable.
Linked Issues check Passed Issue #701 requires event-specific draft handling, documentation, and a regression test. The supplied PR evidence shows that src/commands/ci.ts skips drafts only for pull_request and `pull_request…
Out of Scope Changes check Passed The supplied change summary contains only the issue-related implementation, its regression test, and the related documentation update. The changes do not include the explicitly deferred no-PR-yet beha…
Docstring Coverage Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentat…
Full details: Linked Issues check

Explanation

Issue #701 requires event-specific draft handling, documentation, and a regression test. The supplied PR evidence shows that src/commands/ci.ts skips drafts only for pull_request and pull_request_target, while deployment_status dispatches the existing preview. src/commands/ci-run.spec.ts covers the draft deployment-status case with github-pr-7, and DOCUMENTATION.md describes the two behaviors. The reported test and type-check results also pass. The explicitly deferred no-PR-yet case is not required by #701.

Full details: Out of Scope Changes check

Explanation

The supplied change summary contains only the issue-related implementation, its regression test, and the related documentation update. The changes do not include the explicitly deferred no-PR-yet behavior or another unrelated feature.

Full details: Docstring Coverage

Explanation

Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 1 functions across 2 files. (1 skipped: 1 unsupported.)

Finishing Touches: 💡 1
🛠️ Fix failing CI checks: 💡
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown

Thanks for the PR, @neromtoobad! It links an issue, but that issue isn't assigned to you yet: #701 (unassigned). Per our workflow, claim the issue first by commenting /assign on it (the triage bot assigns you automatically). If it's already assigned to someone else, please coordinate with them or pick another issue — unclaimed-issue PRs are not reviewed. After fixing it, edit the PR description or push a commit to re-run this check. See CONTRIBUTING → Contribution model.

@github-actions

Copy link
Copy Markdown

🏆 Season 4 CLI Improvement Bonus: entry recorded. Thanks, @neromtoobad!

To be counted, this PR needs the following by Oct 29, 11:59 PM PT:

  • ⬜ A linked issue assigned to you: the issue this PR links isn't assigned to you yet. Claim it by commenting /assign on it.
  • ✅ Out of draft.
  • ✅ A Discord username in the description.

Rewards go to merged PRs judged a genuine improvement, and up to three bonus-eligible PRs count per person. Untick the box to withdraw the entry. This comment refreshes when you edit the PR. Full rules.

@github-actions github-actions Bot added needs-issue PR not linked to an issue yet — please open one first and claim it (see CONTRIBUTING) s4-cli-bonus Entered for the Season 4 CLI Improvement Bonus labels Oct 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-issue PR not linked to an issue yet — please open one first and claim it (see CONTRIBUTING) s4-cli-bonus Entered for the Season 4 CLI Improvement Bonus

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Hackathon] fix(ci): a draft PR on the deployment_status trigger gets a green check that no later event replaces

1 participant