[ishanaggarwal04]

rook-assurance.yml and rook-prepare-suite.yml, plus the rook-setup composite action they share, copied as-is from the incident-scribe agent.

[vipulkv]

Continuous Review — Round 1 — ❌ FAIL

Head: b7607e0 Reviewer panel: 2 CI: pass=4
HIGH/CRITICAL found by: glm(1H)
Outside contributor (fork ishanaggarwal04/rook-public, author ishanaggarwal04 is CONTRIBUTOR): reviewed by the diff-only panel (claude, glm) — this PR's code was not checked out.

Per-reviewer summary

Reviewer	Verdict	CRIT	HIGH	MED	LOW	Duration
claude	ABSTAIN_ERROR	0	0	0	0	1s
glm	FAIL	1	0	1	3	530s

Findings

glm — 2 finding(s)

[CRITICAL] PR runs never export the selected scenario IDs, so the release gate silently no-ops on every pull request

• File: samples/incident-scribe/.github/workflows/rook-assurance.yml:216
• Scenario: The select step writes ids to $GITHUB_OUTPUT only inside the workflow_dispatch/schedule else-branch (line 245). In the pull_request branch (lines 215-216) the IDs from ci/select-scenarios.mjs are captured into the local ids variable and used solely by scenario_table "$ids", which ticks the chosen scenarios as 'In this run' in the step summary. Job 2 is gated on needs.select.outputs.ids != '' and jobs 3/4 on the same output, so every PR into main touching incident_scribe/, services/, ci/ or .testmuai/rook/** skips jobs 2-4, the workflow concludes success (skipped jobs satisfy the check), and the summary simultaneously claims the selected scenarios were covered. An agent regression ships green because the gate never ran a single scenario; the only hint is three greyed-out jobs.

[MED] Gate and prepare workflows pin two different Rook CLI versions (0.1.5 vs 0.1.11)

• File: samples/incident-scribe/.github/workflows/rook-assurance.yml:364
• Scenario: The shared rook-setup composite action installs inputs.rook_version, default '0.1.11' (action.yml line 22), and every rook-prepare-suite.yml stage uses it. The assurance run job instead re-implements the install inline and hardcodes --version 0.1.5. A suite generated and rehearsed by 0.1.11 is then gated by 0.1.5, and the gate's parsing of rook-results/run.json (.run_id, .halted) and report.json (.report.totals) depends on the JSON shape of whichever version is pinned in that one step; any 0.1.5-to-0.1.11 schema or behavior difference makes the gate misreport or crash while the prepare flow looked healthy. The inline copy bypassing the composite is exactly how the drift arose.

---

Auto-posted by continuous-review-agent v1.3 (findings now include a suggested fix / code snippet). Contact @vipulkv for feedback.

[vipulkv]

Continuous Review — Round 2 — ⚠️ INCOMPLETE

Head: 0141ab7 Reviewer panel: 2 CI: pass=4
only 1/2 reviewers responded (need ≥2); abstained: claude=ABSTAIN_ERROR
Outside contributor (fork ishanaggarwal04/rook-public, author ishanaggarwal04 is CONTRIBUTOR): reviewed by the diff-only panel (claude, glm) — this PR's code was not checked out.

Per-reviewer summary

Reviewer	Verdict	CRIT	HIGH	MED	LOW	Duration
claude	ABSTAIN_ERROR	0	0	0	0	1s
glm	PASS	0	0	1	0	571s

Findings

glm — 1 finding(s)

[MED] The 'pinned' Rook installer is fetched from the mutable main branch and executed after secrets are already in the job environment

• File: samples/incident-scribe/.github/actions/rook-install/action.yml:20
• Scenario: rook-setup runs its 'Resolve secrets' step first, exporting LT_USERNAME, LT_ACCESS_KEY, LLM_API_KEY and SCRIBE_API_TOKEN to GITHUB_ENV, and rook-assurance's run job does the same before invoking this action. The action then curls https://raw.githubusercontent.com/LambdaTest/rook/main/install.sh and runs it with bash. Any change merged to the public rook repo's main branch (accidental breakage or compromise) immediately changes the code executing in every consumer's CI with those credentials in its environment; the 0.1.11 pin constrains only the --version argument, not the script that actually runs.

---

Auto-posted by continuous-review-agent v1.3 (findings now include a suggested fix / code snippet). Contact @vipulkv for feedback.