test(plugin-dev): add e2e integration tests for plugin lifecycle and dev workflow

[gashcrumb]

Summary

Adds end-to-end integration tests for rhdh-cli plugin dev and the full plugin developer on-ramp workflow (RHIDP-16674).

plugin dev E2E Test Suite (e2e-tests/plugin-dev.test.ts)

• Pre-flight & Validation Errors:

  • Rejection when --rhdh-local-dir is missing / unset
  • Rejection when runtime directory does not exist or lacks required files (compose.yaml, compose-dynamic-plugins-root.yaml, scripts)
  • Rejection when dynamic-plugins.override.yaml is missing
  • Rejection when dynamic-plugins.override.yaml is unconfigured and --configure is omitted
  • Fast-fail error on backend plugins when dist-types/ is missing
  • Rejection when an invalid --container-tool is provided
  • Fast-fail error on update and restart when runtime is not running

• Configuration & Staging:

  • Automated inclusion of configs/dynamic-plugins/rhdh-cli.generated.local.yaml into dynamic-plugins.override.yaml via --configure
  • Generation of rhdh-cli.generated.local.yaml with correct plugin package path (./local-plugins/<plugin-name>), disabled: false, and pullPolicy: Always
  • Plugin filesystem staging into local-plugins/<plugin-name>

• Real Container Engine Lifecycle:

  • Executes against a fast, lightweight Compose fixture stack (nginx:alpine HTTP 200 on :7007 + alpine:latest for the dynamic plugin installer)
  • Automatically discovers compose-capable engines (podman, docker)
  • Exercises real start --configure (up, installer container die/died events, HTTP readiness polling), status (JSON status parsing), update (re-export, re-stage, service restart), and stop --clean (compose down)
  • Supports testing against an existing RHDH Local clone when RHDH_LOCAL_DIR (or E2E_RHDH_LOCAL_DIR) is supplied

Developer On-ramp Timing & Acceptance Verification (e2e-tests/plugin-new.test.ts)

• Added structured timing instrumentation across scaffold, install, tsc, build, test, and export phases
• Asserts that each plugin flow (frontend-plugin, backend-plugin, catalog-processor-module) completes within the 5-minute requirement (< 300s)

CI Integration (.github/workflows/pr.yaml)

• Added container compose setup step in pr.yaml to wire docker-compose symlink and podman-compose so both engines can be validated in CI

Closes #RHIDP-16674

[gashcrumb]

/fs-review

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 7:09 PM UTC · Completed 7:31 PM UTC

Commit: 76c668e · View workflow run →

Runtime: claude · Model: sonnet → claude-sonnet-4-6 · Effort: high · Cost: $5.04

[fullsend-ai-review]

Risk Assessment: moderate (2/5)

Details

Test-heavy PR with one protected-path change (AGENTS.md) and moderate historical churn; 62/38 weighted composite of Tier1=2.0 and Tier2=2.67 yields 2.25, rounding to 2 (moderate).

Previous run

Risk Assessment: low (1/5)

Details

Test-only PR adding e2e integration tests with no production code, CI workflow, or dependency changes; composite score 0.62x1.625 + 0.38x1.17 = 1.45 -> 1 (low), driven by blast radius from 632 line additions offset by a 0.67 test file ratio, single known non-first-time author, and no historical churn or regression patterns on the affected paths.

[fullsend-ai-review]

Review

Findings

Medium

• [protected-path] AGENTS.md — AGENTS.md is a protected governance file. The PR body and RHIDP-16674 citation explain the rationale (documenting the E2E testing contract), providing sufficient context. Human approval is still required for protected-path changes regardless of context. Protected file modified: AGENTS.md.

Low

• [edge-case] e2e-tests/plugin-dev.test.ts:254 — describe.each(availableTools) produces zero test blocks when availableTools is empty, causing the entire lifecycle suite to be silently omitted from Jest output with no skip annotation. A comment at lines 250–253 documents the design intent, but Jest's test summary shows no indication that these tests were skipped rather than absent.
  Remediation: Add an explicit it.skip('lifecycle tests require a compose-capable container engine', ...) guard inside the 'real container lifecycle' describe block that activates when availableTools.length === 0.

• [scope-authorization-execution-gap] e2e-tests/plugin-dev.test.ts:252 — The comment acknowledges that CI runner provisioning is "tracked in follow-up CI configuration" but cites no concrete JIRA or GitHub issue number, making the follow-up work untrackable by future readers of the codebase.
  Remediation: Add an issue reference to the comment (e.g. // tracked in follow-up: RHIDP-XXXXX).

• [intent-coherence] AGENTS.md — The PR body's CI Integration section asserts that .github/workflows/pr.yaml was modified to wire docker-compose and podman-compose support, but that file is not present in the diff. Reviewers reading the PR body may believe lifecycle-test CI coverage was delivered when it was not.
  Remediation: Correct the PR body to note that .github/workflows/pr.yaml was not modified in this PR and that CI runner provisioning is deferred to a follow-up.

---

Next steps:

• /fs-fix — agent addresses review findings automatically
• /fs-fix <your instruction> — agent fixes with your specific guidance
• Push commits directly — review re-runs automatically on push
• /fs-fix-stop — disable automatic fix runs for this PR

Previous run

Review

Findings

Medium

• [intent-coherence] e2e-tests/plugin-new.test.ts:8 — The timing SLO assertion expect(totalMs).toBeLessThan(MAX_ONRAMP_DURATION_MS) uses MAX_ONRAMP_DURATION_MS = 5 minutes, the same value as TEST_TIMEOUT. When a flow takes ≥ 5 minutes, Jest's per-test timeout fires first and the assertion is never reached — making SLO violations unreportable as assertion failures.
  Remediation: Either lower MAX_ONRAMP_DURATION_MS (e.g. 4 minutes) so the assertion can fire before Jest's timeout, or raise TEST_TIMEOUT (e.g. 8 minutes) to give the assertion headroom.

• [scope-authorization-execution-gap] e2e-tests/plugin-dev.test.ts:218 — The container lifecycle suite (describe.each(availableTools)) only runs when a compose-capable engine is detected. The CI workflow changes that would provision such an engine were dropped from this PR. The lifecycle tests (start, status, update, stop) — the primary value of this PR — are silently skipped in CI until a follow-up adds the runner configuration.
  Remediation: Link a follow-up issue tracking the missing CI runner setup, or annotate the describe.each block with a comment explaining the dependency and expected follow-up PR.

Low

• [logic-error] e2e-tests/plugin-dev.test.ts:27 — runExpectingFailure catches its own "unexpected success" throw: the throw new Error('Command expected to fail...') at line 27 sits inside the try block and is immediately caught by the catch at line 29, converting it to a return value. The test still fails via a downstream pattern-match assertion, but the diagnostic is less clear than intended.
  Remediation: Move the success-path throw outside the try-catch (track success with a flag variable, then throw after the catch block).

• [edge-case] e2e-tests/plugin-dev.test.ts:185 — The pre-flight tests for update (line 185) and restart (line 200) fall back to 'podman' via availableTools[0] || 'podman' when no container tools are found. With no podman installed, the CLI fails with "Unable to find podman on PATH" rather than the expected "RHDH Local is not running" message, causing an assertion mismatch.
  Remediation: Guard these two tests with a conditional it.skip when availableTools is empty.

• [edge-case] e2e-tests/plugin-dev.test.ts:213 — describe.each(availableTools) produces zero test cases when availableTools is empty, omitting the entire lifecycle suite without a Jest skip annotation. A console.log at line 53 reports "Detected compose tools: none" but Jest shows no skip indicator.
  Remediation: Add an explicit it.skip guard when availableTools is empty to make the omission visible in the Jest summary.

• [type-correctness] e2e-tests/plugin-dev.test.ts:22 — runExpectingFailure declares options: { cwd?: string } but is called at line 84 with { cwd, env: { ...process.env, RHDH_LOCAL_DIR: '', E2E_RHDH_LOCAL_DIR: '' } }. The env key is not in the declared type. This works at runtime (TypeScript erases the type and runCommand spreads ...options into exec()), and e2e-tests/ is excluded from yarn tsc scope — so no current compile error — but it is a maintenance trap if e2e files are added to the tsc include list.
  Remediation: Widen the options type to { cwd?: string; env?: NodeJS.ProcessEnv } in both runExpectingFailure and runCommand in plugin-export-build.ts.

• [fragile-assertion] e2e-tests/plugin-dev.test.ts:330 — The stop --clean assertion uses the broad negative pattern expect(...).not.toMatch(/error/i), which could produce false test failures if the CLI ever emits benign text containing "error" (e.g. "no errors detected", "error handling configured").
  Remediation: Narrow the pattern to match the CLI's specific error message prefix rather than the case-insensitive word "error".

• [code-organization] e2e-tests/support/rhdh-local-fixture.ts:53 — isComposeAvailable (lines 53–57) and detectAvailableComposeTools (lines 85–89) are exported async wrappers that delegate entirely to their synchronous counterparts. Neither is imported anywhere in the PR or the base branch.
  Remediation: Remove the unused async wrappers, or add an explicit comment if they are intentionally exported for future consumers.

• [naming-convention] e2e-tests/plugin-dev.test.ts:14 — E2E_RHDH_LOCAL_DIR is a new environment variable that influences test timeout (line 16), cleanup strategy (line 247), and which fixture the test uses (real RHDH Local checkout vs lightweight compose fixture). It is not documented in AGENTS.md or in any comment in the test file.
  Remediation: Add a comment block at the top of plugin-dev.test.ts (or a note in AGENTS.md) documenting E2E_RHDH_LOCAL_DIR and its relationship to RHDH_LOCAL_DIR.

---

Next steps:

• /fs-fix — agent addresses review findings automatically
• /fs-fix <your instruction> — agent fixes with your specific guidance
• Push commits directly — review re-runs automatically on push
• /fs-fix-stop — disable automatic fix runs for this PR

[gashcrumb]

/fs-review

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 10:46 AM UTC · Completed 11:03 AM UTC

Commit: b38388f · View workflow run →

Runtime: claude · Model: sonnet → claude-sonnet-4-6 · Effort: high · Cost: $4.46

[fullsend-ai-review]

See the review comment for full details.

[PatAKnight]

Just a couple of comments

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
1.5% Duplication on New Code

See analysis details on SonarQube Cloud

[PatAKnight]

LGTM