Skip to content

fix(e2e): mark Quick Access fixme on Kubernetes and seed the upgrade provider - #5794

Open
zdrapela wants to merge 1 commit into
redhat-developer:mainfrom
zdrapela:fix-quick-access-e2e-main
Open

zdrapela wants to merge 1 commit into
redhat-developer:mainfrom
zdrapela:fix-quick-access-e2e-main

Conversation

@zdrapela

@zdrapela zdrapela commented Nov 9, 2026 •

Copy link
Copy Markdown
Member

Description

The two Quick Access content tests fail on showcase-k8s (AKS/GKE, helm and operator) and in the OCP upgrade lane. The homepage renders and the Quick Access card is present, but shows its Error: Could not fetch data. state because the /developer-hub proxy target (test-backstage-customization-provider) is unreachable:

  • Kubernetes: the provider is OpenShift-only (# Doesn't work on K8s), and on AKS/GKE K8S_CLUSTER_ROUTER_BASE is a raw IP, so the target is not even a valid URL (skipped configuring /developer-hub due to Proxy target is not a valid URL).
  • Upgrade lane: handle_ocp_helm_upgrade never deployed the provider, so GET /api/proxy/developer-hub returned 503.

Both are pre-existing and affect main today. Not a product bug.

Changes

  • Mark both Quick Access content tests test.fixme when IS_OPENSHIFT === "false" — they are known-broken on Kubernetes, not deliberately skipped.
  • Deploy test-backstage-customization-provider in the OCP upgrade lane right after initiate_upgrade_base_deployments, so the build finishes while the baseline RHDH waits for readiness.

Which issue(s) does this PR fix

Jira: https://redhat.atlassian.net/browse/RHDHBUGS-4096

RHDHBUGS-3950 covered the Kubernetes part but was closed without restoring the coverage.

How to test changes / Special notes to the reviewer

…provider

- Mark both Quick Access content tests test.fixme when IS_OPENSHIFT=false.
  The /developer-hub proxy target (test-backstage-customization-provider) is
  OpenShift-only, so AKS/GKE render the card's "Could not fetch data." state.
  Tracked in RHDHBUGS-4096; the previous RHDHBUGS-3950 was closed without
  restoring the coverage.

- Deploy test-backstage-customization-provider in the OCP upgrade lane, which
  never provisioned it and left /api/proxy/developer-hub answering 503.

Assisted-by: OpenCode
@rhdh-qodo-merge

Copy link
Copy Markdown

PR Summary by Qodo

Fix Quick Access E2E coverage in Kubernetes and OCP upgrade lanes

🐞 Bug fix 🧪 Tests 🕐 10-20 Minutes

Grey Divider

AI Description

• Mark two Quick Access content tests as known-broken on Kubernetes, where their provider is
 unavailable.
• Deploy the customization provider in the OCP upgrade lane so the tests can fetch content.
Diagram

graph TD
  Upgrade["OCP upgrade"] --> Baseline["Baseline deployment"] --> Provider["Customization provider"] --> Proxy["Developer Hub proxy"] --> Tests["Quick Access tests"]
  Kubernetes["Kubernetes lane"] --> Fixme["Known-broken tests"]
Loading
High-Level Assessment

The approach matches the existing OCP lanes, which deploy the same provider, while accurately recording unavailable Kubernetes coverage. Enabling the provider on Kubernetes would restore coverage but requires a separate infrastructure change.

Files changed (3) +6 / -4

Bug fix (1) +2 / -0
upgrade.shProvision the customization provider during OCP upgrades +2/-0

Provision the customization provider during OCP upgrades

• Deploys the test customization provider after initiating the baseline deployment and before readiness checks, making its content available to upgrade-lane tests.

.ci/pipelines/jobs/upgrade.sh

Tests (2) +4 / -4
guest-signin-happy-path.spec.tsMark the guest Quick Access check known-broken on Kubernetes +2/-2

Mark the guest Quick Access check known-broken on Kubernetes

• Changes the Kubernetes condition from skip to fixme and points the reason to RHDHBUGS-4096. The check remains active on OpenShift.

e2e-tests/playwright/e2e/guest-signin-happy-path.spec.ts

home-page-customization.spec.tsMark customized Quick Access checks known-broken on Kubernetes +2/-2

Mark customized Quick Access checks known-broken on Kubernetes

• Changes the Kubernetes condition from skip to fixme and updates its tracking issue to RHDHBUGS-4096. The content assertions remain active on OpenShift.

e2e-tests/playwright/e2e/home-page-customization.spec.ts

@sonarqubecloud

sonarqubecloud Bot commented Nov 9, 2026

Copy link
Copy Markdown

@rhdh-qodo-merge

Copy link
Copy Markdown

Code Review by Qodo


View low (1)
Informational
1. Provider deploy failure in upgrade lane goes unnoticed 🐞 Bug
Description
The new deploy_test_backstage_customization_provider call in handle_ocp_helm_upgrade has no `||
return 1 guard, so a failed oc new-app or oc expose` is ignored and the job continues to the
baseline readiness step. If the provider never gets deployed, the Quick Access tests will fail later
with the same 503 this PR is meant to fix, and the root cause will not be visible in the deploy
phase.
Code

.ci/pipelines/jobs/upgrade.sh[148]

+  deploy_test_backstage_customization_provider "${NAME_SPACE}"
Relevance

●●● Strong

Missing error propagation is a trivial, deterministic fix; team accepted similar || return 1
guards (PR #5709). No dismissals or rejections of this pattern found.

PR-#5709

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The function's failures reach the caller only through its return status, and the new call site
discards it, so a deploy error is silent until the Quick Access tests fail. The same unchecked
pattern appears in the other OCP job scripts, so this is a consistency issue in the existing pattern
rather than a new defect in the logic.
Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
In `.ci/pipelines/jobs/upgrade.sh` the new call `deploy_test_backstage_customization_provider "${NAME_SPACE}"` (inside `handle_ocp_helm_upgrade`) ignores its return status. A failed `oc new-app` or `oc expose` is silent, and the job continues until the Quick Access tests fail with a 503.

## Fix Focus Areas
- .ci/pipelines/jobs/upgrade.sh[148-148]
- .ci/pipelines/utils.sh[366-385]

## Recommended Fix
Change the call to `deploy_test_backstage_customization_provider "${NAME_SPACE}" || { log::error "Failed to deploy test-backstage-customization-provider"; return 1; }`, so the failure is logged and the upgrade stops at the deploy phase, matching the existing `check_backstage_running` error handling a few lines below.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Dismiss ↗ | View ↗


@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

Image was built and published successfully. It is available at:

@codecov

codecov Bot commented Nov 9, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 72.94%. Comparing base (0ebef90) to head (7e2e0c4).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5794      +/-   ##
==========================================
- Coverage   78.65%   72.94%   -5.71%     
==========================================
  Files          67       60       -7     
  Lines         726      573     -153     
  Branches      100       83      -17     
==========================================
- Hits          571      418     -153     
  Misses        154      154              
  Partials        1        1              
Flag Coverage Δ *Carryforward flag
rhdh 69.00% <ø> (-6.70%) ⬇️
rhdh-e2e-frontend 71.23% <ø> (ø) Carriedforward from 0ebef90

*This pull request uses carry forward flags. Click here to find out more.

Components Coverage Δ
Backend plugins ∅ <ø> (∅)
Backend app 78.94% <ø> (ø)
Frontend app 68.40% <ø> (ø)

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0ebef90...7e2e0c4. Read the comment docs.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@zdrapela

zdrapela commented Nov 9, 2026

Copy link
Copy Markdown
Member Author

/test e2e-ocp-helm-upgrade-nightly

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant