Repository navigation
fix(ci): unblock disconnected OCP nightlies - #5709
openshift-merge-bot[bot] merged 16 commits into
Conversation
|
/test e2e-ocp-disconnected-helm-nightly |
PR Summary by QodoUnblock disconnected OCP Helm and Operator nightlies
AI Description
Diagram
High-Level Assessment
Files changed (11)
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #5709 +/- ##
==========================================
- Coverage 75.70% 69.00% -6.70%
==========================================
Files 65 58 -7
Lines 708 555 -153
Branches 100 83 -17
==========================================
- Hits 536 383 -153
Misses 171 171
Partials 1 1
Continue to review full report in Codecov by Harness.
🚀 New features to boost your workflow:
|
Code Review by Qodo
View medium (1)🟠 **Medium** 1. Older release smoke jobs stop at mirroring
|
|
/test e2e-ocp-disconnected-helm-nightly |
|
/test e2e-ocp-disconnected-helm-nightly |
|
/test e2e-ocp-disconnected-helm-nightly |
3a2bd03 to
18f4dc8
Compare
|
/test e2e-ocp-disconnected-helm-nightly |
6164795 to
edd3dba
Compare
|
The container image build workflow finished with status: |
The Helm and Operator disconnected nightlies have not passed for over five weeks. Three independent gaps kept them red: - Backend egress to the mirror registry. The chart and the Operator both default-deny backend egress except DNS, HTTPS, Redis, and PostgreSQL, while the CI mirror serves plain HTTP on port 5000. install-dynamic-plugins timed out pulling the catalog index from the bastion. - Operator catalog. A filtered oc-mirror run against the published Red Hat catalog rejects it since rhdh-operator#3822 removed the default `fast` channel, and that catalog carries no 2.x bundles anyway. CI runs now mirror the RHDH CI catalog (quay.io/rhdh/iib:<stream>-v<ocp>-x86_64); its registry.redhat.io/rhdh relatedImages are redirected to quay.io/rhdh with CONTAINERS_REGISTRIES_CONF for the prepare-restricted-environment.sh call. - Smoke plugin set. On the 2.x New Frontend System hub the sign-in page, sidebar, SCM integrations, and guest auth provider are dynamic plugins, so the homepage-only set could not render a login. Mirror the minimal digest pinned set and configure the NFS extensions and signInPage in the app configs. The 2.x hub image and the new plugin set were verified on a live OCP cluster: guest sign-in, sidebar, and the "Welcome back!" homepage heading all render. Assisted-by: OpenCode
- Pick the operator catalog from the chart build type instead of the image registry alone: a CI chart (x.y-N-CI) mirrors the RHDH CI catalog, a released chart (x.y.z) keeps the published catalog. A pinned RC/GA verification on main (TAG_NAME and CHART_VERSION 1.9.4) was handed a `next` catalog for a 1.9 channel filter. - Export the smoke plugin list as a space-separated scalar. bash cannot export arrays, so the `export -f` helpers saw an empty list in a child shell and would have mirrored every digest-pinned plugin. Fail when no package variable was exported. - Document why the mirror egress rule is scoped by port and not by destination: the bastion address is assigned per run and reachable by DNS name only, so an ipBlock would break whenever the bastion is replaced. Assisted-by: OpenCode
The chart mounts <fullname>-dynamic-plugins-registry-auth, and fullname is <release>-<nameOverride> with nameOverride pinned to developer-hub. The job created rhdh-dynamic-plugins-registry-auth instead, so the optional volume was never mounted, install-dynamic-plugins ran without auth.json, and skopeo failed the mirror pull with "authentication required". Pin nameOverride in the values and create the secret under the chart's name. Verified on a live OCP 4.21 cluster (LOCAL_DISCONNECTED): the init container now pulls the catalog index and all five smoke plugins through the mirror. Assisted-by: OpenCode
Helm: after the IDMS/ITMS apply, `oc wait machineconfigpool --for=condition=Updated=True` returned while the MCO had not rendered the new config yet. The nodes were drained a few seconds later, while the hub and PostgreSQL pods were starting, so they were evicted mid-startup. The hub recovery path then ran `oc rollout restart`, starting a second backend next to the first against a freshly initialised database; the two raced on the knex migration lock and the losing backend stayed up with readiness 503 forever. - snapshot each pool's rendered config before the IDMS/ITMS and additionalTrustedCA changes, and wait for the MCO to render before waiting for convergence (poll Updated + updatedMachineCount; warn and continue when nothing is rolling) - start the hub restart grace period only once the backend container has started, and replace the rollout restart with scale to 0, wait for the pods to be gone, scale to 1, so two backends never overlap Operator: OLM v1 installs the bundle with the rhdh-operator-installer ServiceAccount, which cannot grant RBAC it does not hold itself. The installer ClusterRole rendered by prepare-restricted-environment.sh lags behind config/rbac/role.yaml (pods, apiservers/ingresses, devhubplugincatalogs), so the operator ClusterRole creation is refused and the operator ends up running with no permissions. The half-applied release still creates the CRD, so the CRD-only wait passed and the job failed later with a misleading message. - require ClusterExtension Installed=True as well as the CRD, log the install condition at info level, and dump OLM v1 status when the Backstage CR produces no deployment - TEMPORARY: grant the installer the operator's own ClusterRole rules from the same release branch before prepare runs, so the first install attempt succeeds. Remove once redhat-developer/rhdh-operator ships the fix. Verified the RBAC workaround on a live OCP cluster: the same ClusterRole creation is refused before the grant and succeeds after it. Assisted-by: OpenCode
Point the temporary OLM v1 installer workaround at redhat-developer/rhdh-operator#3923 (backport redhat-developer#3924) and RHDHBUGS-4056, so the patch is easy to find and remove once the fix is released. Assisted-by: OpenCode
redhat-developer/rhdh-operator#3923 is merged, so the prepare-restricted-environment.sh fetched from rhdh-operator main renders an installer ClusterRole that covers the bundle permissions (pods, apiservers/ingresses, devhubplugincatalogs). The disconnected operator job downloads that script at runtime, so the workaround is no longer needed on main. release-2.1 keeps the workaround until redhat-developer#3924 merges, because that job fetches the script from rhdh-operator release-2.1. Assisted-by: OpenCode
The operator job took the bundle filter from CHART_VERSION while the catalog tag came from RELEASE_VERSION, so a pinned run on main (for example an RC tag) asked oc-mirror for bundles the `next` catalog does not carry. Both now come from the branch: RELEASE_VERSION for the catalog tag (next on main, 2.1 on release-2.1) and common::get_release_stream for the filter, which needs a major.minor and resolves main to the package.json stream (2.2). Move that helper out of lib/helm.sh while at it: the operator and upgrade jobs use it too, so it lives in lib/common.sh as common::get_release_stream. Assisted-by: OpenCode
Drop the comments that only describe the PR history or restate the code (merged rhdh-operator PRs, the NFS plugin rationale repeated in three files, snapshot double-comments, a duplicated secret-mount paragraph). Keep the notes that explain non-obvious behaviour: the guest sign-in page, nameOverride coupling, CI vs released catalogs, the registries.conf redirect and the additionalTrustedCA node roll. Also log the configured MachineConfigPool timeout instead of a hardcoded 20m, and correct the mirror egress rationale: MIRROR_REGISTRY_URL is the bastion's public EC2 DNS name, which resolves differently from inside the cluster, so there is no stable ipBlock to use. Assisted-by: OpenCode
edd3dba to
cdae8e9
Compare
|
The container image build workflow finished with status: |
|
/agentic_review |
The operator job picked the catalog tag from RELEASE_VERSION and the --filter-versions value from common::get_release_stream. On main those are "next" and the package.json stream (2.2), which agree only while rhdh and rhdh-operator bump together; when one side lags, prepare keeps a fast-X.Y channel the catalog does not carry, or oc-mirror rejects the catalog because its default channel is filtered out. Use filter_versions for the tag as well. Both catalog tags resolve to the same digest today (iib:next-v4.21 == iib:2.2-v4.21), so nightlies are unchanged, and a stream without a published catalog now fails at the skopeo check with a readable error. Assisted-by: OpenCode
- list jq in common.sh dependencies: the moved get_release_stream uses it - rename mcp_all_converged to the underscore prefix the other private helpers use - drop the unreachable exported -eq 0 branch in the smoke plugin export - document the hub grace period as it behaves: it covers the backend start wait, then the Available wait Assisted-by: OpenCode
|
/test e2e-ocp-disconnected-helm-nightly |
|
/config |
🛠️ Wiki configuration file settings:🛠️ Local configuration file settings:
🛠️ Global configuration file settings:🛠️ PR-Agent final configurations:==================== CONFIG ====================
config.fixer_agent_provenance = {'enable': 'platform', 'enable_auto_fix': 'platform', 'enable_fixer_agent': 'platform'}
config.portal_base_url = 'https://app.qodo.ai'
config.is_new_pr = False
config.is_auto_command = False
config.organization_id = 'redhat.com'
config.qodo_llm_gateway_metadata_tags = True
config.fallback_models = ['openai/anthropic/claude-sonnet-5-5_thinking', 'openai/multi-provider/gpt-5.6-terra_thinking', 'openai/xai/grok-4.6_thinking']
config.pr_compliance = {'ENABLE_RULES_PLATFORM': True}
config.model_medium = 'openai/multi-provider/gpt-5.6-terra_thinking'
config.model_turbo = 'openai/multi-provider/gpt-5.6-luna_thinking'
config.fetch_github_apps_from_platform = True
config.ignore_pr_authors = 'REDACTED'
config.model_reasoning = 'openai/vertex_ai/gemini-4-argon'
config.invite_only_mode = True
config.model = 'openai/multi-provider/gpt-6-sol'
config.use_async_clone = True
config.treat_llm_quota_error_as_healthy = False
config.git_provider = 'github'
config.publish_output = True
config.publish_output_no_suggestions = True
config.publish_output_progress = True
config.enable_v1_deprecation_banner = True
config.enable_legacy_command_qodo_banner = True
config.sunset_bypass_commands = []
config.ignore_commands = []
config.verbosity_level = 0
config.log_level = 'INFO'
config.publish_logs = False
config.debug_mode = False
config.use_wiki_settings_file = True
config.use_repo_settings_file = True
config.use_global_settings_file = True
config.use_global_wiki_settings_file = False
config.global_settings_file_overrides_platform = False
config.gate_pr_supplied_ambient_models = True
config.pr_supplied_ambient_model_allowlist = []
config.disable_auto_feedback = False
config.ai_timeout = 300
config.obfuscation_max_input_chars = 200000
config.response_language = 'en-US'
config.clone_repo_instead_of_fetch = True
config.always_clone = False
config.add_repo_metadata = True
config.add_repo_metadata_resolve_references = False
config.add_repo_metadata_max_resolved_lines = 1500
config.clone_repo_time_limit = 600
config.clone_transport_retry = False
config.publish_inline_comments_fallback_batch_size = 5
config.publish_inline_comments_fallback_sleep_time = 2
config.max_model_tokens = 32000
config.custom_model_max_tokens = -1
config.patch_extension_skip_types = ['.md', '.txt']
config.extra_allowed_extensions = []
config.allow_dynamic_context = True
config.allow_forward_dynamic_context = True
config.max_extra_lines_before_dynamic_context = 12
config.patch_extra_lines_before = 5
config.patch_extra_lines_after = 1
config.ai_handler = 'litellm'
config.qodo_llm_gateway_user_header = ''
config.qodo_llm_gateway_user_header_field = 'sender'
config.cli_mode = False
config.disable_platform_features = False
config.trial_git_org_max_invokes_per_month = 30
config.trial_ratio_close_to_limit = 0.8
config.quota_just_exceeded_message = '### Qodo reviews are paused for this user.\n\nTroubleshooting steps vary by plan [Learn more →](https://docs.qodo.ai/subscription-plans#what-you%E2%80%99ll-see-when-reviews-are-paused)\n\n\n**On a Teams plan?**\nReviews resume once this user has a paid seat *and* their Git account is linked in Qodo.\n[Link Git account →](https://docs.qodo.ai/subscription-plans#linking-a-git-account)\n\n**Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center?**\nThese require an Enterprise plan - Contact us\n[Contact us →](https://docs.qodo.ai/qodo-support#support)\n'
config.billing_trial_expiring_message = '**ⓘ Your Qodo trial ends soon.** Ask your workspace admin to set up billing to keep reviews running after the trial. [Manage billing](https://app.qodo.ai/account/billing/manage-subscription?traffic_source=pr_comment)'
config.billing_payment_failure_message = "**ⓘ A recent Qodo payment didn't go through.** Ask your workspace admin to update payment details to keep reviews running. [Manage billing](https://app.qodo.ai/account/billing/manage-subscription?traffic_source=pr_comment)"
config.trial_expiring_notice_enabled = True
config.payment_failure_notice_enabled = True
config.enable_per_org_invite_only = False
config.enable_request_access_msg_on_new_pr = False
config.check_also_invites_field = False
config.allowed_users = []
config.disable_checkboxes = False
config.output_relevant_configurations = False
config.large_patch_policy = 'clip'
config.temperature = 0.2
config.allow_dynamic_context_ab_testing = False
config.choose_dynamic_context_ab_testing_ratio = 0.5
config.ignore_pr_title = ['^\\[Auto\\]', '^Auto']
config.ignore_pr_target_branches = []
config.ignore_pr_source_branches = []
config.ignore_pr_labels = []
config.ignore_ticket_labels = []
config.allow_only_specific_folders = []
config.ignore_repositories = []
config.rules_platform_excluded_repositories = []
config.ignore_bot_pr = True
config.ignore_language_framework = []
config.enable_ai_metadata = True
config.enable_description_image_reachability_probe = True
config.read_file_max_image_bytes = 5242880
config.max_tickets = 10
config.max_tickets_chars = 8000
config.extract_tickets_from_pr_title = False
config.max_tickets_from_title = 1
config.title_extraction_skip_patterns = ['^revert\\b', '^merge\\b', '^release\\b', '^backport\\b', '^bump\\b']
config.prevent_any_approval = False
config.prevent_any_block = False
config.enable_comment_approval = False
config.enable_auto_approval = False
config.auto_approve_for_low_review_effort = -1
config.auto_approve_for_no_suggestions = False
config.ensure_ticket_compliance = False
config.new_diff_format = True
config.new_diff_format_add_external_references = True
config.tasks_queue_ttl_from_dequeue_in_seconds = 1200
config.pr_commands = ['/agentic_describe', '/agentic_review']
config.push_commands = ['/agentic_review']
config.handle_pr_actions = ['opened', 'reopened', 'ready_for_review']
config.handle_push_trigger = False
config.feedback_on_draft_pr = False
config.enable_custom_labels = False
config.custom_labels_discovery_prefixes = ['pr_agent:', 'qodo:']
==================== PR_REVIEWER ====================
pr_reviewer.require_score_review = False
pr_reviewer.require_tests_review = True
pr_reviewer.require_estimate_effort_to_review = True
pr_reviewer.require_can_be_split_review = False
pr_reviewer.require_security_review = True
pr_reviewer.require_todo_scan = False
pr_reviewer.require_ticket_analysis_review = True
pr_reviewer.require_ticket_labels = False
pr_reviewer.require_no_ticket_labels = False
pr_reviewer.check_pr_additional_content = False
pr_reviewer.persistent_comment = True
pr_reviewer.extra_instructions = ''
pr_reviewer.final_update_message = True
pr_reviewer.enable_review_labels_security = True
pr_reviewer.enable_review_labels_effort = True
pr_reviewer.enable_help_text = False
==================== PR_COMPLIANCE ====================
pr_compliance.enabled = True
pr_compliance.enable_rules_platform = True
pr_compliance.rule_providers = []
pr_compliance.rule_providers_max_rules = 0
pr_compliance.filter_rules_by_history_band = False
pr_compliance.assign_skill_rules_to_skill_findings = True
pr_compliance.enable_security_section = True
pr_compliance.enable_ticket_section = True
pr_compliance.enable_codebase_duplication_section = True
pr_compliance.enable_custom_compliance_section = True
pr_compliance.require_ticket_analysis_review = True
pr_compliance.allow_repo_pr_compliance = True
pr_compliance.enable_global_pr_compliance = True
pr_compliance.local_wiki_compliance_str = ''
pr_compliance.global_wiki_pr_compliance = ''
pr_compliance.local_repo_compliance_str = ''
pr_compliance.global_repo_pr_compliance_str = ''
pr_compliance.global_compliance_str = ''
pr_compliance.enable_generic_custom_compliance_checklist = True
pr_compliance.persist_generic_custom_compliance_checklist = False
pr_compliance.display_no_compliance_only = False
pr_compliance.enable_security_compliance = True
pr_compliance.enable_update_pr_compliance_checkbox = True
pr_compliance.enable_todo_scan = False
pr_compliance.enable_ticket_labels = False
pr_compliance.enable_no_ticket_labels = False
pr_compliance.check_pr_additional_content = False
pr_compliance.enable_compliance_labels_security = True
pr_compliance.enable_user_defined_compliance_labels = True
pr_compliance.enable_estimate_effort_to_review = True
pr_compliance.max_rag_components_to_analyze = 5
pr_compliance.min_component_size = 5
pr_compliance.persistent_comment = True
pr_compliance.enable_help_text = False
pr_compliance.extra_instructions = ''
==================== PR_DESCRIPTION ====================
pr_description.publish_labels = False
pr_description.add_original_user_description = True
pr_description.generate_ai_title = False
pr_description.extra_instructions = ''
pr_description.enable_pr_type = True
pr_description.final_update_message = True
pr_description.enable_help_text = False
pr_description.enable_help_comment = False
pr_description.bring_latest_tag = False
pr_description.enable_pr_diagram = True
pr_description.pr_diagram_direction = 'LR'
pr_description.publish_description_as_comment = False
pr_description.publish_description_as_comment_persistent = True
pr_description.enable_semantic_files_types = True
pr_description.collapsible_file_list = 'adaptive'
pr_description.collapsible_file_list_threshold = 8
pr_description.inline_file_summary = False
pr_description.use_description_markers = False
pr_description.include_generated_by_header = True
pr_description.enable_large_pr_handling = True
pr_description.max_ai_calls = 4
pr_description.auto_create_ticket = False
==================== PR_AGENTIC_DESCRIPTION ====================
pr_agentic_description.enable_file_changes_section = True
pr_agentic_description.file_listing_style = 'cards'
pr_agentic_description.publish_as_comment = True
==================== PR_CODE_SUGGESTIONS ====================
pr_code_suggestions.suggestions_depth = 'regular'
pr_code_suggestions.commitable_code_suggestions = False
pr_code_suggestions.decouple_hunks = False
pr_code_suggestions.dual_publishing_score_threshold = -1
pr_code_suggestions.focus_only_on_problems = True
pr_code_suggestions.allow_thumbs_up_down = False
pr_code_suggestions.enable_suggestion_type_reuse = False
pr_code_suggestions.enable_more_suggestions_checkbox = True
pr_code_suggestions.high_level_suggestions_enabled = True
pr_code_suggestions.extra_instructions = ''
pr_code_suggestions.enable_help_text = False
pr_code_suggestions.show_extra_context = False
pr_code_suggestions.persistent_comment = True
pr_code_suggestions.max_history_len = 5
pr_code_suggestions.apply_suggestions_checkbox = True
pr_code_suggestions.enable_chat_in_code_suggestions = True
pr_code_suggestions.apply_limit_scope = True
pr_code_suggestions.suggestions_score_threshold = 0
pr_code_suggestions.new_score_mechanism = True
pr_code_suggestions.new_score_mechanism_th_high = 9
pr_code_suggestions.new_score_mechanism_th_medium = 7
pr_code_suggestions.discard_unappliable_suggestions = False
pr_code_suggestions.num_code_suggestions_per_chunk = 3
pr_code_suggestions.num_best_practice_suggestions = 2
pr_code_suggestions.max_number_of_calls = 3
pr_code_suggestions.demand_code_suggestions_self_review = False
pr_code_suggestions.code_suggestions_self_review_text = '**Author self-review**: I have reviewed the PR code suggestions, and addressed the relevant ones.'
pr_code_suggestions.approve_pr_on_self_review = False
pr_code_suggestions.fold_suggestions_on_self_review = True
pr_code_suggestions.publish_post_process_suggestion_impact = True
pr_code_suggestions.wiki_page_accepted_suggestions = True
pr_code_suggestions.simplify_response = True
==================== PR_CUSTOM_PROMPT ====================
pr_custom_prompt.prompt = 'The code suggestions should focus only on the following:\n- ...\n- ...\n...\n'
pr_custom_prompt.suggestions_score_threshold = 0
pr_custom_prompt.num_code_suggestions_per_chunk = 4
pr_custom_prompt.self_reflect_on_custom_suggestions = True
pr_custom_prompt.enable_help_text = False
==================== PR_ADD_DOCS ====================
pr_add_docs.extra_instructions = ''
pr_add_docs.docs_style = 'Sphinx'
pr_add_docs.file = ''
pr_add_docs.class_name = ''
==================== PR_UPDATE_CHANGELOG ====================
pr_update_changelog.push_changelog_changes = False
pr_update_changelog.extra_instructions = ''
pr_update_changelog.add_pr_link = True
pr_update_changelog.skip_ci_on_push = True
==================== PR_ANALYZE ====================
pr_analyze.enable_help_text = False
==================== PR_TEST ====================
pr_test.enable = True
pr_test.extra_instructions = ''
pr_test.testing_framework = ''
pr_test.num_tests = 3
pr_test.avoid_mocks = True
pr_test.file = ''
pr_test.class_name = ''
pr_test.enable_help_text = False
==================== PR_IMPROVE_COMPONENT ====================
pr_improve_component.num_code_suggestions = 4
pr_improve_component.extra_instructions = ''
pr_improve_component.file = ''
pr_improve_component.class_name = ''
==================== REVIEW_AGENT ====================
review_agent.enabled = True
review_agent.subagent_llm_model = 'openai/multi-provider/gpt-5.6-luna_thinking'
review_agent.llm_model = 'openai/multi-provider/gpt-6-sol_thinking'
review_agent.smart_router_scope_provenance = False
review_agent.ensemble_models = ['openai/multi-provider/gpt-6-sol_thinking', 'openai/anthropic/claude-opus-5-5_thinking']
review_agent.enable_database_persistence = True
review_agent.publish_output = True
review_agent.enable_standard_mode = True
review_agent.enable_skip_mode = True
review_agent.enable_lite_mode = True
review_agent.enable_extended_mode = True
review_agent.override_effort = ''
review_agent.publish_blast_radius_labels = False
review_agent.push_trigger_dedup_enabled = True
review_agent.push_trigger_dedup_ttl_seconds = 60
review_agent.mr_trigger_dedup_ttl_seconds = 300
review_agent.enable_issues_agent = True
review_agent.enable_ticket_context = True
review_agent.max_ticket_fetch_chars = 200000
review_agent.max_ticket_tokens = 'REDACTED'
review_agent.max_requirements_ticket_tokens = 'REDACTED'
review_agent.max_spec_tokens = 'REDACTED'
review_agent.context_summary_llm_model = 'turbo'
review_agent.context_summary_timeout_seconds = 90
review_agent.enable_compliance_agent = True
review_agent.enable_spec_agent = True
review_agent.enable_ui_agent = True
review_agent.enable_figma_findings_revalidation = True
review_agent.enable_rules_findings_revalidation = True
review_agent.enable_persona_agent = False
review_agent.persona_identifier = ''
review_agent.persona_auto_select = True
review_agent.persona_max_count = 2
review_agent.persona_portal_base_url = ''
review_agent.enable_finding_dismissal_link = True
review_agent.enable_deduplication = True
review_agent.enable_conversion_agent = True
review_agent.apply_conversion = True
review_agent.is_advanced_configuration = False
review_agent.enable_precision_agent = False
review_agent.enable_unobserved_citation_filter = False
review_agent.citation_text_timeout_seconds = 60
review_agent.enable_off_change_findings_filter = True
review_agent.enable_cross_repo_agent = True
review_agent.security_scan_timeout_seconds = 180
review_agent.security_scan_max_concurrency = 2
review_agent.enable_smart_router = True
review_agent.a2a_pre_pr_review_enabled = True
review_agent.qodo_review_skill_carry_window_days = 7
review_agent.prepr_max_detailed_binding_checks = 100
review_agent.enable_past_bugs_collector = False
review_agent.enable_past_issues_context = True
review_agent.enable_skills_agent = True
review_agent.enable_skill_steering = True
review_agent.enable_cross_repo_skill_steering = True
review_agent.enable_web_search_tool = False
review_agent.enable_review_md = False
review_agent.serve_pr_repo_via_gitway = False
review_agent.serve_pr_repo_via_gitway_ratio = 1.0
review_agent.pr_repo_gitway_read_timeout_seconds = 30
review_agent.pr_repo_gitway_read_retry_budget_seconds = 60
review_agent.pr_repo_gitway_midrun_clone_timeout_seconds = 120
review_agent.enable_sandbox_usage = False
review_agent.sandbox_provider = 'langsmith'
review_agent.sandbox_configs_repo_name = 'qodo-sandbox-configs'
review_agent.sandbox_config_source = 'git_repo'
review_agent.enable_sandbox_agent_tools = False
review_agent.persistent_comment = True
review_agent.persistent_comment_notification = False
review_agent.add_history_audit = True
review_agent.publish_in_progress_comment = False
review_agent.attribution_runs_retention_days = 30
review_agent.review_components_stats_retention_days = 90
review_agent.enable_incremental_review = True
review_agent.enable_fix_loop_optimization = True
review_agent.fix_loop_min_rounds = 2
review_agent.fix_loop_recency_window_minutes = 5760
review_agent.fix_loop_attribution_wait_seconds = 15
review_agent.fix_loop_intent_llm_model = 'turbo'
review_agent.fix_loop_intent_timeout_seconds = 20
review_agent.fix_loop_intent_fallback_models = []
review_agent.fix_loop_intent_max_author_tokens = 'REDACTED'
review_agent.fix_loop_drop_below_rank = 3
review_agent.fix_loop_narrow_agents = True
review_agent.enable_previous_findings = True
review_agent.skip_review_on_empty_incremental_diff = True
review_agent.enable_a2a_auto_fix = False
review_agent.enable_review_on_fix_pr = False
review_agent.rules_enabled = True
review_agent.requirements_gap_enabled = True
review_agent.llm_call_timeout = 180
review_agent.sub_agent_timeout_seconds = 2400
review_agent.llm_call_overall_timeout = 300
review_agent.finalization_llm_call_timeout_seconds = 240
review_agent.weak_models_harness_enabled = False
review_agent.weak_models = ['laguna']
review_agent.weak_models_llm_call_overall_timeout = 120
review_agent.weak_models_vllm_thinking_switch = True
review_agent.weak_models_max_llm_calls = 30
review_agent.smart_router_llm_model = 'turbo'
review_agent.smart_router_max_llm_calls = 4
review_agent.smart_router_disabled_on_open_pr = False
review_agent.smart_router_extra_instructions = ''
review_agent.router_extended_floor_hunks = 18
review_agent.router_extended_floor_lines = 200
review_agent.lite_mode_llm_model = 'turbo'
review_agent.compliance_ensemble_models = []
review_agent.compliance_llm_model = 'turbo'
review_agent.enable_explorer_subagent = True
review_agent.enable_downstream_impact_subagent = False
review_agent.downstream_impact_llm_model = 'turbo'
review_agent.downstream_impact_timeout_seconds = 150
review_agent.enable_split_review = False
review_agent.split_review_min_changed_lines = 2500
review_agent.split_review_min_changed_files = 15
review_agent.split_review_llm_model = 'medium'
review_agent.split_review_max_slices = 8
review_agent.split_review_timeout_seconds = 900
review_agent.spec_llm_model = ''
review_agent.persona_llm_model = ''
review_agent.persona_selector_llm_model = ''
review_agent.conversion_llm_model = 'turbo'
review_agent.conversion_batching_mode = 'batch'
review_agent.conversion_batch_size = 10
review_agent.conversion_exploration_rate = 0.1
review_agent.cross_repo_llm_model = ''
review_agent.precision_llm_model = 'turbo'
review_agent.security_llm_model = ''
review_agent.memory_llm_model = ''
review_agent.precision_max_llm_calls = 10
review_agent.precision_max_concurrency = 5
review_agent.langsmith_project_name = 'review-agent'
review_agent.max_tokens_for_file = 'REDACTED'
review_agent.single_unified_diff_tokens_limit = 'REDACTED'
review_agent.offload_test_files = True
review_agent.offload_docs_files = True
review_agent.max_llm_calls = 100
review_agent.max_llm_calls_limit = 100
review_agent.warn_when_remaining = 3
review_agent.enable_agent_time_limit = True
review_agent.max_agent_time = 900
review_agent.time_warn_when_remaining_seconds = 120
review_agent.compliance_batch_size = 35
review_agent.compliance_batch_concurrency = 5
review_agent.past_bugs_max_results = 10
review_agent.past_bugs_llm_model = 'turbo'
review_agent.skills_agent_dirs = ['.qodo/skills', '.claude/skills', '.cursor/skills', '.agents/skills', 'skills']
review_agent.skill_discovery_timeout_seconds = 30
review_agent.pass_previous_findings_to_agents = True
review_agent.prepr_pass_previous_findings_to_agents = False
review_agent.deduplication_llm_max_tokens = 'REDACTED'
review_agent.publishing_action_level_rank_threshold = 0
review_agent.comments_location_policy = 'both'
review_agent.publish_inline_acknowledgment = False
review_agent.enable_commitable_suggestions = False
review_agent.commitable_suggestions_max_lines = 5
review_agent.commitable_suggestions_max_concurrency = 5
review_agent.prefer_single_line_comments = False
review_agent.issues_user_guidelines = ''
review_agent.compliance_user_guidelines = ''
review_agent.enable_ticket_spec_hop = False
review_agent.additional_repos = []
review_agent.demand_self_review = False
review_agent.self_review_text = '**Author self-review**: I have reviewed the code review findings, and addressed the relevant ones.'
review_agent.approve_pr_on_self_review = False
==================== REVIEW_AGENT_UX ====================
review_agent_ux.finding_overflow_count = 3
review_agent_ux.resolved_overflow_count = 0
review_agent_ux.expand_description = False
review_agent_ux.expand_code = False
review_agent_ux.expand_evidence = False
review_agent_ux.expand_recommendation = False
review_agent_ux.expand_prompt = False
review_agent_ux.render_description = True
review_agent_ux.render_code = True
review_agent_ux.render_recommendation = True
review_agent_ux.render_evidence = True
review_agent_ux.show_prompt = True
review_agent_ux.show_context_used = False
review_agent_ux.sub_categories_display = 'verbose'
review_agent_ux.use_images_and_animations = False
review_agent_ux.resolved_badge_display = 'both'
review_agent_ux.resolved_findings_placement = 'resolved_section'
review_agent_ux.show_resolved_findings = True
review_agent_ux.display_mode = 'default'
review_agent_ux.findings_group_by = 'action_level'
review_agent_ux.findings_label_coloring = 'simplified'
review_agent_ux.severity_terminology = 'high_medium_low'
review_agent_ux.findings_group_icon = False
review_agent_ux.findings_show_empty_groups = False
review_agent_ux.findings_overflow_include_group_label = True
review_agent_ux.finding_labels = ['type']
review_agent_ux.findings_show_counters = True
review_agent_ux.findings_show_zero_counters = False
review_agent_ux.findings_count_dimension = 'type'
review_agent_ux.findings_order_by = 'relevance'
review_agent_ux.findings_new_first = True
review_agent_ux.show_comment_footer = False
review_agent_ux.trial_banner_style = 'pre'
review_agent_ux.trial_banner_position = 'footer'
review_agent_ux.show_daily_tips = False
review_agent_ux.show_dividers = True
review_agent_ux.link_findings_to_inline_comments = True
review_agent_ux.show_subsection_names = False
review_agent_ux.show_persistent_review_section = False
review_agent_ux.findings_overflow = {'bugs': 3, 'ux_issue': 3, 'ui_issues': 3, 'cross_repo': 3, 'informational': 0, 'team_insights': 3, 'skill_insights': 3, 'action_required': -1, 'rule_violations': 3, 'security_issues': 3, 'requirement_gaps': 3, 'remediation_recommended': 0}
==================== PR_HELP ====================
pr_help.force_local_db = False
pr_help.num_retrieved_snippets = 5
==================== PR_NEW_ISSUE ====================
pr_new_issue.label_to_prompt_part = {'general': 'general question', 'feature': 'feature request (may already be addressed in the documentation)', 'bug': 'possible bug report (may be a by design behavior)'}
pr_new_issue.supported_repos = ['qodo-ai/pr-agent']
==================== PR_HELP_DOCS ====================
pr_help_docs.repo_url = ''
pr_help_docs.repo_default_branch = 'main'
pr_help_docs.docs_path = 'docs'
pr_help_docs.exclude_root_readme = False
pr_help_docs.supported_doc_exts = ['.md', '.mdx', '.rst']
pr_help_docs.enable_help_text = False
==================== PR_SIMILAR_ISSUE ====================
pr_similar_issue.skip_comments = False
pr_similar_issue.force_update_dataset = False
pr_similar_issue.max_issues_to_scan = 500
pr_similar_issue.vectordb = 'pinecone'
==================== PR_FIND_SIMILAR_COMPONENT ====================
pr_find_similar_component.class_name = ''
pr_find_similar_component.file = ''
pr_find_similar_component.search_from_org = False
pr_find_similar_component.allow_fallback_less_words = True
pr_find_similar_component.number_of_keywords = 5
pr_find_similar_component.number_of_results = 5
==================== BEST_PRACTICES ====================
best_practices.auto_best_practices_str = ''
best_practices.wiki_best_practices_str = ''
best_practices.global_wiki_best_practices = ''
best_practices.local_repo_best_practices_str = ''
best_practices.global_repo_best_practices_str = ''
best_practices.global_best_practices_str = ''
best_practices.organization_name = ''
best_practices.max_lines_allowed = 2000
best_practices.enable_global_best_practices = True
best_practices.allow_repo_best_practices = True
best_practices.enabled = True
==================== AUTO_BEST_PRACTICES ====================
auto_best_practices.enable_auto_best_practices = True
auto_best_practices.utilize_auto_best_practices = True
auto_best_practices.extra_instructions = ''
auto_best_practices.min_suggestions_to_auto_best_practices = 10
auto_best_practices.number_of_days_to_update = 30
auto_best_practices.max_patterns = 5
auto_best_practices.minimal_date_to_update = '2025-01-26'
==================== JIRA ====================
jira.jira_client_id = 'REDACTED'
jira.jira_app_secret = 'REDACTED'
==================== LINEAR ====================
linear.linear_client_id = 'REDACTED'
==================== PR_TO_TICKET ====================
pr_to_ticket.default_base_url = ''
pr_to_ticket.default_project_key = 'REDACTED'
pr_to_ticket.fallback_to_git_provider_issues = True
pr_to_ticket.direct_update_compliance = False
==================== PR_LIFECYCLE_EVENTS ====================
pr_lifecycle_events.project_id = 'qodo-saas-shared-services'
pr_lifecycle_events.transport = 'pubsub'
pr_lifecycle_events.enabled = True
pr_lifecycle_events.topic = 'events-v2'
pr_lifecycle_events.publish_timeout_seconds = 5.0
==================== github_app ====================
github_app.ignore_bot_pr = False
github_app.bot_user = 'github-actions[bot]'
github_app.override_deployment_type = True
github_app.handle_pr_actions = ['opened', 'reopened', 'ready_for_review']
github_app.pr_commands = ['/agentic_describe', '/agentic_review']
github_app.feedback_on_draft_pr = False
github_app.handle_push_trigger = False
github_app.push_commands = ['/agentic_review']
github_app.attribute_closed_pr_to_opener = False |
The operator job fell back to the published Red Hat catalog whenever the image registry was not registry.redhat.io or the chart was not a CI tag. That path cannot work: prepare-restricted-environment.sh writes only fast-X.Y channels into the ImageSetConfiguration, so oc-mirror rejects the published catalog because its default fast channel is filtered out, and the catalog carries no 2.x bundles at all. Every disconnected run is branch-aligned, and the hub image comes from the bundle's RELATED_IMAGE_backstage, so TAG_NAME, IMAGE_* and CHART_VERSION never reached the deployment. Use the CI catalog (quay.io/rhdh/iib:<stream>-v<ocp>-x86_64) and its registries.conf redirect unconditionally, and log that only the newest bundle of the branch's stream is tested. Assisted-by: OpenCode
Putting deploy_rhdh_operator on the left of || turns errexit off for its whole body, so a failed envsubst, yq or oc apply fell through to the 5-minute deployment poll and then reported a missing deployment, which pointed at OLM. Return from deploy_rhdh_operator as soon as rendering or applying the CR fails, and use a neutral message in the operator job (the OLM v1 dump stays as diagnostics). The Helm hub bounce scaled the Deployment back up even when the old pod was still present after 180s, recreating the two-backend race it is meant to prevent. Treat that timeout as a recovery failure. Assisted-by: OpenCode
disconnected::snapshot_mcp_rendered_configs hid oc failures by returning an empty success: a failed pre-change read skipped render detection, and a failed read during detection looked like a changed config. Retry the read three times and fail on an error or an empty answer, skip failed reads in the render loop, and require the pre-change snapshot. Both jobs now stop when it cannot be taken. Rename the MCP identifiers to machineconfigpool, since MCP reads as Model Context Protocol in this repository. Document why the IDMS and ITMS applies share one wait: a pool's spec moves to the newest rendered config, so a config rendered during the first rollout is still awaited. Assisted-by: OpenCode
The smoke set was spelled out in six places: the array, the case in _smoke_plugin_package_var, two require_vars lists and two templates. A template entry the array did not know about rendered an empty package: and only failed at pod runtime. Render both templates through one helper that maps the array to the package variables, requires them, and rejects a package: left empty by envsubst. Assisted-by: OpenCode
- the CI mirror serves TLS on port 5000, not plain HTTP: the openshift/release step curls https://<mirror>/v2/ with the vault CA - nameOverride pins the chart fullname for the registry-auth secret; the egress NetworkPolicy selects on pod labels - the smoke app config is a subset of dynamic-plugins-config.yaml, not a mirror - the Backstage CR installs the minimal NFS plugin set, not only the homepage Assisted-by: OpenCode
|
/test e2e-ocp-disconnected-helm-nightly |
A GA verification run (--image-registry registry.redhat.io with a released x.y.z chart) should pair with the published Red Hat catalog, and the existing image and chart overrides already select it. Restore that branch. It cannot keep the version filter: the published catalog declares defaultChannel fast, and oc-mirror rejects a channels list that leaves the default channel out, which is why GA runs failed at mirroring before. Pass --filter-versions '*' for the published catalog (prepare then mirrors every rhdh channel, still scoped to the rhdh package) and keep the branch-stream filter for the CI catalog, so the nightlies are unchanged. Assisted-by: OpenCode
|
/test e2e-ocp-disconnected-helm-nightly |
|
|
The container image build workflow finished with status: |
05beb15
into
redhat-developer:main



Summary
The disconnected OCP Helm and Operator nightlies (main and release-2.1) have not passed for over five weeks. Four independent gaps kept them red:
.ci/pipelines/ ├── lib/common.sh # get_release_stream moved from helm.sh ├── lib/disconnected/namespace.sh # + allow_mirror_registry_egress ├── lib/disconnected/plugins.sh # homepage-only -> minimal NFS plugin set ├── resources/disconnected/ │ ├── dynamic-plugins-homepage.yaml # -> dynamic-plugins-smoke.yaml (5 plugins) │ └── app-config-rhdh-disconnected-smoke.yaml # + NFS app.extensions, signInPage ├── value_files/values_disconnected-smoke.yaml # same plugin set + NFS config └── jobs/ ├── ocp-disconnected-helm.sh # apply egress rule; chart-name auth secret └── ocp-disconnected-operator.sh # apply egress rule; CI operator catalogBackend egress to the mirror registry. The chart and the Operator default-deny backend egress except DNS, HTTPS, Redis and PostgreSQL. The CI mirror (bastion) serves TLS on port 5000, so the
install-dynamic-pluginsskopeo pull of the plugin catalog timed out. The CI-only allow-all rule added in fix(e2e): harden NFS settings pin and Docs coverage #5503 is applied byapply_yaml_filesonly, which the disconnected handlers skip. The newdisconnected::allow_mirror_registry_egressapplies a targeted rule for the mirror's port (Helm + Operator pod selectors) and is skipped when the mirror is served on 443 (LOCAL_DISCONNECTED).Operator job: mirror the RHDH CI catalog, not the published one. Since rhdh-operator#3822 (chore(deps): update tombi-toml/setup-tombi digest to f7cb38e #3821 on release-2.1) a filtered oc-mirror run rejects the published catalog because its default
fastchannel is filtered out. That catalog also has no 2.x bundles (max 1.10.6), so even an unfiltered run installed the stale 1.10 operator. CI runs now usequay.io/rhdh/iib:<stream>-v<ocp>-x86_64, where<stream>is the samecommon::get_release_streamvalue passed to--filter-versions(2.2 on main, 2.1 on release-2.1), so the catalog and thefast-X.Ychannel it is filtered to cannot disagree and a stream without a published catalog fails at the pre-flightskopeo inspect.CONTAINERS_REGISTRIES_CONFredirects itsregistry.redhat.io/rhdhrelatedImages toquay.io/rhdhfor theprepare-restricted-environment.shcall only. CI runs mirror the newest CI bundle of the branch's stream: the hub image comes from the bundle (RELATED_IMAGE_backstage), so pinnedTAG_NAME,IMAGE_*andCHART_VERSIONsettings do not apply to them. A GA verification run (registry.redhat.iowith a releasedx.y.zchart) keeps the published Red Hat catalog, with--filter-versions '*': filtering it tofast-X.Ydrops the catalog's defaultfastchannel, which oc-mirror rejects.Smoke setup for the 2.x New Frontend System hub. The homepage-only plugin set predates the NFS migration. On 2.x the sign-in page (app-auth), sidebar (app-defaults), SCM integrations (app-integrations) and guest auth provider are dynamic plugins, so guest sign-in could not render. Both install methods now mirror that minimal digest-pinned set, plus the NFS
app.extensionsconfig and an explicitsignInPage: guest.Helm registry auth secret name. The chart mounts
<fullname>-dynamic-plugins-registry-authwithoptional: true, where fullname is<release>-<nameOverride>. The job created<release>-dynamic-plugins-registry-auth, soinstall-dynamic-pluginshad noauth.jsonand the mirror pull failed with "authentication required". The secret now uses the chart's name and the values file pinsnameOverride: developer-hub.Second round: what the first re-run exposed
The fixes above got both jobs past mirroring, plugin install and the operator catalog. Two more problems surfaced, both in pre-existing helpers, plus one upstream bug:
Node drains during the deploy (Helm).
disconnected::wait_machineconfigpool_updatedusedoc wait --for=condition=Updated=True, which returns while the MCO has not rendered the IDMS/ITMS config yet. The workers were drained a few seconds later, evicting the hub and PostgreSQL pods mid-startup. It now snapshots each pool's rendered config before the change (retrying failed reads) and waits for the MCO to pick it up before waiting for convergence.The Helm recovery restart started two backends.
ensure_helm_hub_after_postgresranoc rollout restart, keeping the old pod serving while a second backend booted against a freshly initialised database. They raced on the knex migration lock and the losing backend stayed up with readiness 503 forever. The bounce is now scale-to-zero then scale-to-one, the wait gives a pod that is still extracting plugins time to start its backend first, and the bounce refuses to scale up while an old pod remains.Operator: OLM v1 installer RBAC (upstream). OLM installs the bundle with
rhdh-operator-installer, which cannot grant RBAC it does not hold. The installer ClusterRole rendered byprepare-restricted-environment.shlags behind the bundle (pods,apiservers/ingresses,devhubplugincatalogs), so the operator ClusterRole creation is refused and the operator runs with no permissions. Tracked in RHDHBUGS-4056, fixed in fix: grant the bundle RBAC to the OLM v1 installer ServiceAccount rhdh-operator#3923 (backport chore(e2e): topology flaky test fix #3927). feat(e2e): add orchestrator RBAC tests and deployment fixes #3923 is merged: the job downloadsprepare-restricted-environment.shfrom rhdh-operatormainat runtime, so this branch installs with the fixed installer role (the temporary verification workaround has been removed). Our wait now requiresClusterExtension Installed=Trueas well as the CRD, so a half-applied release fails fast with the real error instead of a misleading "deployment not created".deploy_rhdh_operatoralso returns as soon as rendering or applying the CR fails.Scope
The disconnected jobs exist only on
mainandrelease-2.1, and both now target 2.x hubs: the smoke plugin set is the minimal New Frontend System set and the operator stream follows the branch. Verification of 1.x releases is not covered by these jobs.Which issue(s) does this PR fix
Jira: https://redhat.atlassian.net/browse/RHDHBUGS-4056 (installer RBAC, fixed in redhat-developer/rhdh-operator#3923)
Failing job: https://prow.ci.openshift.org/view/gs/test-platform-results-public/logs/periodic-ci-redhat-developer-rhdh-main-e2e-ocp-disconnected-helm-nightly/2112001170120642560
Evidence
dial tcp 10.0.x.x:5000; the Operator job fails earlier onthe default channel fast was filtered out.quay.io/rhdh/rhdh-hub-rhel10:next), chart 2.1-169-CI with the new values:/renders "Select a sign-in method" with the Guest Enter button,Welcome back!.Testing
shellcheck --severity=warningover all tracked.shfiles: clean.prettier --checkover every changed.sh/.yaml: clean.helm templatewith the rendered values: chart app-config deep-merges to the expectedapp.extensions+auth.providers.guest+signInPage.:nextand:2.1catalogs (5 digest-pinned refs each).oc apply --dry-run=server.registry.redhat.io/rhdhandregistry.stage.redhat.io/rhdhrelatedImage digests resolve through theregistries.confredirect (and fail without it).quay.io/rhdh/iib:2.2-v4.21-x86_64hasdefaultChannel: fast-2.2and channelfast-2.2(same digest asnext-v4.21);2.1-v4.21mirrors that forfast-2.1.oc(snapshot retry, failed reads ignored, render detected, nothing-to-roll, degraded pool, scale 0/1, no restart when Available, no scale-up while a pod remains, emptypackage:rejected).deploy_rhdh_operatorreturns immediately on a failedoc applyinstead of polling for five minutes.What the checks would miss: an oc-mirror m2m regression specific to the bastion mirror.
/test e2e-ocp-disconnected-{helm,operator}-nightlycovers it.Merge Danger
Door: two-way (revert restores the previous jobs).
Blast radius: CI-only, disconnected nightlies.
How to revert
git revert <commit>. The helpers are used only by the disconnected jobs.PR acceptance criteria
.ci/pipelines)/test e2e-ocp-disconnected-helm-nightly,/test e2e-ocp-disconnected-operator-nightly)