Skip to content

fix(scorecard): simplify dora plugin config - #4715

Open
dzemanov wants to merge 1 commit into
redhat-developer:mainfrom
dzemanov:scorecard/simplify-dora-config
Open

fix(scorecard): simplify dora plugin config#4715
dzemanov wants to merge 1 commit into
redhat-developer:mainfrom
dzemanov:scorecard/simplify-dora-config

Conversation

@dzemanov

Copy link
Copy Markdown
Member

Hey, I just made a Pull Request!

For DORA YAML default configuration, all DORA metrics should use the same definition for deployments, incidents and pull requests.

Simplifies configuration from:

scorecard:
  plugins:
   # Optional DORA source-data retention and freshness controls
    dora:
      dataRetentionDays: 365
      staleAfterMs: 60000
      deploymentLookbackMs: 172800000
      incidentsLookbackMs: 5000
  metricProviders:
   dora:
     deploymentFrequency:
       options:
         productionEnvironments: ['production']
         collectors:
           incidents:
             id: jira:incidents
           deployments:
             id: github:deployments
             # id: github:deploymentWorkflowRuns
             # input:
             #   workflowName: Create Test Deployment on PR Merge
     medianLeadTimeForChanges:
       options:
         collectors:
           deployments:
             id: github:deployments
           deploymentPullRequests:
             id: github:deploymentPullRequests
     changeFailureRate:
       options:
         collectors:
           deployments:
             id: github:deployments
           incidents:
             id: jira:incidents
     meanTimeToRestore:
       options:
         collectors:
           incidents:
             id: jira:incidents
             input:
               issueType: CustomIncident

To single source of truth for incidents, deployments and deploymentPullRequests:

scorecard:
  plugins:
    dora:
      dataRetentionDays: 365
      staleAfterMs: 60000
      deploymentLookbackMs: 172800000
      incidentsLookbackMs: 5000
      productionEnvironments: ['production']
      collectors:
        incidents:
          id: jira:incidents
          input:
            issueType: CustomIncident
        deployments:
          id: github:deployments
        # input:
        #   workflowName: Create Test Deployment on PR Merge
        deploymentPullRequests:
          id: github:deploymentPullRequests

Using patch changeset as dora not released yet and version 0.0.0.
Also updated documentation and removed duplicate information.

Fixes

Fixes https://redhat.atlassian.net/browse/RHDHBUGS-3747

✔️ Checklist

  • A changeset describing the change and affected packages. (more info)
  • Added or Updated documentation
  • Tests for new functionality and regression tests for bug fixes
  • Screenshots attached (for UI changes)

Co-authored-by: Cursor <cursoragent@cursor.com>
Signed-off-by: Dominika Zemanovicova <dzemanov@redhat.com>
@rhdh-gh-app

rhdh-gh-app Bot commented Sep 11, 2026

Copy link
Copy Markdown

Changed Packages

Package Name Package Path Changeset Bump Current Version
@red-hat-developer-hub/backstage-plugin-scorecard-backend-module-dora workspaces/scorecard/plugins/scorecard-backend-module-dora patch v0.0.0

@rhdh-qodo-merge

Copy link
Copy Markdown

PR Summary by Qodo

Centralize DORA collector and environment configuration

🐞 Bug fix ⚙️ Configuration changes 📝 Documentation 🧪 Tests 🕐 40+ Minutes

Grey Divider

AI Description

• Centralizes shared DORA collectors and production environments under scorecard.plugins.dora.
• Keeps metric-provider configuration focused on schedules and thresholds.
• Consolidates collector guidance and verifies shared parsing across all DORA metrics.
Diagram

graph TD
  CFG["Shared DORA config"] -->|shared settings| PARSER["DORA config parser"] -->|selects| DEP["Deployment collector"] -->|feeds| DM["Deployment metrics"]
  PARSER -->|selects| INC["Incident collector"] -->|feeds| IM["Incident metrics"]
  PARSER -->|selects| PR["Pull request collector"] -->|feeds| DM
  INC -->|failure data| DM
Loading
High-Level Assessment

The shared plugin-level configuration is the appropriate approach because collector identities and production-environment rules are common inputs rather than metric-specific behavior. Retaining per-provider overrides would preserve duplication and allow inconsistent DORA results, while a separate configuration service would add unnecessary indirection for static Backstage configuration.

Files changed (16) +570 / -979

Bug fix (1) +61 / -66
DoraConfig.tsParse shared settings for every DORA metric +61/-66

Parse shared settings for every DORA metric

• Adds a shared parser for all collector roles and production environments under 'scorecard.plugins.dora'. Metric-specific parsers now project only their required settings, while sync validation consistently uses the shared path constant.

workspaces/scorecard/plugins/scorecard-backend-module-dora/src/metricProviders/DoraConfig.ts

Refactor (1) +4 / -3
constants.tsCentralize the DORA plugin config path +4/-3

Centralize the DORA plugin config path

• Introduces 'DORA_PLUGIN_CONFIG_PATH' for consistent configuration lookup and validation messages. Also clarifies the data-retention documentation comment.

workspaces/scorecard/plugins/scorecard-backend-module-dora/src/constants.ts

Tests (5) +121 / -225
DoraChangeFailureRateProvider.test.tsTest change-failure collectors from shared config +12/-17

Test change-failure collectors from shared config

• Updates custom deployment and incident collector test configuration to use 'scorecard.plugins.dora.collectors'.

workspaces/scorecard/plugins/scorecard-backend-module-dora/src/metricProviders/DoraChangeFailureRateProvider.test.ts

DoraConfig.test.tsVerify shared DORA configuration parsing +80/-158

Verify shared DORA configuration parsing

• Adds coverage for shared defaults, custom collectors, input hashes, and production environments. Updates metric projections and validation-message assertions to use the centralized config path.

workspaces/scorecard/plugins/scorecard-backend-module-dora/src/metricProviders/DoraConfig.test.ts

DoraDeploymentFrequencyProvider.test.tsTest deployment frequency with shared settings +8/-16

Test deployment frequency with shared settings

• Moves custom deployment collector and production-environment fixtures from provider options into the plugin-level DORA configuration.

workspaces/scorecard/plugins/scorecard-backend-module-dora/src/metricProviders/DoraDeploymentFrequencyProvider.test.ts

DoraMedianLeadTimeForChangesProvider.test.tsTest lead-time metrics with shared settings +15/-24

Test lead-time metrics with shared settings

• Updates deployment, pull-request, and production-environment fixtures to exercise the centralized DORA configuration.

workspaces/scorecard/plugins/scorecard-backend-module-dora/src/metricProviders/DoraMedianLeadTimeForChangesProvider.test.ts

DoraMedianTimeToRestoreProvider.test.tsTest restoration metrics with shared incidents +6/-10

Test restoration metrics with shared incidents

• Moves the custom incident collector fixture to 'scorecard.plugins.dora.collectors.incidents'.

workspaces/scorecard/plugins/scorecard-backend-module-dora/src/metricProviders/DoraMedianTimeToRestoreProvider.test.ts

Documentation (7) +346 / -588
dora-simplified-collectors-config.mdDeclare the shared DORA configuration patch +5/-0

Declare the shared DORA configuration patch

• Adds a patch changeset documenting the move of collector and production-environment settings into the shared DORA plugin configuration.

workspaces/scorecard/.changeset/dora-simplified-collectors-config.md

README.mdDocument centralized DORA setup +89/-56

Document centralized DORA setup

• Adds shared app configuration, entity annotation requirements, collector behavior, and data-identity guidance. Reorganizes metric and collector content to eliminate repeated provider-specific examples.

workspaces/scorecard/plugins/scorecard-backend-module-dora/README.md

dora-collectors.mdAdd centralized DORA collector reference +238/-0

Add centralized DORA collector reference

• Documents default and custom collector configuration, required inputs and outputs, entity annotations, and data-identity refresh behavior for every DORA collector role.

workspaces/scorecard/plugins/scorecard-backend-module-dora/docs/dora-collectors.md

change-failure-rate.mdReference shared collectors for change failure rate +4/-170

Reference shared collectors for change failure rate

• Removes duplicated per-provider options and collector contracts. Retains the metric-specific dependency on deployment and incident collectors and links to centralized guidance.

workspaces/scorecard/plugins/scorecard-backend-module-dora/docs/metrics/change-failure-rate.md

deployment-frequency.mdReference the shared deployment collector +3/-114

Reference the shared deployment collector

• Replaces repeated deployment collector configuration and contract details with a concise dependency list and links to centralized documentation.

workspaces/scorecard/plugins/scorecard-backend-module-dora/docs/metrics/deployment-frequency.md

median-lead-time-for-changes.mdReference shared lead-time collectors +4/-163

Reference shared lead-time collectors

• Removes duplicated provider options and collector instructions in favor of centralized app configuration and collector documentation.

workspaces/scorecard/plugins/scorecard-backend-module-dora/docs/metrics/median-lead-time-for-changes.md

median-time-to-restore.mdReference the shared incident collector +3/-85

Reference the shared incident collector

• Condenses the metric's collector section to its incident-data dependency and directs users to the centralized configuration and contract reference.

workspaces/scorecard/plugins/scorecard-backend-module-dora/docs/metrics/median-time-to-restore.md

Other (2) +38 / -97
app-config.yamlUse one DORA collector configuration +23/-45

Use one DORA collector configuration

• Defines production environments and all three collector roles under 'scorecard.plugins.dora'. Removes repeated collector options from individual metric providers while preserving their schedules.

workspaces/scorecard/app-config.yaml

config.d.tsMove shared options into the DORA plugin schema +15/-52

Move shared options into the DORA plugin schema

• Adds production environments and deployment, incident, and pull-request collectors to 'scorecard.plugins.dora'. Removes these options from each metric provider's configuration type.

workspaces/scorecard/plugins/scorecard-backend-module-dora/config.d.ts

@fullsend-ai-review

fullsend-ai-review Bot commented Sep 11, 2026

Copy link
Copy Markdown

🤖 Review · ❌ Terminated · Started 4:54 PM UTC · Ended 5:16 PM UTC

Commit: dc4b8e6 · View workflow run →

@sonarqubecloud

Copy link
Copy Markdown

@rhdh-qodo-merge

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (3) 📘 Rule violations (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

Grey Divider


Remediation recommended

1. DORA collector links lead nowhere 🐞 Bug ⚙ Maintainability
Description
README.md links the new DORA collectors section to ../../scorecard-backend/docs/collectors.md
and ./docs/collectors.md, even though the valid targets are
../scorecard-backend/docs/collectors.md and ./docs/dora-collectors.md. From the DORA module
root, the first path escapes plugins and the second references the wrong filename, so readers
cannot reach either the shared collector guide or the module-specific DORA guide.
Code

workspaces/scorecard/plugins/scorecard-backend-module-dora/README.md[127]

+DORA plugin uses [**collectors**](../../scorecard-backend/docs/collectors.md) to gather necessary data for metrics calculation from various sources. See [collectors.md](./docs/collectors.md) for more information.
Relevance

●●● Strong

Broken documentation links are deterministic maintainability defects; recent DORA README fixes were
accepted.

PR-#4319

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The README is located in the DORA module root, and its existing introduction already demonstrates
the valid relative path to the shared guide in the sibling scorecard-backend module. The newly
added local guide is named docs/dora-collectors.md, confirming that both links in the new overview
target nonexistent paths; past PR 3526 also documents the same broken-relative-link failure pattern
in Scorecard provider documentation.

workspaces/scorecard/plugins/scorecard-backend-module-dora/README.md[5-5]
workspaces/scorecard/plugins/scorecard-backend-module-dora/docs/dora-collectors.md[1-3]
workspaces/scorecard/plugins/scorecard-backend-module-dora/README.md[125-127]
workspaces/scorecard/plugins/scorecard-backend/docs/collectors.md[1-3]
PR-#3526

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The new DORA collectors paragraph contains two links that do not resolve from the module README: the shared Scorecard collector link has one parent directory too many, and the local guide is named `dora-collectors.md`, not `collectors.md`.

## Fix Focus Areas
- workspaces/scorecard/plugins/scorecard-backend-module-dora/README.md[125-127]

## Recommended Fix
Change the shared collector link to `../scorecard-backend/docs/collectors.md` and the module-specific link to `./docs/dora-collectors.md`.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


2. Incident setup selects an unregistered collector 🐞 Bug ≡ Correctness
Description
The default incident configuration in dora-collectors.md sets the collector ID to
github:incidents even though the documented, registered, and DORA-default collector is
jira:incidents. Copying the snippet overrides the parser’s Jira default with an ID that has no
GitHub implementation, so an incident-based metric reaches an unknown-collector failure instead of
collecting data.
Code

workspaces/scorecard/plugins/scorecard-backend-module-dora/docs/dora-collectors.md[R151-154]

+      collectors:
+        incidents:
+          id: github:incidents
+```
Relevance

●●● Strong

The default snippet contradicts the documented registered collector and would cause runtime failure;
correctness fixes are accepted.

PR-#4319

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The guide labels the snippet as the default configuration and identifies Jira as the available
default incident collector, while both the DORA default constant and the Jira module registration
use jira:incidents; no GitHub collector with the documented github:incidents ID exists in the
repository.

workspaces/scorecard/plugins/scorecard-backend-module-dora/docs/dora-collectors.md[107-115]
workspaces/scorecard/plugins/scorecard-backend-module-dora/src/constants.ts[17-20]
workspaces/scorecard/plugins/scorecard-backend-module-jira/src/collectors/JiraIncidentsCollector.ts[51-51]
workspaces/scorecard/plugins/scorecard-backend-module-dora/docs/dora-collectors.md[145-154]
workspaces/scorecard/plugins/scorecard-backend-module-dora/src/constants.ts[17-21]
workspaces/scorecard/plugins/scorecard-backend-module-jira/src/collectors/JiraIncidentsCollector.ts[50-52]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The documented default incident configuration uses `github:incidents`, but incidents are provided by the Jira Scorecard module and the registered DORA default collector is `jira:incidents`.

## Fix Focus Areas
- workspaces/scorecard/plugins/scorecard-backend-module-dora/docs/dora-collectors.md[145-154]

## Recommended Fix
Replace `id: github:incidents` with `id: jira:incidents` in the default incident configuration snippet, keeping it consistent with the registered collector and the other examples.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


3. Lead-time setup omits pull request collection 🐞 Bug ≡ Correctness
Description
The Median Lead Time for Changes guide lists incidents rather than deploymentPullRequests as its
second required collector. A custom setup following this list can omit pull-request data even though
the provider registers and synchronizes deployment pull requests to calculate lead times between
deployments.
Code

workspaces/scorecard/plugins/scorecard-backend-module-dora/docs/metrics/median-lead-time-for-changes.md[R35-38]

+Median Lead Time for Changes requires following collectors to gather necessary data for calculation:

-### Use custom collectors
+- `deployments`: for deployment data
+- `incidents`: for incidents data
Relevance

●●● Strong

The metric guide names the wrong collector; correcting required documentation is a direct,
deterministic fix aligned with PR intent.

PR-#4319

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The provider declares only deployments and deployment pull requests as collector dependencies and
passes the latter into pull-request synchronization for each deployment interval; incidents are not
part of this metric’s configuration or calculation path.

workspaces/scorecard/plugins/scorecard-backend-module-dora/src/metricProviders/DoraConfig.ts[51-55]
workspaces/scorecard/plugins/scorecard-backend-module-dora/src/metricProviders/DoraMedianLeadTimeForChangesProvider.ts[95-98]
workspaces/scorecard/plugins/scorecard-backend-module-dora/src/metricProviders/DoraMedianLeadTimeForChangesProvider.ts[153-163]
workspaces/scorecard/plugins/scorecard-backend-module-dora/docs/metrics/median-lead-time-for-changes.md[33-40]
workspaces/scorecard/plugins/scorecard-backend-module-dora/src/metricProviders/DoraMedianLeadTimeForChangesProvider.ts[95-99]
workspaces/scorecard/plugins/scorecard-backend-module-dora/src/metricProviders/DoraMedianLeadTimeForChangesProvider.ts[150-163]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The Median Lead Time for Changes documentation lists the incidents collector as required, although the metric requires deployments and deployment pull requests to calculate lead times between deployments.

## Fix Focus Areas
- workspaces/scorecard/plugins/scorecard-backend-module-dora/docs/metrics/median-lead-time-for-changes.md[35-40]

## Recommended Fix
Replace the `incidents` bullet with `deploymentPullRequests: for pull request data linked to deployments`.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Context sources
⚠️ Tickets: not configured — ticket URL found in PR but could not be fetched — check ticket provider credentials
✅ Compliance rules (platform): 11 rules
✅ Cross-repo context — repo relationships
Review mode: 🧠 Deep: This changes shared runtime configuration parsing across multiple DORA metric paths, with substantial independent logic and test/config/documentation edits that could introduce several subtle behavior or compatibility defects.

Grey Divider

Tip of the day
💡 Did you know, you can switch off images and animations for a plain-text comment

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

@rhdh-qodo-merge rhdh-qodo-merge Bot added documentation Improvements or additions to documentation Tests Bug fix labels Sep 11, 2026
@rhdh-qodo-merge

Copy link
Copy Markdown

Important

The /generate_labels command by Qodo is sunsetting on the 1st of October 2026 and will no longer be available. We recommend switching to the latest Qodo review capabilities. Learn more

@codecov

codecov Bot commented Sep 11, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 62.72%. Comparing base (befccc2) to head (dc4b8e6).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff            @@
##             main    #4715     +/-   ##
=========================================
  Coverage   62.72%   62.72%             
=========================================
  Files        2636     2636             
  Lines      105422   105426      +4     
  Branches    29522    29490     -32     
=========================================
+ Hits        66125    66129      +4     
- Misses      37503    38763   +1260     
+ Partials     1794      534   -1260     
Flag Coverage Δ *Carryforward flag
adoption-insights 84.77% <ø> (ø) Carriedforward from befccc2
ai-integrations 78.80% <ø> (ø) Carriedforward from befccc2
app-defaults 53.07% <ø> (ø) Carriedforward from befccc2
augment 46.67% <ø> (ø) Carriedforward from befccc2
boost 83.46% <ø> (ø) Carriedforward from befccc2
bulk-import 73.12% <ø> (ø) Carriedforward from befccc2
cost-management 13.35% <ø> (ø) Carriedforward from befccc2
dcm 73.47% <ø> (ø) Carriedforward from befccc2
e2e-adoption-insights 60.00% <ø> (ø) Carriedforward from befccc2
e2e-extensions 62.31% <ø> (ø) Carriedforward from befccc2
e2e-global-header 49.71% <ø> (ø) Carriedforward from befccc2
e2e-homepage 61.11% <ø> (ø) Carriedforward from befccc2
e2e-intelligent-assistant 46.09% <ø> (ø) Carriedforward from befccc2
e2e-orchestrator 49.52% <ø> (ø) Carriedforward from befccc2
e2e-orchestrator-plugin 49.51% <ø> (ø) Carriedforward from befccc2
e2e-quickstart 55.21% <ø> (ø) Carriedforward from befccc2
e2e-scorecard 50.05% <ø> (ø) Carriedforward from befccc2
e2e-theme 16.36% <ø> (ø) Carriedforward from befccc2
extensions 57.37% <ø> (ø) Carriedforward from befccc2
global-floating-action-button 71.18% <ø> (ø) Carriedforward from befccc2
global-header 67.88% <ø> (ø) Carriedforward from befccc2
homepage 48.39% <ø> (ø) Carriedforward from befccc2
install-dynamic-plugins 71.94% <ø> (ø) Carriedforward from befccc2
intelligent-assistant 77.26% <ø> (ø) Carriedforward from befccc2
konflux 91.98% <ø> (ø) Carriedforward from befccc2
lightspeed 69.02% <ø> (ø) Carriedforward from befccc2
mcp-integrations 84.46% <ø> (ø) Carriedforward from befccc2
orchestrator 72.02% <ø> (ø) Carriedforward from befccc2
quickstart 63.74% <ø> (ø) Carriedforward from befccc2
sandbox 79.56% <ø> (ø) Carriedforward from befccc2
scorecard 88.21% <100.00%> (+<0.01%) ⬆️
theme 87.91% <ø> (ø) Carriedforward from befccc2
translations 5.12% <ø> (ø) Carriedforward from befccc2
x2a 77.18% <ø> (ø) Carriedforward from befccc2

*This pull request uses carry forward flags. Click here to find out more.


Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update befccc2...dc4b8e6. Read the comment docs.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@fullsend-ai-review

Copy link
Copy Markdown

🤖 Finished Review · ❌ Failure (agent timed out after 20m0s without completing (timeout: 20m0s)) · Started 4:54 PM UTC · Completed 5:16 PM UTC

Commit: dc4b8e6 · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Bug fix documentation Improvements or additions to documentation Tests workspace/scorecard

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant