Skip to content

CMP-4571: Update coverage baseline after concurrent merges - #1365

Merged
yuumasato merged 1 commit into
ComplianceAsCode:masterfrom
guzalv:fix-coverage-baseline
Aug 21, 2026
Merged

CMP-4571: Update coverage baseline after concurrent merges#1365
yuumasato merged 1 commit into
ComplianceAsCode:masterfrom
guzalv:fix-coverage-baseline

Conversation

@guzalv

@guzalv guzalv commented Aug 20, 2026

Copy link
Copy Markdown

What

Update coverage-baseline.txt: cmd/manager coverage improved from 24.1% to 24.4%.

Why

The coverage gate (make test-unit) is failing on master and all open PRs because recent merges improved coverage beyond the recorded baseline. The gate rejects the build when the baseline is stale.

Changes

One-line change in coverage-baseline.txt, generated by make update-coverage-baseline.

@openshift-ci

openshift-ci Bot commented Aug 20, 2026

Copy link
Copy Markdown

Hi @guzalv. Thanks for your PR.

I'm waiting for a ComplianceAsCode member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work.

Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@guzalv
guzalv marked this pull request as ready for review August 20, 2026 09:03
@openshift-ci
openshift-ci Bot requested review from Vincent056 and rhmdnd August 20, 2026 09:03
@guzalv guzalv changed the title Update coverage baseline CMP-4571: Update coverage baseline after concurrent merges Aug 20, 2026
@openshift-ci-robot

Copy link
Copy Markdown
Collaborator

@guzalv: This pull request references CMP-4571 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "5.1.0" version, but no target version was set.

Details

In response to this:

What

Update coverage-baseline.txt: cmd/manager coverage improved from 24.1% to 24.4%.

Why

The coverage gate (make test-unit) is failing on master and all open PRs because recent merges improved coverage beyond the recorded baseline. The gate rejects the build when the baseline is stale.

Changes

One-line change in coverage-baseline.txt, generated by make update-coverage-baseline.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@taimurhafeez

Copy link
Copy Markdown
Collaborator

/ok-to-test

@github-actions

Copy link
Copy Markdown

🤖 To deploy this PR, run the following command:

make catalog-deploy CATALOG_IMG=ghcr.io/complianceascode/compliance-operator-catalog:1365-6300809a3badb0ecd0a2f8bb03712fcd4390994b

@yuumasato yuumasato added this to the 1.10.0 milestone Aug 20, 2026
@yuumasato

Copy link
Copy Markdown
Member

@guzalv please set up commit signing:
https://docs.github.com/en/authentication/managing-commit-signature-verification/signing-commits

cmd/manager coverage improved from 24.1% to 24.4% after recent
merges. Update the baseline so the coverage gate passes.
@guzalv
guzalv force-pushed the fix-coverage-baseline branch from 6300809 to a6db1b6 Compare August 20, 2026 13:34
@openshift-ci openshift-ci Bot removed the lgtm label Aug 20, 2026
@yuumasato

Copy link
Copy Markdown
Member

/ok-to-test

@github-actions

Copy link
Copy Markdown

🤖 To deploy this PR, run the following command:

make catalog-deploy CATALOG_IMG=ghcr.io/complianceascode/compliance-operator-catalog:1365-a6db1b6d604d8a5e16b1466ba848d694ba41564e

@yuumasato yuumasato left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@yuumasato

Copy link
Copy Markdown
Member

The e2e-aws-parallel failure is unrelated to this PR.

TestTimeoutDisabledWithZeroValue (added in #982) has a scan name collision with TestScanSettingBindingUsesDefaultScanSetting — both create ScanSettingBindings referencing the ocp4-cis Profile, causing the operator to create ComplianceScans with the same name in the same namespace. When the faster test finishes and its cleanup deletes the shared ocp4-cis scan, the slower test fails with "ocp4-cis" not found.

This is a pre-existing flaky test bug, not a regression from the coverage baseline change.

2026/08/20 15:23:13 Waiting for availability of ocp4-cis compliancescan
2026/08/20 15:23:13 All scans in ComplianceSuite have finished (test-timeout-disabled-with-zero-value-binding)
=== NAME  TestTimeoutDisabledWithZeroValue
    main_test.go:5919: failed to get scan ocp4-cis: compliancescans.compliance.openshift.io "ocp4-cis" not found
--- FAIL: TestTimeoutDisabledWithZeroValue (1835.09s)
FAIL
FAIL	github.com/ComplianceAsCode/compliance-operator/tests/e2e/parallel	2146.668s
FAIL
make: *** [Makefile:650: e2e-parallel] Error 1

@guzalv Would you be okay addressing this issue in this PR as well?
3096f16

@yuumasato

Copy link
Copy Markdown
Member

Or we could just retest and see if the race condition manifests again.
/test e2e-aws-parallel

@guzalv

guzalv commented Aug 21, 2026

Copy link
Copy Markdown
Author

The e2e-aws-parallel failure is unrelated to this PR.

TestTimeoutDisabledWithZeroValue (added in #982) has a scan name collision with TestScanSettingBindingUsesDefaultScanSetting — both create ScanSettingBindings referencing the ocp4-cis Profile, causing the operator to create ComplianceScans with the same name in the same namespace. When the faster test finishes and its cleanup deletes the shared ocp4-cis scan, the slower test fails with "ocp4-cis" not found.

This is a pre-existing flaky test bug, not a regression from the coverage baseline change.

2026/08/20 15:23:13 Waiting for availability of ocp4-cis compliancescan
2026/08/20 15:23:13 All scans in ComplianceSuite have finished (test-timeout-disabled-with-zero-value-binding)
=== NAME  TestTimeoutDisabledWithZeroValue
    main_test.go:5919: failed to get scan ocp4-cis: compliancescans.compliance.openshift.io "ocp4-cis" not found
--- FAIL: TestTimeoutDisabledWithZeroValue (1835.09s)
FAIL
FAIL	github.com/ComplianceAsCode/compliance-operator/tests/e2e/parallel	2146.668s
FAIL
make: *** [Makefile:650: e2e-parallel] Error 1

@guzalv Would you be okay addressing this issue in this PR as well? 3096f16

Let’s see how the retest goes. In general I prefer to create separate PRs when an unrelated issue pops up. But merging with test failures is not nice and fixing that separately would mean an extra ci loop + wait time here. So if re-run fails I’ll make a fix right here! Thanks for the context

@yuumasato

Copy link
Copy Markdown
Member

I agree with separating as well.
It is just that another PR ran into the same flaky issue as well, so not sure how frequent that is.

@yuumasato yuumasato left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@openshift-ci

openshift-ci Bot commented Aug 21, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: guzalv, taimurhafeez, yuumasato

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:
  • OWNERS [taimurhafeez,yuumasato]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@yuumasato
yuumasato merged commit 92fbe7a into ComplianceAsCode:master Aug 21, 2026
16 of 19 checks passed
@yuumasato

Copy link
Copy Markdown
Member

Okay, I proposed the fix for the test in #1367

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants