Skip to content

Make TestRegistryProviderList order-independent - #3956

Draft
grokspawn wants to merge 1 commit into
operator-framework:masterfrom
grokspawn:fix-registry-provider-test-order
Draft

grokspawn wants to merge 1 commit into
operator-framework:masterfrom
grokspawn:fix-registry-provider-test-order

Conversation

@grokspawn

Copy link
Copy Markdown
Contributor

Summary

TestRegistryProviderList/OneCatalog/ManyPackages/OneMissingBundle/Elided fails intermittently with the wrong package elided: the assertion expects [has-bundle] but gets [no-bundle], preceded by:

error getting bundle, eliding channel channel=alpha error="no bundle found"
eliding package: error converting to packagemanifest err="packagemanifest has no valid channels"

Root cause

RegistryProvider.refreshCache (pkg/package-server/provider/registry.go) spawns one goroutine per package returned by ListPackages; each goroutine calls client.GetPackage(...) and then, inside newPackageManifest, client.GetBundleForChannel(...).

The test programs its counterfeiter stubs with call-index-bound returns (GetPackageReturnsOnCall, GetBundleForChannelReturnsOnCall). Counterfeiter stubs do not inspect request arguments: ReturnsOnCall(i) hands response i to whichever goroutine happens to issue the i-th call. When the two package goroutines interleave between their GetPackage and GetBundleForChannel calls, the responses cross-wire: the goroutine processing the "has-bundle" package receives the index-0 error and elides it, while the goroutine processing "no-bundle" receives the index-1 good bundle and caches a manifest for it.

The provider is correct — a real registry answers per request; only the test assumed a serialization that refreshCache does not provide. The race detector does not flag this because counterfeiter fakes are internally mutex-synchronized: it is a semantic ordering dependency, not a data race.

The remaining index-bound stub sites in this file are single-package, multi-channel scenarios: newPackageManifest processes a package's channels serially in sorted order within one goroutine, so those are deterministic and left unchanged.

Fix

Replace the index-bound stubs with argument-aware stub functions keyed on the requested package and channel, preserving the scenario semantics (no-bundle's channel errors, has-bundle resolves) while being independent of goroutine scheduling.

Validation

  • go test -race -count=5 -run TestRegistryProviderList ./pkg/package-server/provider/ passes
  • full pkg/package-server/provider suite passes under -race

Failing run (downstream OpenShift CI, go test -race -count=1; the identical tree passed the day before and the package is unchanged between the runs):
https://prow.ci.openshift.org/view/gs/test-platform-results-public/pr-logs/pull/openshift_operator-framework-olm/1396/pull-ci-openshift-operator-framework-olm-main-unit-olm/2116233294835093504

refreshCache fetches each package and its per-channel bundles from its
own goroutine, but the OneCatalog/ManyPackages/OneMissingBundle/Elided
case programs its counterfeiter stubs with call-index-bound returns
(GetPackageReturnsOnCall / GetBundleForChannelReturnsOnCall).

Counterfeiter stubs do not inspect request arguments: ReturnsOnCall(i)
hands response i to whichever goroutine happens to issue the i-th
call. When the two package goroutines interleave between their
GetPackage and GetBundleForChannel calls, the responses cross-wire:
the goroutine processing the "has-bundle" package receives the
index-0 "no bundle found" error and elides it, while the goroutine
processing "no-bundle" receives the index-1 good bundle and caches a
manifest for it. The test then fails with the wrong package elided
(expected [has-bundle], actual [no-bundle]).

The provider is correct -- a real registry answers per request; only
the test assumed a serialization that refreshCache does not provide.
The race detector does not flag this because counterfeiter fakes are
internally mutex-synchronized: it is a semantic ordering dependency,
not a data race.

Replace the index-bound stubs with argument-aware stub functions that
key responses off the requested package and channel, keeping the
scenario semantics (no-bundle's channel errors, has-bundle resolves)
while being independent of goroutine scheduling.

Observed failing run (go test -race -count=1, downstream OpenShift CI):
https://prow.ci.openshift.org/view/gs/test-platform-results-public/pr-logs/pull/openshift_operator-framework-olm/1396/pull-ci-openshift-operator-framework-olm-main-unit-olm/2116233294835093504

Signed-off-by: grokspawn <jordan@nimblewidget.com>
@openshift-ci

openshift-ci Bot commented Nov 2, 2026

Copy link
Copy Markdown

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@openshift-ci openshift-ci Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Nov 2, 2026
@openshift-ci

openshift-ci Bot commented Nov 2, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign dtfranz for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@coderabbitai

coderabbitai Bot commented Nov 2, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant