feat(release): submit the operator bundle to both catalogs on every release (#656) - #660
Conversation
…elease (#656) Every release now opens its own bundle pull request against k8s-operatorhub/community-operators and community-operators-prod. Until now that was two hand-made submissions per release, and both catalogs sat at 0.9.59 while the product reached 0.14.1, so 0.14.0 was never submitted. spec.replaces leaves the committed bundle. Its only correct value names the version immediately preceding ours in the catalog being submitted to, which is external state this repo cannot derive, so CATALOG_REPLACES made the bundle correct only by coincidence and stale by default. The value is derived from the catalog at submission time and injected into the bundle copy; the absence is asserted so it cannot be reintroduced. Both upstream gates reject an unlinked graph and neither accepts olm.skipRange as a substitute: opm index add --mode replaces prunes the previous bundle on the hub, and opm validate reports "multiple channel heads found in graph" on the FBC side. Detection of a pending submission is path-based, because the hub rewrites submission titles on every push.
cevheri
added
enhancement
There was a problem hiding this comment.
🟡 Changes recommended
Release concurrency can discard queued releases, and same-version pull requests from other branches can produce duplicate submissions.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Automates per-release OLM bundle submissions to both community operator catalogs while deriving catalog-specific update-graph predecessors.
Changes:
- Adds the catalog submission workflow and decision helper.
- Moves
spec.replacesinjection to submission time. - Updates channel inventory, documentation, and tests.
File summaries
| File | Description |
|---|---|
.github/workflows/operator-release.yml |
Adds catalog submission jobs and release serialization. |
scripts/operator-catalog-submission.mjs |
Implements submission decisions and bundle patching. |
scripts/distribution-check.mjs |
Adds the channel switch and automated-PR status. |
distribution/channels.yaml |
Enables automated operator catalog submissions. |
operator/Makefile |
Removes repository-owned replaces stamping. |
operator/config/manifests/bases/libredb-studio-operator.clusterserviceversion.yaml |
Removes the base placeholder. |
operator/bundle/manifests/libredb-studio-operator.clusterserviceversion.yaml |
Removes the committed predecessor pointer. |
docs/DISTRIBUTION.md |
Documents automation and update-graph behavior. |
docs/CHANNELS.md |
Marks the channel as an automated PR. |
tests/unit/operator-catalog-submission.test.ts |
Tests decision logic and CLI behavior. |
tests/unit/operator-bundle-update-graph.test.ts |
Asserts that committed bundles omit replaces. |
tests/unit/distribution-check.test.ts |
Tests switch and summary behavior. |
Review details
- Files reviewed: 12/12 changed files
- Comments generated: 4
- Review effort level: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| # submission's "an older submission is still open" guard, which reads the | ||
| # catalog's default branch and so cannot see a submission opened seconds | ||
| # earlier by the other run. | ||
| group: operator-release |
| const blockers = new Map(); | ||
| for (const submission of openSubmissions) { | ||
| for (const found of submission.versions) { | ||
| if (found !== version && !blockers.has(found)) { |
| 401|403) | ||
| echo "::error::${IMAGE}:${VERSION} is not anonymously pullable (HTTP ${CODE}). The catalog CI pulls without credentials - make the GHCR package public: github.com/orgs/libredb/packages" | ||
| exit 1 ;; |
| ### Manual steps still open | ||
|
|
||
| - **Operator first listings — done.** The controller image was the first half: | ||
| - **Operator first listings and per-release submissions: done.** |
…urrency Three defects from review, all in the submission path. A global concurrency group would have dropped a release outright: GitHub keeps one pending run per group, so a third release cancels the second while the first runs, whatever cancel-in-progress says, and a cancelled operator-release means no image and no submission for that version. Per-ref is restored. The race it was meant to serialize is loud instead: two submissions naming the same predecessor fail the second merge upstream with "multiple channel heads". The same-version exemption opened the duplicate it exists to prevent. create-pull-request only ever updates its own branch, so an open pull request for our version on any other branch or fork is a second submission. Only the managed branch on our own fork is exempt now, which needs the pull request's head and so a second read per candidate. The forks moved to the libredb organization by transfer, which is what keeps parent pointing at the upstream, and the matrix wiring is now asserted offline: each fork paired with the fork of its own upstream, org-owned, release_config a boolean on the FBC catalog only.
|
All four addressed, two of them real defects. Thanks. Concurrency. Right, and worse than the race it was guarding. GitHub keeps one pending run per group, so a third release cancels the second while the first runs, Same-version exemption. Correct, and it opened the duplicate it exists to prevent. Image visibility. The workflow was right and the description was wrong. A private GHCR package breaks the channel silently, so it stays a hard failure; the docs now list the three conditions that fail rather than skip: image not anonymously pullable, fork parent not the upstream, GitHub API read failed. Stale status text. Fixed. Both catalogs now carry 0.14.1 (k8s-operatorhub/community-operators#9219 and community-operators-prod#11106), and the manual One note on the failing check: |
|
There was a problem hiding this comment.
🟡 Changes recommended
The fork comparison is malformed, missing parent paths can silently skip submissions, and releases remain vulnerable to the documented concurrency race.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
- Files reviewed: 12/12 changed files
- Comments generated: 3
- Review effort level: Balanced
| # Keyed on the ref on purpose, even though a global group would serialize | ||
| # the submission's "an open submission for another version" guard, which | ||
| # reads the catalog's default branch and so cannot see a pull request opened | ||
| # seconds earlier by another run. |
| # catalog's default branch and fails loudly if it cannot, so a fork | ||
| # whose main carries commits the upstream lacks has to stop us here | ||
| # rather than upstream. | ||
| STATUS=$(gh api "repos/${UPSTREAM}/compare/main...${FORK%%/*}:${FORK##*/}:main" --jq .status) |
| if (error.code === "ENOENT") { | ||
| return null; |
|
Second round: one real defect, one claim that does not hold, one that is about this description rather than the code. Thanks for the ENOENT one, it is a genuine hole.
The compare ref is not malformed. Measured against both real repositories just now: The compare endpoint accepts Concurrency: the code is deliberate and this description was stale. You are right that |
Closes #656.
Every release now opens its own bundle pull request against both community operator catalogs.
Until today that was two hand-made submissions per release, and the cost was measurable rather than theoretical: both catalogs sat at 0.9.59 while the product reached 0.14.1, so 0.14.0 was never submitted at all.
What lands
submit-catalogsin.github/workflows/operator-release.yml, a matrix over the two catalogs withneeds: build-and-push, because the catalog pipelines pull the controller image onto a real cluster and it has to exist and be public first.scripts/operator-catalog-submission.mjs: every decision as a pure function, with the CLI as a thin shell. 50 tests.operatorhub-communityjoinsSWITCHABLE_CHANNEL_IDS, so the submission is switched fromdistribution/channels.yamllike every other optional channel.spec.replacesis removed from the committed bundle, and its absence is now asserted.The design decision, and why it reversed
The pointer's only correct value names the version immediately preceding ours in the catalog being submitted to.
That is external state: it lags our releases by however many submissions are unmerged, and the two catalogs disagree with each other.
CATALOG_REPLACESinoperator/Makefiledeclared it, which made the committed bundle correct only by coincidence and stale by default - a cost paid on every release, and exactly the fourth hand-maintained version string the operator docs say must not exist.So the value is derived at submission time from the catalog itself and injected into the bundle copy, and nothing in the repo declares it.
The consequence looks like an omission and is not: the committed bundle on its own does not pass
opm index add --mode replaces.tests/unit/operator-bundle-update-graph.test.tsasserts that absence rather than documenting it, because the obvious "fix" is to add a declared value back.What was measured, not assumed
Two upstream gates reject an unlinked graph, and neither accepts
olm.skipRangeas a substitute:opm index add --mode replacesadd prunes bundle libredb-studio-operator.v0.9.59 ... skips/replaces []opm validatemultiple channel heads found in graph: v0.14.1, v0.9.59The first was reproduced locally against the same command with a local registry and two bundle images: replaces-mode with a skipRange alone exits 1, with
spec.replacesexits 0, and--mode semverexits 0.The second is from community-operators-prod#11106's pipeline log.
Note the trap in the pair:
check_dangling_bundlesin operatorcert seeds its graph from_resolve_skip_rangeand so passes on a skipRange alone, and it is decorated@skip_fbcso it never runs for the FBC catalog. A green static check is not evidence that a submission will build.Three more findings that shaped the code:
operator [N] [CI] libredb-studio-operator (0.9.59). The job reads each candidate PR's changed files instead.gh repo sync --sourcecannot be trusted. It callsPOST /merge-upstreamfirst, and that endpoint takes neither a source nor a force parameter, so a fork pointing at the wrong parent would sync from the wrong repository and still exit 0. The job asserts the parent, callsmerge-upstreamdirectly, then asserts the fork isidenticalorbehind.ci.yamlreviewers on the base branch; nothing reads the fork owner or the commit author.OPERATOR_CATALOG_TOKENis a classic PAT on a listed account, andsignoff: truewrites the trailer from the committer input, so committer and author are both set to the same real identity.An older unmerged submission is a hard stop
The predecessor comes from the catalog's
main, so a submission that is open but unmerged is invisible to it.Submitting past a pending version points the graph over it, and once both merge the pending bundle is dangling.
That is what lost VictoriaMetrics 0.48.2, and their record is why it is worth a hard stop rather than a warning:
victoriametrics-botlanded 45 of 50 submissions on operatorhub.io, and four of the five that did not land are versions the catalog has never carried.Skips are notices, never failures
Every "nothing to do" branch exits 0 with a
::notice::naming the condition: channel switched off, token absent, image not anonymously pullable, operator not listed yet (a first listing stays manual), version already listed, older submission still open.Two things it refuses rather than guesses: a GitHub API read that fails stops the job instead of defaulting to "submit", and a fork with the wrong parent stops it before anything is mutated.
One honesty fix came out of this:
docs/CHANNELS.mdlabelled everything that is notci_publishas "Manual". The row now reads "Automated PR", because we open the pull request and somebody else merges it.Verification
bun run format,lint(0 errors),typecheck,chart:check,channels:showcase:check,readme:check,security:check,distribution:matrix --check,knip: clean.docs/superpowers/drafts trip the backlog citation guard.make -C operator bundleregenerates with no diff beyondcreatedAt, so the required bundle-freshness job stays green.opm index add --mode replacesto confirm it fails, which is the intended state.Review
Two workflow fan-outs ran over this before it was opened: a recon pass on the upstream assumptions and a six-lens adversarial review of the diff, each finding verified by a second agent against the real code.
The review confirmed 44 defects and every one is fixed here. The six that mattered most:
blockingSubmissionsfiltered only on "not our version", so any open PR editingoperators/libredb-studio-operator/0.9.59/would have stopped every release with a green run. A version the catalog already carries is no longer treated as pending.readOperatorEntriesswallowed every error, so a changed upstream layout or a wrong path read as "first submission is manual" and exited 0. Only a missing path means that now; anything else throws.curl -sfkilled the step at exit 22 before the "make it public" message could run. It branches on the status code now, and retries transient failures.compareVersionswas never compared at patch level, thewithReplacesanchor was never run against the real 380-line CSV, and the threespec.replacesabsence guards had no positive control, so a broken helper would have satisfied all of them.submittable, and the submission job skips a prerelease instead of failing on it.Also fixed: the workflow is serialized so two releases cannot race the pending-submission guard, the staged bundle is asserted to be the bundle for this version, a skip that means a release did not reach a catalog is a
::warning::rather than a::notice::, the skip names the blocking pull request number, the token is no longer handed to steps that only do unauthenticated reads, and the docs no longer contradict the code aboutskipRange.