Skip to content

Build Images fails on main: Quay returns 401 for the apps-pack-* repositories #85

Closed

Description

@jbouder

Summary

Since #84 moved image builds onto the org reusable workflow (nebari-dev/.github pack-build-image.yaml), every push to main fails the Build Images workflow. The workflow pushes to GHCR and Quay in one step. GHCR succeeds, but Quay rejects all five images with 401 Unauthorized:

  • quay.io/nebari/apps-pack-operator
  • quay.io/nebari/apps-pack-api
  • quay.io/nebari/apps-pack-ui
  • quay.io/nebari/apps-pack-mcp
  • quay.io/nebari/apps-pack-proxy

The repositories don't exist on Quay, and the org's Quay robot account can't create them on push. First failing run: https://github.com/nebari-dev/apps-pack/actions/runs/39264549201 (aca8ce6).

Impact

  • ghcr.io/nebari-dev/apps-pack/<image>:sha-<short> images are still published, so deployments that pull from GHCR work.
  • Signing runs after the push, so images built since ci: build, sign and release through the org reusable workflows; scan images #84 are unsigned.
  • The Build Images check stays red on main, and a chart-bump release through pack-release.yaml will hit the same wall when it mirrors to Quay.

Fix

  1. A Quay org admin for nebari creates the five repositories above.
  2. They give the org robot account used by pack-build-image.yaml write access to each one.
  3. Re-run the failed Build Images workflow on main and confirm the images are signed on both registries.

Alternative: if Quay isn't wanted for this pack, set the reusable workflow's input to skip Quay, if it has one, or revert #84's build-image.yaml change.

Activity

  1. jbouder commented on Oct 19, 2026

    @jbouder
    ContributorAuthor

    A lead for the fix: pack-release.yaml (at the pinned a6c757c) says nebari-dev/helm-repository's release-helm-charts.yml runs an ensure-quay-repos step with a QUAY_API_TOKEN scoped to that repo. pack-build-image.yaml has no equivalent for image repositories, so they have to be created by hand, or the same ensure step has to be added to the image workflow upstream in nebari-dev/.github.

  2. jbouder commented on Oct 22, 2026

    @jbouder
    ContributorAuthor

    Still failing on 8fbc0dd (#168): https://github.com/nebari-dev/apps-pack/actions/runs/39708219707. All five publish jobs get the same Quay 401, including quay.io/nebari/apps-pack-apps-authz, which wasn't in the original list. The body is updated to list five repositories.

    The authz image is still needed after #168. The sidecar no longer enforces groups, but Apps that can idle and image Apps still run it in passthrough mode. GHCR publishing still works: ghcr.io/nebari-dev/apps-pack/apps-*:sha-8fbc0dd exists for all five images on amd64 and arm64.

  3. jbouder commented on Oct 22, 2026

    @jbouder
    ContributorAuthor

    Since #169 (6338bd9) the images drop the doubled apps-, so the five Quay repositories to create are:

    • quay.io/nebari/apps-pack-operator
    • quay.io/nebari/apps-pack-authz
    • quay.io/nebari/apps-pack-api
    • quay.io/nebari/apps-pack-ui
    • quay.io/nebari/apps-pack-mcp

    These match the other packs on the shared workflow (llm-serving-pack-operator, …). The body is updated. The old apps-pack-apps-* names were never created and are no longer needed.

  4. changed the title [-]Build Images fails on main: Quay returns 401 for the apps-pack-apps-* repositories[/-] [+]Build Images fails on main: Quay returns 401 for the apps-pack-* repositories[/+] on Oct 22, 2026
  5. jbouder commented on Oct 22, 2026

    @jbouder
    ContributorAuthor

    The authz sidecar is being renamed to proxy in #174 (it no longer checks tokens or groups since #168), so the fifth repository to create is quay.io/nebari/apps-pack-proxy, not apps-pack-authz. The body is updated. The other four names are unchanged.

    Please create the repositories before the 0.2.0 release PR merges: the release builds and pins the images on the same push.

  6. viniciusdc commented on Oct 22, 2026

    @viniciusdc

    @jbouder repos have been created and bot account was given "write" access on quay over those new repos.

  7. jbouder commented on Oct 26, 2026

    @jbouder
    ContributorAuthor

    Works now as expected

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Fields

    Priority

    None yet

    Start date

    None yet

    Target date

    None yet

    Size

    None yet

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions