Remote not signed

[dhs-rec]

Version
From pulp status:

      "versions": {
        "deb": "3.10.0",
        "gem": "0.8.0",
        "npm": "0.9.0",
        "rpm": "3.38.4",
        "core": "3.114.2",
        "file": "3.114.2",
        "maven": "0.25.1",
        "ostree": "2.6.1",
        "python": "3.32.1",
        "ansible": "0.30.0",
        "certguard": "3.114.2",
        "container": "2.28.1",
        "hugging_face": "0.3.0"
      }

Describe the bug
I followed the quick start instructions using Docker to setup a test installation and then the "Repository Synchronization" chapter to create an on-demand remote for the upstream Debian APT repository, using these commands:

export NAME=debian
pulp deb remote create \
  --name ${NAME} \
  --url https://deb.debian.org/debian \
  --proxy-url ${PROXY_URL} \
  --policy on_demand \
  --component '' \
  --architecture all \
  --architecture amd64 \
  --architecture arm64 \
  --distribution trixie \
  --distribution trixie-updates \
  --distribution trixie-backports
pulp deb repository create --name=${NAME} --remote=${NAME}
pulp deb repository sync --name=${NAME}
pulp deb publication create --repository=${NAME}
pulp deb distribution create --name=${NAME} --base-path=${NAME} --repository=${NAME}

But when running apt-get update after replacing the upstream repository URL in /etc/apt/sources.list.d/debian.sources with the one found in the output of the last command, I got the following error messages:

E: The repository 'http://<hostname redacted>:8080/pulp/content/debian trixie Release' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.

To Reproduce

1. Replace URI in /etc/apt/sources.list.d/debian.sources with the one found in the output of the last command
2. Run apt-get update

Expected behavior
Package lists should have been updated from the Pulp repository with no error.

[quba42]

To sign publications, see the relevant publication docs: https://pulpproject.org/pulp_deb/docs/user/guides/publish/#metadata-signing

The alternative is to configure your APT clients to allow consumption of unsigned repositories (which is strongly discouraged by current versions of Debian or Ubuntu).

The other alternative is to create a verbatim type publication to mirror upstream metadata including the upstream signatures. However, this will not work for all situations and repos, since pulp_deb may not support synchronization of every aspect of an upstream repository, which may clash with use of the upstream metadata.

[dhs-rec]

@quba42, thanks for your help. I started with a fresh installation and repeated all the commands above, this time adding --type=verbatim when creating the publication. Now I get a different error from apt-get update:

E: Failed to fetch http://<hostname redacted>:8080/pulp/content/debian/dists/trixie-updates/main/i18n/Translation-en  404  Not Found [IP: 10.11.12.13 8080]
W: Some index files failed to download. They have been ignored, or old ones used instead.
N: Missing Signed-By in the sources.list(5) entry for 'http://<hostname redacted>:8080/pulp/content/debian'

Of course, there is a Signed-By: entry in the sources file:

Types: deb
#URIs: https://deb.debian.org/debian
URIs: http://<hostname redacted>:8080/pulp/content/debian
Suites: trixie trixie-updates
Components: main contrib non-free non-free-firmware
Signed-By: /usr/share/keyrings/debian-archive-keyring.gpg

Everything works fine again if I switch back to the original upstream URI.

[rayzilt]

Did you setup the singing part as quba42 mentioned? (https://pulpproject.org/pulp_deb/docs/user/guides/signing_service/)

Did you run the command pulp deb publication create --repository=${NAME} --signing-service=${NAME_OF_YOUR_SIGNING)
Eventually "Signed-By:" should point to your own key.

[dhs-rec]

@rayzilt, no, I didn't. Why should I do this for a remote repository that I just want to cache? That really doesn't make sense to me, since stuff in that repository should have been signed with upstream's key(s), right?

If Signed-By: pointed to the wrong key the error message would have been a different one.

[quba42]

Now I get a different error from apt-get update:

E: Failed to fetch http://<hostname redacted>:8080/pulp/content/debian/dists/trixie-updates/main/i18n/Translation-en  404  Not Found [IP: 10.11.12.13 8080]

I am going to focus on this message since it is an E: for "Error". The other two, W: for "Warning" and N: for "Notification", are less serious and could be ignored. And I am going to quote myself:

The other alternative is to create a verbatim type publication to mirror upstream metadata including the upstream signatures. However, this will not work for all situations and repos, since pulp_deb may not support synchronization of every aspect of an upstream repository, which may clash with use of the upstream metadata.

This is exactly the kind of situation I meant. The upstream repo contains translation files. pulp_deb does not sync translation files. APT considers the combination of upstream metadata (with translation files) and pulp published repo (without translation files) an error.

=> This means verbatim publications are broken for upstream repos containing translation files until pulp_deb adds support for syncing them.

The only remaining option then is to use a normal publication with a signing service to get a pulp published repo with signature. Or else to disable signature checking on the host.

All of this currently works as designed (though it would be nice if someone could extend the design to sync translation files so verbatim becomes usable for repos containing them). We have an issue for this: #408, but as you can see it has been open for years...

[dhs-rec]

@quba42, thanks for the explanation.

Wouldn't pull-through caching be the best solution here? I mean, this is exactly what Nexus or Artifactory do and it just works...