feat!(helm): Allow full customization of urls and ingress, move network settings under networking

[corneliusroemer]

feat!(deployment): group networking values under networking and derive Ingress hostnames from the public URLs

Feature requested by @florianzwagemaker who also did initial implementation in #6699. This PR builds on his work there.

⚠️ BREAKING CHANGES

Rename in your values files:

before (root)	after
public.backendUrl	networking.publicHosts.backendUrl
public.websiteUrl	networking.publicHosts.websiteUrl
public.keycloakUrl	networking.publicHosts.keycloakUrl
public.lapisUrlTemplate	networking.publicHosts.lapisUrlTemplate, append /%organism% path if not present
host	networking.host
subdomainSeparator	networking.subdomainSeparator
enforceHTTPS	networking.enforceHTTPS
traefikVersion	networking.traefikVersion

Why

• Configuring Loculus for anything other than the <service>-<host> subdomain layout was limited and partially broken.
• Networking-related values were scattered across the root of values.yaml — host,
  subdomainSeparator, enforceHTTPS, traefikVersion, public — with no indication that they
  belong together.
• Ingress hostnames were hardcoded to <prefix><subdomainSeparator><host> in five different
  templates. A deployment whose services do not live on sibling subdomains of one base domain (e.g.
  loculus.x.y.com + api.loculus.x.y.com + auth.loculus.x.y.com) could not be expressed at all.
• public.backendUrl / websiteUrl / keycloakUrl / lapisUrlTemplate changed only what
  the applications advertise, never what the Ingress routes. Setting them produced a deployment
  that hands out URLs no Ingress rule answers — with no error.
• ingest, ena-submission and autoapprove each re-derived the backend URL as
  https://backend<sep><host> rather than using the shared helper, so public.backendUrl never
  reached those configs.

What this changes

All networking configuration now lives in one block with an explicit three-layer model, documented
in the schema:

networking:
  host: myhost.com # (1) convention: base domain of the deployment
  subdomainSeparator: "-" # (1) convention: backend-myhost.com, lapis-myhost.com, ...
  publicHosts: {} # (2) override the public URL of an individual service
  ingressHosts: {} # (3) override the Ingress hostname of an individual service
  enforceHTTPS: true
  traefikVersion: 3

Inheritance runs in exactly one direction:

host + subdomainSeparator  →  publicHosts.<service>Url  →  ingressHosts.<service>
   (the convention)            (what apps advertise)        (what the cluster routes)

• Set nothing but networking.host and you get today's behaviour, unchanged.
• Set a publicHosts URL and both the advertised URL and that service's Ingress rule (and its
  TLS SAN) follow it.
• Set an ingressHosts entry only for the rare case where cluster routing must differ from the
  public URL — a CDN or external reverse proxy that terminates the public name. This changes routing
  only; advertised URLs are untouched.

Overrideable: website, backend, keycloak, lapis.

Further behaviour changes beyond the renames

1. Keycloak redirectUris are built from the website URL instead of https://<host> +
   http://<host>, so they are correct when the website is not at https://<host>. The plaintext
   http:// variant of the public host is now emitted only when networking.enforceHTTPS is
   false.
2. The www. website rule is now conditional. It is emitted only when the website is still on
   networking.host and that host does not already start with www.; the redirect-www-middleware
   is attached only when the rule exists. Previously www.<host> was emitted unconditionally, so
   host: www.example.org produced an unvalidatable www.www.example.org TLS SAN — and with a
   custom website hostname, a www. SAN with no DNS record would fail the ACME order and take the
   whole TLS secret, including the working host, down with it.

Verification (by Claude)

• No output change at default values. Rendering ingressroute, lapis-ingress,
  keycloak-config-map, loculus-website-config, docs-preview, ingest-config,
  autoapprove-config and ena-submission-config for both environment=server and
  environment=local is byte-identical to main, apart from the intended redirectUris change.
• Scenario matrix rendered and checked:
  • publicHosts only → website / backend / keycloak / lapis Ingress hostnames and TLS SANs all
    follow; www. rule correctly suppressed; Keycloak frontendUrl and redirectUris follow.
  • publicHosts + ingressHosts → Ingress routes the internal hostnames while the advertised URLs
    and redirectUris keep the public ones.
  • host: www.example.org → no www.www. rule or SAN.
  • path in a publicHosts URL → fails in server environment, renders fine in local.
  • runDevelopmentS3=false (external S3 endpoint) → no failure from the MinIO hostname derivation.
  • stale top-level host / subdomainSeparator, and removed ingressHosts.docs → rejected by the
    schema.
• values_e2e_and_dev.yaml renders the same service URLs as before on the path deploy.py --for-e2e
  actually builds.

Manual testing

• Loculus preview works
• Pathoplexus preview works (after required values.yaml migration)

🚀 Preview: https://helm-url-flex-simplified.loculus.org

[Copilot]

Pull request overview

This PR introduces a breaking Helm values re-organization by grouping all networking-related configuration under networking, and updates Helm templates to derive Ingress hostnames from the configured public URLs (with optional per-service overrides). This addresses previously inconsistent behavior where “public” URLs affected only advertised URLs but not actual Ingress routing.

Changes:

• Move host, subdomainSeparator, enforceHTTPS, traefikVersion, and public.* into networking.* and document the 3-layer resolution model in values.schema.json.
• Update Ingress and service config templates to use networking.publicHosts / networking.ingressHosts and shared URL helpers.
• Build Keycloak redirectUris from the website URL and make the www. website rule conditional.

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
kubernetes/loculus/values.yaml	Moves networking settings under networking in the default chart values.
kubernetes/loculus/values.schema.json	Defines the new networking block schema (public/ingress host layering) and removes old root keys.
kubernetes/loculus/values_e2e_and_dev.yaml	Migrates dev/e2e values to networking.host.
kubernetes/loculus/templates/lapis-ingress.yaml	Derives LAPIS ingress host via networking.ingressHosts or URL-derived hostname.
kubernetes/loculus/templates/keycloak-config-map.yaml	Derives Keycloak redirect URIs from the website URL and gates plaintext redirect URIs on enforceHTTPS.
kubernetes/loculus/templates/ingressroute.yaml	Derives website/backend/keycloak ingress hosts from URLs and makes the www. rule conditional.
kubernetes/loculus/templates/ingest-config.yaml	Switches backend URL derivation to the shared loculus.backendUrl helper.
kubernetes/loculus/templates/ena-submission-config.yaml	Switches backend URL derivation to the shared loculus.backendUrl helper.
kubernetes/loculus/templates/docs-preview.yaml	Moves docs preview hostname derivation to networking.*.
kubernetes/loculus/templates/autoapprove-config.yaml	Switches backend URL derivation to the shared loculus.backendUrl helper.
kubernetes/loculus/templates/_urls.tpl	Centralizes URL construction under networking.publicHosts and adds helpers for ingress hostname + routability assertion.
kubernetes/loculus/templates/_common-metadata.tpl	Uses the shared LAPIS URL template helper for runtime config generation.
docs/src/content/docs/for-administrators/setup-with-k3d-and-nginx.mdx	Updates admin setup docs to the new networking.publicHosts structure.
deploy.py	Updates deployment helper flags to set networking.host and networking.publicHosts.

Suppressed comments (2)

kubernetes/loculus/values.schema.json:1503

• The schema/docstring says networking.publicHosts.lapisUrlTemplate “must contain %organism% as a placeholder”, but the schema doesn’t enforce that. If a user sets a URL without %organism%, generateExternalLapisUrls will silently produce the same URL for every organism. Add a pattern that allows empty string or requires %organism% when set.

            "enableSubmissionNavigationItem": {
              "groups": ["website"],
              "type": "boolean",
              "default": true,
              "description": "Whether the website should show \"Submit\" link in the top navigation bar."
            },

kubernetes/loculus/values.schema.json:1510

• The networking.ingressHosts description says “Only services with a networking.publicHosts URL can be overridden here”, but the chart logic allows overriding ingress hosts even when using convention-derived defaults (i.e. without setting publicHosts). This sentence is misleading and should be removed or rephrased to match the actual behavior.

              "type": "boolean",
              "default": true,
              "description": "Whether to completely disable submission related pages. Setting this to false is useful when hosting Loculus for analysis-only purposes."
            }
          }

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[corneliusroemer]

Preview is broken - not sure if we can test preview before making changes to another repo that apply to all previews.

Full argo error:

Details

Failed sync attempt to 6c321a235c3087b6660e4a58562764e96b6c1f01: one or more objects failed to apply, reason: Ingress.networking.k8s.io "minio-ingress" is invalid: [spec.rules[0].host: Invalid value: "s3-%!s(<nil>)": a lowercase RFC 1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*'), spec.tls[0].hosts[0]: Invalid value: "s3-%!s(<nil>)": a lowercase RFC 1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*')],Ingress.networking.k8s.io "lapis-ingress" is invalid: [spec.rules[0].host: Invalid value: "lapis-%!s(<nil>)": a lowercase RFC 1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*'), spec.tls[0].hosts[0]: Invalid value: "lapis-%!s(<nil>)": a lowercase RFC 1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*')],Ingress.networking.k8s.io "lapis-redirect-ingress" is invalid: spec.rules[0].host: Invalid value: "lapis-%!s(<nil>)": a lowercase RFC 1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*'),Ingress.networking.k8s.io "loculus-backend-ingress" is invalid: [spec.rules[0].host: Invalid value: "backend-%!s(<nil>)": a lowercase RFC 1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*'), spec.tls[0].hosts[0]: Invalid value: "backend-%!s(<nil>)": a lowercase RFC 1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*')],Ingress.networking.k8s.io "loculus-docs-ingress" is invalid: [spec.rules[0].host: Invalid value: "docs-%!s(<nil>)": a lowercase RFC 1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*'), spec.tls[0].hosts[0]: Invalid value: "docs-%!s(<nil>)": a lowercase RFC 1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*')],Ingress.networking.k8s.io "loculus-keycloak-ingress" is invalid: [spec.rules[0].host: Invalid value: "authentication-%!s(<nil>)": a lowercase RFC 1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*'), spec.tls[0].hosts[0]: Invalid value: "authentication-%!s(<nil>)": a lowercase RFC 1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*')],Ingress.networking.k8s.io "loculus-website-ingress" is invalid: [spec.rules[0].host: Invalid value: "%!s(<nil>)": a lowercase RFC 1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*'), spec.tls[0].hosts[0]: Invalid value: "%!s(<nil>)": a lowercase RFC 1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*')] (retried 5 times).

Update: Hacked around this by hardcoding networking.host to the expected value for the preview deployment. Need to revert before merge. And post-merge need to edit the argo appset renaming .host -> networking.host

[florianzwagemaker]

Thanks @corneliusroemer for the substantial effort in this!
I've tested the behaviors such as the URL derivation and the publicHosts/ingressHosts precedence and everything appears to work as intended.

This implementation looks to be a lot cleaner and easier than my earlier approach, especially the inheritance here is a lot more predictable in the long run.

tl;dr this looks great to me!