Skip to content

Add DPF HBN-BGP discovery, generation and deployment foundations - #394

Draft
almaslennikov wants to merge 17 commits into
dev/dpf-discoveryfrom
dev/dpf-hbn-bgp
Draft

almaslennikov wants to merge 17 commits into
dev/dpf-discoveryfrom
dev/dpf-hbn-bgp

Conversation

@almaslennikov

@almaslennikov almaslennikov commented Nov 13, 2026 •

Copy link
Copy Markdown
Collaborator

Adds HBN-BGP discovery and manifest generation to the independent DPF target, stacked on #348. This remains a draft: the full deployment CLI and runtime qualification are unfinished.

l8k discover --target dpf --use-case hbn-bgp applies profile defaults and user overrides, reads selected hosts/DPU BMCs, inspects the management cluster, and reads Cumulus configuration and LLDP/BGP state. Missing, mismatched or unverifiable required facts block readiness; inventory completion is reported separately. Management-network mode probes the selected API/BMC paths from Pod and node networks with UID-bound results and cleanup. Explicit BMC TLS bypass remains labelled unverified. Tenant placement, VIP/VRID conflicts and backing storage are checked separately from user declarations of address reservation. Discovery never changes switches. Once provisioning inputs are complete, discovery verifies the selected charts, derives management-side caller permissions from the retained plan, and submits scoped authorization reviews against the current cluster identity. Missing inputs, denied or incomplete reviews, cancellation and identity changes block readiness. Passed caller checks are summarized once on the console; individual results remain in the report. Controller, tenant, host and switch access remain separate checks.

l8k generate --target dpf --use-case hbn-bgp --user-config site.yaml --save-deployment-files bundle verifies release-selected chart archives, renders management and HBN resources, validates their schemas and references, and publishes one ordered bundle atomically. It retains the source/resolved configuration, exact selected devices, tenant allocation declarations, hardware flavor and software locks. Host/gateway address reservations are distinct from ordinary IPAM allocation; configuring host addresses requires the later DPU/node/subnet join. Physical and host-facing interface selectors encode the intended tenant and exact selected DPU hostnames; live selector uniqueness remains a deployment check. HBN resource quantities feed both Helm container requests and DPF service-to-flavor accounting. Retained-plan verification rejects bundles whose selectors or accounting were altered; older preview bundles require regeneration. Generation does not establish deployment readiness.

The private executor retains the same desired bundle across management bootstrap, tenant creation and selected-device enrollment; public authorization checks independently verify that bundle before deriving permissions. It performs scoped prerequisite checks, create-or-verify ownership checks, refreshed namespace/controller/tenant identities and durable operation reporting. Enrollment inspects the complete resource scope before activating controllers or shared credentials, verifies selected Administrator access, and requires exact current Device/Node identity plus credential/certificate/trust readiness. Conflicts, unrelated enrollments and partial enrollment stop dependent effects without adoption or deletion. Credentials and password hashes are excluded from bundles/reports.

Software preparation follows those stages using the same retained plan. It checks download/tunnel permissions, creates or verifies only the retained BlueFieldSoftware, waits for current Downloaded/Ready conditions, then hashes the actual ISO and selected PLDM bytes served by DPF's cache through a native Go port-forward with public-CA TLS verification. Registry/leader/shared-storage identities and the selected manager image reference are bound around the read. Partial, redirected, transformed or mismatched streams, replacement and cleanup failures block the receipt. No custom image or controller shell is needed. This verifies served bytes at observation time; it does not attest later BMC consumption or mutable chart bytes fetched asynchronously by Argo. HBN chart runtime checks must use the available source/version/application contract without inventing a consumed digest.

Procedural user documentation covers inputs, defaults, operation locations, outputs, recovery and workflow diagrams. Engineering contracts describe retained-plan execution and partial effects; lab details remain outside user guides.

Validation: the software-preparation increment passed three independent council reviews, focused race tests through real enrollment/tenant boundaries and HTTPS hashing, an existing-ready control-plane composition test, cancellation-after-receipt testing, full local build, pinned lint (0 issues), offline documentation contracts (237 examples), and strict HTML generation. The full deployment-package race suite passed on software-preparation commit50903db (792.150s). The subsequent renderer correction was accepted by two independent council reviewers after fixing a documentation overclaim, and passed the renderer race suite, full generation race suite (including selector/accounting tamper rejection), build, pinned lint, workflow generator checks, documentation contracts and strict HTML generation. Hosted CI43572143096 and43573798642 passed build/lint but failed tests: the DPF SPDY fixture closed during a shutdown frame, chart cancellation could be misreported as an empty archive, and one run exceeded the deployment-package timeout. The current correction drains the test shutdown frame, preserves cancellation after successful HTTP responses/reads, and reuses immutable generated test fixtures while retaining fresh execution state and production bundle validation. The latest run also hit an unchanged Host discovery polling-count assertion; Host behavior is outside this change. The final correction passed the full local deployment-package race/coverage suite (867.982s, 75.3% statement coverage), target/discovery race suites, build, pinned lint, documentation contracts and strict HTML generation. Hosted CI for this new revision is pending. These checks do not establish runtime qualification.

Live qualification so far: native Linux/amd64 discovery produced all 18 management-network results and confirmed probe cleanup; inventory-mode discovery detected the existing tenant VRID placement conflict; generation published a validated seven-stage bundle. Native registry adapter testing verified the current leader/cache/config/public-CA binding, TLS response and tunnel cleanup, but intentionally read no software artifact. A subsequent live inventory run completed with 130 management caller-authorization checks passed and no duplicate check IDs, while HBN readiness remained blocked by setup prerequisites and remaining unimplemented checks. No switch configuration changed and no HBN deployment success is claimed.

Remaining: resolve actual tenant allocation and host119 cabling, exercise controller enrollment/software staging, implement remaining workload and physical-effect stages, expose the complete deployment CLI, and verify BGP convergence and host traffic. Sentry is out of scope.

Signed-off-by: Alexander Maslennikov <amaslennikov@nvidia.com>
Signed-off-by: Alexander Maslennikov <amaslennikov@nvidia.com>
Signed-off-by: Alexander Maslennikov <amaslennikov@nvidia.com>
Validate immutable generated intent before API access, check external prerequisites and scoped permissions, and compose namespace, retained Helm and operator configuration stages with durable operation results. Reject adoption, runtime spec drift, stale readiness and post-effect identity failures.

Document the management-only receipt boundary; tenant, enrollment and HBN runtime stages remain in development.

Signed-off-by: Alexander Maslennikov <amaslennikov@nvidia.com>
@almaslennikov almaslennikov changed the title Add DPF HBN-BGP discovery and native generation Add DPF HBN-BGP discovery, generation and management bootstrap Nov 14, 2026
Retain the release-owned tenant Kubernetes version separately from management Kubernetes, verify controller endpoint and keepalived configuration, and bind in-memory tenant credentials to fresh management and tenant identities. Add bounded readiness waiting and retained-plan access handoff with durable failures.

Reserve a profile-configurable host address in each workload subnet and distinguish this plan from runtime host configuration. Update procedural documentation and regressions; complete tenant creation and HBN deployment remain in development.

Signed-off-by: Alexander Maslennikov <amaslennikov@nvidia.com>
@almaslennikov almaslennikov changed the title Add DPF HBN-BGP discovery, generation and management bootstrap Add DPF HBN-BGP discovery, generation and deployment foundations Nov 14, 2026
Signed-off-by: Alexander Maslennikov <amaslennikov@nvidia.com>
Signed-off-by: Alexander Maslennikov <amaslennikov@nvidia.com>
Signed-off-by: Alexander Maslennikov <amaslennikov@nvidia.com>
Signed-off-by: Alexander Maslennikov <amaslennikov@nvidia.com>
Signed-off-by: Alexander Maslennikov <amaslennikov@nvidia.com>
Compose management and tenant execution with scoped BMC enrollment, exact shared credential reuse, identity-bound device readiness and durable partial failure reporting. Compare tenant VRRP placement using observed selectors and retain required uncertainty.

Signed-off-by: Alexander Maslennikov <amaslennikov@nvidia.com>
Signed-off-by: Alexander Maslennikov <amaslennikov@nvidia.com>
Signed-off-by: Alexander Maslennikov <amaslennikov@nvidia.com>
Signed-off-by: Alexander Maslennikov <amaslennikov@nvidia.com>
Signed-off-by: Alexander Maslennikov <amaslennikov@nvidia.com>
Signed-off-by: Alexander Maslennikov <amaslennikov@nvidia.com>
Signed-off-by: Alexander Maslennikov <amaslennikov@nvidia.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant