Repository navigation
Add DPF HBN-BGP discovery, generation and deployment foundations - #394
Draft
almaslennikov wants to merge 17 commits into
Draft
almaslennikov wants to merge 17 commits into
almaslennikov wants to merge 17 commits into
Conversation
Signed-off-by: Alexander Maslennikov <amaslennikov@nvidia.com>
Signed-off-by: Alexander Maslennikov <amaslennikov@nvidia.com>
Signed-off-by: Alexander Maslennikov <amaslennikov@nvidia.com>
Validate immutable generated intent before API access, check external prerequisites and scoped permissions, and compose namespace, retained Helm and operator configuration stages with durable operation results. Reject adoption, runtime spec drift, stale readiness and post-effect identity failures. Document the management-only receipt boundary; tenant, enrollment and HBN runtime stages remain in development. Signed-off-by: Alexander Maslennikov <amaslennikov@nvidia.com>
Retain the release-owned tenant Kubernetes version separately from management Kubernetes, verify controller endpoint and keepalived configuration, and bind in-memory tenant credentials to fresh management and tenant identities. Add bounded readiness waiting and retained-plan access handoff with durable failures. Reserve a profile-configurable host address in each workload subnet and distinguish this plan from runtime host configuration. Update procedural documentation and regressions; complete tenant creation and HBN deployment remain in development. Signed-off-by: Alexander Maslennikov <amaslennikov@nvidia.com>
Signed-off-by: Alexander Maslennikov <amaslennikov@nvidia.com>
Signed-off-by: Alexander Maslennikov <amaslennikov@nvidia.com>
Signed-off-by: Alexander Maslennikov <amaslennikov@nvidia.com>
Signed-off-by: Alexander Maslennikov <amaslennikov@nvidia.com>
Signed-off-by: Alexander Maslennikov <amaslennikov@nvidia.com>
Compose management and tenant execution with scoped BMC enrollment, exact shared credential reuse, identity-bound device readiness and durable partial failure reporting. Compare tenant VRRP placement using observed selectors and retain required uncertainty. Signed-off-by: Alexander Maslennikov <amaslennikov@nvidia.com>
Signed-off-by: Alexander Maslennikov <amaslennikov@nvidia.com>
Signed-off-by: Alexander Maslennikov <amaslennikov@nvidia.com>
Signed-off-by: Alexander Maslennikov <amaslennikov@nvidia.com>
Signed-off-by: Alexander Maslennikov <amaslennikov@nvidia.com>
Signed-off-by: Alexander Maslennikov <amaslennikov@nvidia.com>
Signed-off-by: Alexander Maslennikov <amaslennikov@nvidia.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds HBN-BGP discovery and manifest generation to the independent DPF target, stacked on #348. This remains a draft: the full deployment CLI and runtime qualification are unfinished.
l8k discover --target dpf --use-case hbn-bgpapplies profile defaults and user overrides, reads selected hosts/DPU BMCs, inspects the management cluster, and reads Cumulus configuration and LLDP/BGP state. Missing, mismatched or unverifiable required facts block readiness; inventory completion is reported separately. Management-network mode probes the selected API/BMC paths from Pod and node networks with UID-bound results and cleanup. Explicit BMC TLS bypass remains labelled unverified. Tenant placement, VIP/VRID conflicts and backing storage are checked separately from user declarations of address reservation. Discovery never changes switches. Once provisioning inputs are complete, discovery verifies the selected charts, derives management-side caller permissions from the retained plan, and submits scoped authorization reviews against the current cluster identity. Missing inputs, denied or incomplete reviews, cancellation and identity changes block readiness. Passed caller checks are summarized once on the console; individual results remain in the report. Controller, tenant, host and switch access remain separate checks.l8k generate --target dpf --use-case hbn-bgp --user-config site.yaml --save-deployment-files bundleverifies release-selected chart archives, renders management and HBN resources, validates their schemas and references, and publishes one ordered bundle atomically. It retains the source/resolved configuration, exact selected devices, tenant allocation declarations, hardware flavor and software locks. Host/gateway address reservations are distinct from ordinary IPAM allocation; configuring host addresses requires the later DPU/node/subnet join. Physical and host-facing interface selectors encode the intended tenant and exact selected DPU hostnames; live selector uniqueness remains a deployment check. HBN resource quantities feed both Helm container requests and DPF service-to-flavor accounting. Retained-plan verification rejects bundles whose selectors or accounting were altered; older preview bundles require regeneration. Generation does not establish deployment readiness.The private executor retains the same desired bundle across management bootstrap, tenant creation and selected-device enrollment; public authorization checks independently verify that bundle before deriving permissions. It performs scoped prerequisite checks, create-or-verify ownership checks, refreshed namespace/controller/tenant identities and durable operation reporting. Enrollment inspects the complete resource scope before activating controllers or shared credentials, verifies selected Administrator access, and requires exact current Device/Node identity plus credential/certificate/trust readiness. Conflicts, unrelated enrollments and partial enrollment stop dependent effects without adoption or deletion. Credentials and password hashes are excluded from bundles/reports.
Software preparation follows those stages using the same retained plan. It checks download/tunnel permissions, creates or verifies only the retained BlueFieldSoftware, waits for current Downloaded/Ready conditions, then hashes the actual ISO and selected PLDM bytes served by DPF's cache through a native Go port-forward with public-CA TLS verification. Registry/leader/shared-storage identities and the selected manager image reference are bound around the read. Partial, redirected, transformed or mismatched streams, replacement and cleanup failures block the receipt. No custom image or controller shell is needed. This verifies served bytes at observation time; it does not attest later BMC consumption or mutable chart bytes fetched asynchronously by Argo. HBN chart runtime checks must use the available source/version/application contract without inventing a consumed digest.
Procedural user documentation covers inputs, defaults, operation locations, outputs, recovery and workflow diagrams. Engineering contracts describe retained-plan execution and partial effects; lab details remain outside user guides.
Validation: the software-preparation increment passed three independent council reviews, focused race tests through real enrollment/tenant boundaries and HTTPS hashing, an existing-ready control-plane composition test, cancellation-after-receipt testing, full local build, pinned lint (0 issues), offline documentation contracts (237 examples), and strict HTML generation. The full deployment-package race suite passed on software-preparation commit50903db (792.150s). The subsequent renderer correction was accepted by two independent council reviewers after fixing a documentation overclaim, and passed the renderer race suite, full generation race suite (including selector/accounting tamper rejection), build, pinned lint, workflow generator checks, documentation contracts and strict HTML generation. Hosted CI43572143096 and43573798642 passed build/lint but failed tests: the DPF SPDY fixture closed during a shutdown frame, chart cancellation could be misreported as an empty archive, and one run exceeded the deployment-package timeout. The current correction drains the test shutdown frame, preserves cancellation after successful HTTP responses/reads, and reuses immutable generated test fixtures while retaining fresh execution state and production bundle validation. The latest run also hit an unchanged Host discovery polling-count assertion; Host behavior is outside this change. The final correction passed the full local deployment-package race/coverage suite (867.982s, 75.3% statement coverage), target/discovery race suites, build, pinned lint, documentation contracts and strict HTML generation. Hosted CI for this new revision is pending. These checks do not establish runtime qualification.
Live qualification so far: native Linux/amd64 discovery produced all 18 management-network results and confirmed probe cleanup; inventory-mode discovery detected the existing tenant VRID placement conflict; generation published a validated seven-stage bundle. Native registry adapter testing verified the current leader/cache/config/public-CA binding, TLS response and tunnel cleanup, but intentionally read no software artifact. A subsequent live inventory run completed with 130 management caller-authorization checks passed and no duplicate check IDs, while HBN readiness remained blocked by setup prerequisites and remaining unimplemented checks. No switch configuration changed and no HBN deployment success is claimed.
Remaining: resolve actual tenant allocation and host119 cabling, exercise controller enrollment/software staging, implement remaining workload and physical-effect stages, expose the complete deployment CLI, and verify BGP convergence and host traffic. Sentry is out of scope.