AIBrix NeoCloud Node

Kubernetes bridge to elastic cloud GPUs.

AIBrix NeoCloud Node exposes burst GPU capacity from specialized cloud providers through a Kubernetes virtual node. The command source lives at cmd/neocloud-node, and release builds publish the binary as neocloud-node.

Project Coordinates

• Project: AIBrix NeoCloud Node
• Repo: aibrix/neocloud-node
• Module: github.com/aibrix/neocloud-node
• Binary: neocloud-node
• Image: ghcr.io/aibrix/neocloud-node

Status

This project is experimental and not production-ready.

The current implementation focuses on a reliable Kubernetes control-plane path: durable workload bindings, provider-neutral Pod transformation, idempotent remote lifecycle reconciliation, fake-provider conformance tests, and provider adapters for RunPod, Lambda Cloud, and Vast.ai.

Real cloud smoke tests are gated by explicit environment variables because they create paid GPU resources.

What Works Today

• A Kubernetes virtual node backed by Virtual Kubelet.
• CloudPodBinding state for durable Pod-to-provider lifecycle tracking.
• Provider-neutral transformation for a constrained single-container GPU Pod subset.
• Fake provider tests for local control-plane validation.
• RunPod Pod lifecycle adapter with create/get/delete/list-by-external-name.
• Lambda Cloud VM adapter with cloud-init runner callback support.
• Vast.ai marketplace adapter with offer search, price/reliability constraints, create/get/delete, deterministic recovery, and paginated resource listing.
• Provider error classification, fallback hooks, orphan sweeper mechanics, and basic burn-rate metrics.

Current Boundaries

NeoCloud Node intentionally does not claim full Kubernetes Pod semantics. The current provider path does not support cluster-internal Service networking, CNI/CSI, PVCs, hostPath, privileged workloads, sidecars, kubectl exec, attach, or port-forward. Container logs are not wired yet; provider-native console links or log collection are future work.

Lambda Cloud support is still experimental. The adapter and smoke script exist, but VM runner heartbeat timeout, logs, and stronger failure recovery need more work before it should be treated as a production provider.

Local Verification

Development requires Go 1.25.13 or newer and Kustomize 5.4 or newer. Docker is optional and is only needed for container builds.

make verify
make test-race
make docker-build

Install On Kubernetes

The default deployment uses the fake provider so the control-plane path can be tested without cloud credentials. The Kustomize bases under config/ are the canonical deployment source. deploy/neocloud-node.yaml is the generated, single-file equivalent for users who do not use Kustomize.

kubectl apply -k config/default
kubectl -n aibrix-system rollout status deploy/aibrix-neocloud-node
kubectl apply -f config/samples/fake-pod.yaml
kubectl get pod neocloud-node-fake-smoke -w
kubectl get cloudpodbindings

For a pinned image or environment-specific settings, create a Kustomize overlay that references config/default and patches the Deployment. Run make manifests after changing a base so the generated installation manifest stays current.

To switch the deployment to RunPod, set GPU_BURST_PROVIDER=runpod and provide RUNPOD_API_KEY through the aibrix-neocloud-node-runpod Secret referenced by the deployment. Provider credentials are never included in the repository.

To use Vast.ai, set GPU_BURST_PROVIDER=vast and provide VAST_API_KEY through the optional aibrix-neocloud-node-vast Secret. The adapter defaults to 20 GiB of disk, verified hosts, reliability of at least 0.95, and on-demand pricing. Pods can narrow placement with cloud.aibrix.ai/vast-verified, cloud.aibrix.ai/vast-machine-id, cloud.aibrix.ai/vast-disk-gb, and cloud.aibrix.ai/vast-offer-id. The generic cloud.aibrix.ai/max-price annotation is enforced during offer search and again before rental.

RunPod E2E Smoke Test

The real RunPod smoke test is gated so it never creates cloud resources by accident. It builds the local binary, runs the controller against the current Kubernetes context, creates one A40 RunPod pod, verifies Kubernetes Pod / CloudPodBinding state, deletes the pod, and checks RunPod for leftovers by external name.

export RUNPOD_API_KEY=...
RUNPOD_E2E=1 hack/e2e/runpod-smoke.sh

Defaults:

• GPU type: NVIDIA A40
• GPU count: 1
• virtual node: neocloud-node-runpod-e2e
• binary: /tmp/neocloud-node-e2e

Optional capacity-classification check:

RUNPOD_E2E=1 RUNPOD_E2E_CAPACITY_CHECK=1 hack/e2e/runpod-smoke.sh

The capacity check requests NVIDIA A40 with COMMUNITY + spot. If capacity is unavailable, the binding condition must be Capacity; if capacity is available, the script deletes the created pod.

Lambda Cloud E2E Smoke Test

The Lambda smoke test is also gated. It prefers a single A10 instance, falls back to a single A100 only when A10 capacity is unavailable, and refuses other instance types by default.

export LAMBDA_E2E=1
export LAMBDA_API_KEY=...
export LAMBDA_E2E_SSH_KEY_NAMES=public-key-dev
hack/e2e/lambda-smoke.sh

Vast.ai E2E Smoke Tests

Both Vast tests are explicitly gated because they create a paid instance. The provider smoke exercises the Go adapter directly, including deterministic create recovery, readiness, paginated listing, deletion, and final lookup:

export VAST_API_KEY=...
VAST_E2E=1 hack/e2e/vast-provider-smoke.sh

The Kubernetes smoke additionally builds and runs the local virtual-kubelet controller, creates a Pod and CloudPodBinding, waits for the remote instance, deletes the Pod, and verifies that no Vast resource remains by ID or label:

export VAST_API_KEY=...
VAST_E2E=1 hack/e2e/vast-smoke.sh

Defaults use one verified GTX 1070, 20 GiB disk, reliability >= 0.95, and a maximum rental price of $0.10/hour. Override them with VAST_E2E_GPU_TYPE, VAST_E2E_DISK_GB, VAST_E2E_MIN_RELIABILITY, and VAST_E2E_MAX_PRICE.

Resource Broker Direction

NeoCloud Node is the Kubernetes-facing runtime. The lower-level resource broker API is planned as a reusable layer for AIBrix batch, serving, and virtual-node integrations. That broker will own catalog, planning, lease, fallback, and cost interfaces; NeoCloud Node will consume it rather than embedding long-term resource strategy directly in the Virtual Kubelet adapter.