Skip to content

Expose the LLM request router to split-cluster workers #689

Description

@mikeyrcamp

Describe the bug

The self-managed multi-cluster topology currently advertises llm-request-router.nvcf.svc.cluster.local:50071 to workers. That address is local to the control-plane cluster and is not resolvable or reachable from a separate compute cluster.

The local multi-cluster tooling has no LLM request-router alias Service or Endpoints in the compute cluster, no port-50071 k3d mapping, and no Gateway listener or TCPRoute for the router. The current BDD does not launch an LLM worker, so it does not exercise this connection.

Expected behavior

Split-cluster workers receive an LLM request-router address that resolves and is reachable from the compute cluster, and local multi-cluster tests verify the connection path.

Suggested scope

  • Expose TCP port 50071 from the control-plane cluster through a Gateway and TCPRoute, load balancer, or equivalent supported path.
  • Add the required k3d host-port plumbing for local multi-cluster testing.
  • Add a compute-cluster alias Service and Endpoints, or define and document an external DNS endpoint.
  • Keep tests/bdd/fixtures/self-managed-local-bdd-multi.yaml and the raw seed in tests/bdd/godog_test.go aligned.
  • Add an end-to-end test that launches an LLM worker across clusters.
  • Document the split-cluster network, DNS, routing, and TLS requirements.

Context

Follow-up to #678. That pull request wires the configured address through the NVCF API, but intentionally does not add cross-cluster network exposure.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions