Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 75 additions & 0 deletions deploy/helm/openshell/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,77 @@ namespace. The gateway and workspace releases can then be upgraded and removed
independently. Use Kubernetes `operator` workspace mode when one gateway serves
multiple pre-provisioned workspace namespaces.

## Cluster-scoped vs namespaced objects

Most objects in this chart are namespaced and land in the release namespace.
Only two are cluster-scoped:

| Object | Default name |
| --- | --- |
| `ClusterRole` | `<fullname>-node-reader-<release namespace>` |
| `ClusterRoleBinding` | `<fullname>-node-reader-<release namespace>` |

By default the release creates both, so an install by a cluster-admin is
unchanged. On clusters where cluster-scoped RBAC is owned by a different team,
split the install in two.

A cluster-admin applies the cluster-scoped objects once per gateway
ServiceAccount, rendered from the same values the release uses:

```shell
helm template openshell oci://ghcr.io/nvidia/openshell/helm-chart --version <version> \
--namespace openshell -f my-values.yaml \
--set rbac.clusterScoped.create=true \
--set agentSandbox.preflight.enabled=false \
--show-only templates/clusterrole.yaml \
--show-only templates/clusterrolebinding.yaml | kubectl apply -f -
```

A namespace-admin then installs and upgrades the release with cluster-scoped
objects omitted, using [`ci/values-namespace-admin.yaml`](ci/values-namespace-admin.yaml)
or the equivalent `--set`:

```shell
helm upgrade --install openshell oci://ghcr.io/nvidia/openshell/helm-chart --version <version> \
--namespace openshell -f my-values.yaml \
--set supervisor.sandboxRuntime.networkPolicyEnforced=true \
--set rbac.clusterScoped.create=false
```

The gateway ServiceAccount name and namespace do not change, so the
pre-created `ClusterRoleBinding` keeps matching the release. This works with
`serviceAccount.create=false` too: the `ClusterRoleBinding` subject follows
`serviceAccount.name`, so render the admin step with the same values.

### Migrating an existing release

Helm deletes objects that leave a release manifest, so setting
`rbac.clusterScoped.create=false` on a release that already owns the
`ClusterRole` and `ClusterRoleBinding` deletes them. The gateway then loses
TokenReview until a cluster-admin re-applies them. Hand ownership over first, as
cluster-admin, so nothing is deleted:

```shell
kubectl annotate clusterrole "openshell-node-reader-<namespace>" \
helm.sh/resource-policy=keep --overwrite
kubectl annotate clusterrolebinding "openshell-node-reader-<namespace>" \
helm.sh/resource-policy=keep --overwrite
```

The objects then survive the upgrade that sets the flag, and the cluster-admin
owns them from that point on. Fresh installs need no such step.

`rbac.clusterScoped.create` is independent of
`server.drivers.kubernetes.workspaceMode`. Managed and operator modes change
what the `ClusterRole` contains, but they never force the namespaced release to
apply it. Re-run the cluster-admin step after changing values that affect the
`ClusterRole` rules.

Set `rbac.create=false` to also omit the namespaced sandbox `Role` and
`RoleBinding`. The certgen hook and credential driver RBAC keep their own flags
(`pkiInitJob.enabled` and
`server.credentialDrivers.kubernetesSecrets.rbac.create`).

## Prerequisites

The Kubernetes Agent Sandbox CRDs and controller must be installed on the cluster before deploying OpenShell. Install them with:
Expand Down Expand Up @@ -230,6 +301,10 @@ discovery endpoint or its TLS CA.
| probes.startup.failureThreshold | int | `30` | Startup probe failure threshold before the container is killed. |
| probes.startup.periodSeconds | int | `2` | Startup probe period, in seconds. |
| probes.startup.timeoutSeconds | int | `1` | Startup probe timeout, in seconds. |
| rbac.clusterScoped.clusterRoleBindingName | string | `""` | Name for the ClusterRoleBinding. Empty uses the `<fullname>-node-reader-<release namespace>` default. |
| rbac.clusterScoped.clusterRoleName | string | `""` | Name for the ClusterRole. Empty uses the `<fullname>-node-reader-<release namespace>` default. |
| rbac.clusterScoped.create | bool | `true` | Create the cluster-scoped ClusterRole and ClusterRoleBinding. Disable for a namespace-admin install where a cluster-admin applies them separately; the gateway ServiceAccount name and namespace are unchanged, so a pre-created ClusterRoleBinding still matches. |
| rbac.create | bool | `true` | Create the RBAC objects that grant the gateway ServiceAccount access. Disable to supply the namespaced sandbox Role/RoleBinding and the cluster-scoped ClusterRole/ClusterRoleBinding out of band. The certgen hook and credential driver RBAC keep their own flags. |
| replicaCount | int | `1` | Number of OpenShell gateway replicas. Values greater than 1 require server.externalDbSecret because the default SQLite backend is per pod. |
| resources | object | `{}` | Gateway pod resource requests and limits. |
| sandboxRuntime.image.pullPolicy | string | `""` | Sandbox runtime image pull policy. Defaults to the gateway image pull policy when empty. |
Expand Down
71 changes: 71 additions & 0 deletions deploy/helm/openshell/README.md.gotmpl
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,77 @@ namespace. The gateway and workspace releases can then be upgraded and removed
independently. Use Kubernetes `operator` workspace mode when one gateway serves
multiple pre-provisioned workspace namespaces.

## Cluster-scoped vs namespaced objects

Most objects in this chart are namespaced and land in the release namespace.
Only two are cluster-scoped:

| Object | Default name |
| --- | --- |
| `ClusterRole` | `<fullname>-node-reader-<release namespace>` |
| `ClusterRoleBinding` | `<fullname>-node-reader-<release namespace>` |

By default the release creates both, so an install by a cluster-admin is
unchanged. On clusters where cluster-scoped RBAC is owned by a different team,
split the install in two.

A cluster-admin applies the cluster-scoped objects once per gateway
ServiceAccount, rendered from the same values the release uses:

```shell
helm template openshell oci://ghcr.io/nvidia/openshell/helm-chart --version <version> \
--namespace openshell -f my-values.yaml \
--set rbac.clusterScoped.create=true \
--set agentSandbox.preflight.enabled=false \
--show-only templates/clusterrole.yaml \
--show-only templates/clusterrolebinding.yaml | kubectl apply -f -
```

A namespace-admin then installs and upgrades the release with cluster-scoped
objects omitted, using [`ci/values-namespace-admin.yaml`](ci/values-namespace-admin.yaml)
or the equivalent `--set`:

```shell
helm upgrade --install openshell oci://ghcr.io/nvidia/openshell/helm-chart --version <version> \
--namespace openshell -f my-values.yaml \
--set supervisor.sandboxRuntime.networkPolicyEnforced=true \
--set rbac.clusterScoped.create=false
```

The gateway ServiceAccount name and namespace do not change, so the
pre-created `ClusterRoleBinding` keeps matching the release. This works with
`serviceAccount.create=false` too: the `ClusterRoleBinding` subject follows
`serviceAccount.name`, so render the admin step with the same values.

### Migrating an existing release

Helm deletes objects that leave a release manifest, so setting
`rbac.clusterScoped.create=false` on a release that already owns the
`ClusterRole` and `ClusterRoleBinding` deletes them. The gateway then loses
TokenReview until a cluster-admin re-applies them. Hand ownership over first, as
cluster-admin, so nothing is deleted:

```shell
kubectl annotate clusterrole "openshell-node-reader-<namespace>" \
helm.sh/resource-policy=keep --overwrite
kubectl annotate clusterrolebinding "openshell-node-reader-<namespace>" \
helm.sh/resource-policy=keep --overwrite
```

The objects then survive the upgrade that sets the flag, and the cluster-admin
owns them from that point on. Fresh installs need no such step.

`rbac.clusterScoped.create` is independent of
`server.drivers.kubernetes.workspaceMode`. Managed and operator modes change
what the `ClusterRole` contains, but they never force the namespaced release to
apply it. Re-run the cluster-admin step after changing values that affect the
`ClusterRole` rules.

Set `rbac.create=false` to also omit the namespaced sandbox `Role` and
`RoleBinding`. The certgen hook and credential driver RBAC keep their own flags
(`pkiInitJob.enabled` and
`server.credentialDrivers.kubernetesSecrets.rbac.create`).

## Prerequisites

The Kubernetes Agent Sandbox CRDs and controller must be installed on the cluster before deploying OpenShell. Install them with:
Expand Down
24 changes: 24 additions & 0 deletions deploy/helm/openshell/ci/values-namespace-admin.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

# Namespace-admin overlay — renders only namespaced objects.
#
# Use this when a cluster-admin applies the gateway ClusterRole and
# ClusterRoleBinding once, out of band, and the OpenShell release is installed
# and upgraded by an installer that holds no cluster-scoped permissions.
#
# Generate the cluster-scoped objects for the cluster-admin step from the same
# release values, then apply them as cluster-admin:
# helm template openshell oci://ghcr.io/nvidia/openshell/helm-chart \
# --version <version> --namespace openshell -f my-values.yaml \
# --set rbac.clusterScoped.create=true \
# --set agentSandbox.preflight.enabled=false \
# --show-only templates/clusterrole.yaml \
# --show-only templates/clusterrolebinding.yaml | kubectl apply -f -
#
# The gateway ServiceAccount name and release namespace are unchanged, so the
# pre-created ClusterRoleBinding still matches this release.

rbac:
clusterScoped:
create: false
60 changes: 60 additions & 0 deletions deploy/helm/openshell/templates/_helpers.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,66 @@ default to enabled so upgrades with --reuse-values preserve the old topology.
{{- if $enabled -}}true{{- end -}}
{{- end }}

{{/*
Whether this chart owns gateway RBAC objects. Missing legacy values default to
enabled so upgrades with --reuse-values preserve the old topology.
*/}}
{{- define "openshell.rbacCreate" -}}
{{- $rbac := .Values.rbac | default dict -}}
{{- $create := true -}}
{{- if hasKey $rbac "create" -}}
{{- $create = get $rbac "create" -}}
{{- end -}}
{{- if $create -}}true{{- end -}}
{{- end }}

{{/*
The rbac.clusterScoped values map, tolerating missing legacy values.
*/}}
{{- define "openshell.clusterScopedRbacValues" -}}
{{- $rbac := .Values.rbac | default dict -}}
{{- $clusterScoped := dict -}}
{{- if hasKey $rbac "clusterScoped" -}}
{{- $clusterScoped = get $rbac "clusterScoped" | default dict -}}
{{- end -}}
{{- toYaml $clusterScoped -}}
{{- end }}

{{/*
Whether this chart owns the cluster-scoped ClusterRole and ClusterRoleBinding.
Disable for a namespace-admin install where a cluster-admin applies them
separately. Missing legacy values default to enabled.
*/}}
{{- define "openshell.clusterRbacCreate" -}}
{{- if include "openshell.rbacCreate" . -}}
{{- $clusterScoped := include "openshell.clusterScopedRbacValues" . | fromYaml -}}
{{- $create := true -}}
{{- if hasKey $clusterScoped "create" -}}
{{- $create = get $clusterScoped "create" -}}
{{- end -}}
{{- if $create -}}true{{- end -}}
{{- end -}}
{{- end }}

{{/*
Name of the gateway ClusterRole. The release namespace is part of the default
name so multiple releases on one cluster do not collide.
*/}}
{{- define "openshell.clusterRoleName" -}}
{{- $clusterScoped := include "openshell.clusterScopedRbacValues" . | fromYaml -}}
{{- $default := printf "%s-node-reader-%s" (include "openshell.fullname" .) .Release.Namespace -}}
{{- default $default (get $clusterScoped "clusterRoleName") -}}
{{- end }}

{{/*
Name of the gateway ClusterRoleBinding.
*/}}
{{- define "openshell.clusterRoleBindingName" -}}
{{- $clusterScoped := include "openshell.clusterScopedRbacValues" . | fromYaml -}}
{{- $default := printf "%s-node-reader-%s" (include "openshell.fullname" .) .Release.Namespace -}}
{{- default $default (get $clusterScoped "clusterRoleBindingName") -}}
{{- end }}

{{/*
Gateway image reference. Uses image.tag when set; falls back to .Chart.AppVersion
so a released chart automatically pulls the matching image without extra overrides.
Expand Down
4 changes: 3 additions & 1 deletion deploy/helm/openshell/templates/clusterrole.yaml
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
{{- if include "openshell.clusterRbacCreate" . }}
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

{{- $workspaceMode := .Values.server.drivers.kubernetes.workspaceMode | default "shared" }}
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: {{ include "openshell.fullname" . }}-node-reader-{{ .Release.Namespace }}
name: {{ include "openshell.clusterRoleName" . }}
labels:
{{- include "openshell.labels" . | nindent 4 }}
rules:
Expand Down Expand Up @@ -165,3 +166,4 @@ rules:
- update
{{- end }}
{{- end }}
{{- end }}
6 changes: 4 additions & 2 deletions deploy/helm/openshell/templates/clusterrolebinding.yaml
Original file line number Diff line number Diff line change
@@ -1,17 +1,19 @@
{{- if include "openshell.clusterRbacCreate" . }}
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: {{ include "openshell.fullname" . }}-node-reader-{{ .Release.Namespace }}
name: {{ include "openshell.clusterRoleBindingName" . }}
labels:
{{- include "openshell.labels" . | nindent 4 }}
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: {{ include "openshell.fullname" . }}-node-reader-{{ .Release.Namespace }}
name: {{ include "openshell.clusterRoleName" . }}
subjects:
- kind: ServiceAccount
name: {{ include "openshell.serviceAccountName" . }}
namespace: {{ .Release.Namespace }}
{{- end }}
2 changes: 1 addition & 1 deletion deploy/helm/openshell/templates/role.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{{- $workspaceMode := .Values.server.drivers.kubernetes.workspaceMode | default "shared" -}}
{{- if and (eq $workspaceMode "shared") (include "openshell.workspaceResourcesEnabled" .) }}
{{- if and (eq $workspaceMode "shared") (include "openshell.workspaceResourcesEnabled" .) (include "openshell.rbacCreate" .) }}
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
apiVersion: rbac.authorization.k8s.io/v1
Expand Down
2 changes: 1 addition & 1 deletion deploy/helm/openshell/templates/rolebinding.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{{- $workspaceMode := .Values.server.drivers.kubernetes.workspaceMode | default "shared" -}}
{{- if and (eq $workspaceMode "shared") (include "openshell.workspaceResourcesEnabled" .) }}
{{- if and (eq $workspaceMode "shared") (include "openshell.workspaceResourcesEnabled" .) (include "openshell.rbacCreate" .) }}
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
apiVersion: rbac.authorization.k8s.io/v1
Expand Down
40 changes: 40 additions & 0 deletions deploy/helm/openshell/tests/clusterrole_test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -186,3 +186,43 @@ tests:
apiGroups: [""]
resources: ["secrets"]
verbs: ["create", "list", "delete"]

- it: omits the ClusterRole when cluster-scoped RBAC is disabled
set:
rbac.clusterScoped.create: false
asserts:
- hasDocuments:
count: 0

- it: omits the ClusterRole when all chart-managed RBAC is disabled
set:
rbac.create: false
asserts:
- hasDocuments:
count: 0

- it: omits the ClusterRole independently of the workspace mode
set:
rbac.clusterScoped.create: false
server.drivers.kubernetes.workspaceMode: managed
asserts:
- hasDocuments:
count: 0

- it: uses the configured ClusterRole name
set:
rbac.clusterScoped.clusterRoleName: platform-openshell-node-reader
asserts:
- equal:
path: metadata.name
value: platform-openshell-node-reader

- it: creates the ClusterRole when legacy values omit the rbac block
set:
rbac: null
asserts:
- hasDocuments:
count: 1
- equal:
path: metadata.name
value: openshell-node-reader-my-namespace
Loading
Loading