Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -217,6 +217,10 @@ jobs:
- 'tests/require-baseline-workers-for-hcloud-volumes/**'
- 'tests/audit-privileged-rbac/**'
- 'tests/auto-vpa/**'
# The platform's copy of the catalog auto-vpa GeneratingPolicy: its
# fixtures and its system-namespace exclusion test (#5130).
- 'tests/auto-vpa-generating-policy-*/**'
- 'scripts/tests/test-auto-vpa-generating-policy-exclusion.sh'
- 'scripts/tests/test-audit-privileged-rbac.sh'
# Renders the charts whose roles that policy excludes and applies the
# policy to their RBAC, so a chart bump fails its own PR (#3792).
Expand Down Expand Up @@ -1628,6 +1632,8 @@ jobs:
bash scripts/tests/test-tetragon-agent-priority.sh
shellcheck scripts/tests/test-audit-privileged-rbac.sh
bash scripts/tests/test-audit-privileged-rbac.sh
shellcheck scripts/tests/test-auto-vpa-generating-policy-exclusion.sh
bash scripts/tests/test-auto-vpa-generating-policy-exclusion.sh
shellcheck scripts/tests/test-arc-controller-availability.sh
bash scripts/tests/test-arc-controller-availability.sh
shellcheck scripts/tests/test-restrict-tenant-secret-stores.sh
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
---
# ADOPTED from devantler-tech/kyverno-policies
# policies/best-practices/auto-vpa-generating-policy.yaml at 44ed118 (#5130), which
# registers three GeneratingPolicy objects in one file. The platform keeps one
# resource per file, so it is split into auto-vpa-deployment.yaml (which also
# carries the catalog's own header), auto-vpa-statefulset.yaml and
# auto-vpa-daemonset.yaml. Each policy is copied verbatim except for one marked
# PLATFORM ADDITION: a namespaceSelector that keeps it out of the system
# namespaces, as the classic best-practices/auto-vpa.yaml did. The generate
# expressions must stay byte-for-byte the catalog's (whitespace aside): both static
# producer guards pin them (scripts/guard-consumer-discovery-conservation.sh and
# scripts/tests/crossplane-egress-policy-rules.yaml). The classic VPAs are named
# <name>, these are <name>-<kind>, so running both forms at once would put two VPAs
# on every workload: register these only after the classic policy's VPAs are gone.
apiVersion: policies.kyverno.io/v1
kind: GeneratingPolicy
metadata:
name: auto-vpa-daemonset
annotations:
policies.kyverno.io/title: Auto VPA for DaemonSets (GeneratingPolicy)
policies.kyverno.io/category: Autoscaling
policies.kyverno.io/severity: low
policies.kyverno.io/subject: DaemonSet
policies.kyverno.io/minversion: 1.19.0
policies.kyverno.io/description: >-
Generates a CPU-only recommendation-bounded VerticalPodAutoscaler for each
DaemonSet. GeneratingPolicy form of the classic auto-vpa rule of the same name.
spec:
# Parity with the classic rules' `synchronize: true` + `generateExisting: true`:
# on GeneratingPolicy both live under spec.evaluation and default to false, so
# omitting them would silently drop behavior the classic policy guarantees.
evaluation:
generateExisting:
enabled: true
synchronize:
enabled: true
matchConstraints:
resourceRules:
- apiGroups: ["apps"]
apiVersions: ["v1"]
operations: ["CREATE", "UPDATE"]
resources: ["daemonsets"]
# PLATFORM ADDITION (the only change from the catalog file): prod ships
# static VPAs for the controllers in these namespaces
# (k8s/providers/hetzner/infrastructure/vertical-pod-autoscalers/), so a
# generated one would put a second VPA on each of those pods.
namespaceSelector:
matchExpressions:
- key: kubernetes.io/metadata.name
operator: NotIn
values:
- kube-system
- kube-public
- kube-node-lease
- kyverno
generate:
- expression: >-
generator.Apply(
string(object.metadata.namespace),
[
{
"apiVersion": dyn("autoscaling.k8s.io/v1"),
"kind": dyn("VerticalPodAutoscaler"),
"metadata": dyn({
"name": string(object.metadata.name) + "-daemonset",
"namespace": string(object.metadata.namespace)
}),
"spec": dyn({
"targetRef": dyn({
"apiVersion": "apps/v1",
"kind": "DaemonSet",
"name": string(object.metadata.name)
}),
"updatePolicy": dyn({"updateMode": "InPlaceOrRecreate"}),
"resourcePolicy": dyn({
"containerPolicies": [
{
"containerName": dyn("*"),
"controlledResources": dyn(["cpu"]),
"controlledValues": dyn("RequestsAndLimits"),
"minAllowed": dyn({"cpu": "50m"}),
"maxAllowed": dyn({"cpu": "1"})
}
]
})
})
}
]
)
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
---
# ADOPTED from devantler-tech/kyverno-policies
# policies/best-practices/auto-vpa-generating-policy.yaml at 44ed118 (#5130), which
# registers three GeneratingPolicy objects in one file. The platform keeps one
# resource per file, so it is split into auto-vpa-deployment.yaml (which also
# carries the catalog's own header), auto-vpa-statefulset.yaml and
# auto-vpa-daemonset.yaml. Each policy is copied verbatim except for one marked
# PLATFORM ADDITION: a namespaceSelector that keeps it out of the system
# namespaces, as the classic best-practices/auto-vpa.yaml did. The generate
# expressions must stay byte-for-byte the catalog's (whitespace aside): both static
# producer guards pin them (scripts/guard-consumer-discovery-conservation.sh and
# scripts/tests/crossplane-egress-policy-rules.yaml). The classic VPAs are named
# <name>, these are <name>-<kind>, so running both forms at once would put two VPAs
# on every workload: register these only after the classic policy's VPAs are gone.
#
# The catalog's header follows. In this repository the classic file is the rollback
# form, and "the root kustomization.yaml" means this folder's kustomization.yaml.
# "Under the same name" below is not true here: the classic VPA is <name> and this
# one is <name>-deployment, so running both forms gives every workload two VPAs.
#
# The catalog's auto-vpa: a port of the classic policies/best-practices/auto-vpa.yaml
# ClusterPolicy rules to the GeneratingPolicy API, which Kyverno keeps after classic
# ClusterPolicy generate rules are retired. The classic ClusterPolicy is a separate file.
#
# THIS IS THE REGISTERED FORM in the root kustomization.yaml. The classic file stays in the
# repository, unregistered, as the rollback form until every consumer has migrated: never
# apply both to one cluster, because both generate a VerticalPodAutoscaler for the same
# workload under the same name. The README's "Migrating to the GeneratingPolicy form"
# section is the consumer rollout and rollback procedure.
Comment thread
coderabbitai[bot] marked this conversation as resolved.
#
# Split into one policy per workload kind, mirroring the classic policy's three rules:
# GeneratingPolicy applies spec.matchConstraints to the whole policy, so three kinds with
# three different VPA bodies cannot share one policy without branching inside CEL.
#
# FLOOR: Kyverno 1.19.0. On 1.18.x spec.evaluation.generateExisting is schema-accepted but
# does not fire for PRE-EXISTING workloads (kyverno/kyverno#15722, a regression against the
# classic rule; fixed by kyverno/kyverno#16041 for 1.19.0 and never backported), so an
# existing workload would get its VPA only on its next CREATE/UPDATE admission. The
# minversion annotations below declare that floor.
apiVersion: policies.kyverno.io/v1
kind: GeneratingPolicy
metadata:
name: auto-vpa-deployment
annotations:
policies.kyverno.io/title: Auto VPA for Deployments (GeneratingPolicy)
policies.kyverno.io/category: Autoscaling
policies.kyverno.io/severity: low
policies.kyverno.io/subject: Deployment
policies.kyverno.io/minversion: 1.19.0
policies.kyverno.io/description: >-
Generates a recommendation-bounded VerticalPodAutoscaler for each Deployment.
GeneratingPolicy form of the classic auto-vpa rule of the same name.
spec:
# Parity with the classic rules' `synchronize: true` + `generateExisting: true`:
# on GeneratingPolicy both live under spec.evaluation and default to false, so
# omitting them would silently drop behavior the classic policy guarantees.
evaluation:
generateExisting:
enabled: true
synchronize:
enabled: true
matchConstraints:
resourceRules:
- apiGroups: ["apps"]
apiVersions: ["v1"]
operations: ["CREATE", "UPDATE"]
resources: ["deployments"]
# PLATFORM ADDITION (the only change from the catalog file): prod ships
# static VPAs for the controllers in these namespaces
# (k8s/providers/hetzner/infrastructure/vertical-pod-autoscalers/), so a
# generated one would put a second VPA on each of those pods.
namespaceSelector:
matchExpressions:
- key: kubernetes.io/metadata.name
operator: NotIn
values:
- kube-system
- kube-public
- kube-node-lease
- kyverno
generate:
# Every value is dyn(): CEL infers a map's value type from its first entry, so a
# resource literal mixing strings with nested maps fails to compile without it.
# Fields read off `object` are dyn and need string() before concatenation.
- expression: >-
generator.Apply(
string(object.metadata.namespace),
[
{
"apiVersion": dyn("autoscaling.k8s.io/v1"),
"kind": dyn("VerticalPodAutoscaler"),
"metadata": dyn({
"name": string(object.metadata.name) + "-deployment",
Comment thread
devantler marked this conversation as resolved.
"namespace": string(object.metadata.namespace)
}),
Comment thread
devantler marked this conversation as resolved.
"spec": dyn({
"targetRef": dyn({
"apiVersion": "apps/v1",
"kind": "Deployment",
"name": string(object.metadata.name)
}),
"updatePolicy": dyn({"updateMode": "InPlaceOrRecreate"}),
Comment thread
devantler marked this conversation as resolved.
"resourcePolicy": dyn({
"containerPolicies": [
{
"containerName": dyn("*"),
"controlledResources": dyn(["cpu", "memory"]),
"controlledValues": dyn("RequestsAndLimits"),
"minAllowed": dyn({"cpu": "50m", "memory": "64Mi"}),
"maxAllowed": dyn({"cpu": "3", "memory": "6Gi"})
}
]
})
})
}
]
)
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
---
# ADOPTED from devantler-tech/kyverno-policies
# policies/best-practices/auto-vpa-generating-policy.yaml at 44ed118 (#5130), which
# registers three GeneratingPolicy objects in one file. The platform keeps one
# resource per file, so it is split into auto-vpa-deployment.yaml (which also
# carries the catalog's own header), auto-vpa-statefulset.yaml and
# auto-vpa-daemonset.yaml. Each policy is copied verbatim except for one marked
# PLATFORM ADDITION: a namespaceSelector that keeps it out of the system
# namespaces, as the classic best-practices/auto-vpa.yaml did. The generate
# expressions must stay byte-for-byte the catalog's (whitespace aside): both static
# producer guards pin them (scripts/guard-consumer-discovery-conservation.sh and
# scripts/tests/crossplane-egress-policy-rules.yaml). The classic VPAs are named
# <name>, these are <name>-<kind>, so running both forms at once would put two VPAs
# on every workload: register these only after the classic policy's VPAs are gone.
apiVersion: policies.kyverno.io/v1
kind: GeneratingPolicy
metadata:
name: auto-vpa-statefulset
annotations:
policies.kyverno.io/title: Auto VPA for StatefulSets (GeneratingPolicy)
policies.kyverno.io/category: Autoscaling
policies.kyverno.io/severity: low
policies.kyverno.io/subject: StatefulSet
policies.kyverno.io/minversion: 1.19.0
policies.kyverno.io/description: >-
Generates a recommendation-bounded VerticalPodAutoscaler for each StatefulSet.
GeneratingPolicy form of the classic auto-vpa rule of the same name.
spec:
# Parity with the classic rules' `synchronize: true` + `generateExisting: true`:
# on GeneratingPolicy both live under spec.evaluation and default to false, so
# omitting them would silently drop behavior the classic policy guarantees.
evaluation:
generateExisting:
enabled: true
synchronize:
enabled: true
matchConstraints:
resourceRules:
- apiGroups: ["apps"]
apiVersions: ["v1"]
operations: ["CREATE", "UPDATE"]
resources: ["statefulsets"]
# PLATFORM ADDITION (the only change from the catalog file): prod ships
# static VPAs for the controllers in these namespaces
# (k8s/providers/hetzner/infrastructure/vertical-pod-autoscalers/), so a
# generated one would put a second VPA on each of those pods.
namespaceSelector:
matchExpressions:
- key: kubernetes.io/metadata.name
operator: NotIn
values:
- kube-system
- kube-public
- kube-node-lease
- kyverno
generate:
- expression: >-
generator.Apply(
string(object.metadata.namespace),
[
{
"apiVersion": dyn("autoscaling.k8s.io/v1"),
"kind": dyn("VerticalPodAutoscaler"),
"metadata": dyn({
"name": string(object.metadata.name) + "-statefulset",
"namespace": string(object.metadata.namespace)
}),
"spec": dyn({
"targetRef": dyn({
"apiVersion": "apps/v1",
"kind": "StatefulSet",
"name": string(object.metadata.name)
}),
"updatePolicy": dyn({"updateMode": "Initial"}),
"resourcePolicy": dyn({
"containerPolicies": [
{
"containerName": dyn("*"),
"controlledResources": dyn(["cpu", "memory"]),
"controlledValues": dyn("RequestsAndLimits"),
"minAllowed": dyn({"cpu": "50m", "memory": "64Mi"}),
"maxAllowed": dyn({"cpu": "3", "memory": "6Gi"})
}
]
})
})
}
]
)
15 changes: 8 additions & 7 deletions k8s/bases/infrastructure/cluster-policies/kustomization.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,14 @@ resources:
- best-practices/add-recommended-labels.yaml
- best-practices/add-resource-defaults.yaml
- best-practices/add-security-context.yaml
# best-practices/auto-vpa.yaml is deliberately NOT registered: it is step 1
# of the shared catalog's two-reconcile move to its GeneratingPolicy form
# (devantler-tech/kyverno-policies#1). Removing the classic policy deletes
# the VPAs it generated; the GeneratingPolicy form is registered in a later
# revision, once no `generate.kyverno.io/policy-name=auto-vpa` VPA remains,
# so the two forms never run at once. The file stays, still tested, as the
# rollback form until that move is complete.
# The shared catalog's GeneratingPolicy form of auto-vpa (#5130,
# devantler-tech/kyverno-policies#1). best-practices/auto-vpa.yaml, the classic
# form, is deliberately NOT registered: it stays, still tested, as the
# rollback form. Never register both, because they name their VPAs
# differently and every workload would get two.
- best-practices/auto-vpa-daemonset.yaml
- best-practices/auto-vpa-deployment.yaml
- best-practices/auto-vpa-statefulset.yaml
- best-practices/audit-privileged-rbac.yaml
- best-practices/disable-default-sa-automount.yaml
- best-practices/disallow-latest-tag.yaml
Expand Down
Loading
Loading