Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions api/v1alpha1/envoyproxy_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -263,6 +263,11 @@ type EnvoyProxySpec struct {
// MergeType controls how this EnvoyProxy merges with less specific configurations
// in the hierarchy (EnvoyGateway defaults < GatewayClass < Gateway).
// If unset, this EnvoyProxy completely replaces less specific settings.
// When merging, if this EnvoyProxy sets envoyDeployment or envoyDaemonSet and a less
// specific configuration sets the other, the less specific workload is converted to
// the selected kind: its container, pod and name settings are kept, and settings that
// only apply to the previous kind (including its patch, and envoyHpa when switching
// to envoyDaemonSet) are dropped.
// Note: this field has no effect when set in EnvoyGateway's default EnvoyProxySpec.
// +kubebuilder:validation:Enum=Replace;StrategicMerge;JSONMerge
// +optional
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -952,6 +952,11 @@ spec:
MergeType controls how this EnvoyProxy merges with less specific configurations
in the hierarchy (EnvoyGateway defaults < GatewayClass < Gateway).
If unset, this EnvoyProxy completely replaces less specific settings.
When merging, if this EnvoyProxy sets envoyDeployment or envoyDaemonSet and a less
specific configuration sets the other, the less specific workload is converted to
the selected kind: its container, pod and name settings are kept, and settings that
only apply to the previous kind (including its patch, and envoyHpa when switching
to envoyDaemonSet) are dropped.
Note: this field has no effect when set in EnvoyGateway's default EnvoyProxySpec.
enum:
- Replace
Expand Down
2 changes: 1 addition & 1 deletion charts/gateway-helm/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -165,7 +165,7 @@ helm uninstall eg -n envoy-gateway-system
| global.images.envoyGateway.image | string | `nil` | Full image for the Envoy Gateway control plane Deployment installed by this chart. |
| global.images.envoyGateway.pullPolicy | string | `nil` | Image pull policy for the Envoy Gateway control plane Deployment. Default behavior: latest images will be Always else IfNotPresent. |
| global.images.envoyGateway.pullSecrets | list | `[]` | Pull secrets for the Envoy Gateway control plane Deployment. |
| global.images.envoyProxy.image | string | `""` | Full image for the managed Envoy Proxy data plane. This updates the generated `envoyProxy` config and does not change the `envoy-gateway` control plane Deployment image. If not specified, the default image built into `envoy-gateway` is used. |
| global.images.envoyProxy.image | string | `"docker.io/envoyproxy/envoy:distroless-dev"` | Full image for the managed Envoy Proxy data plane. This updates the generated `envoyProxy` config and does not change the `envoy-gateway` control plane Deployment image. Defaults to the Envoy Proxy image this Envoy Gateway release is built for. An image set inline under `config.envoyGateway.envoyProxy` takes precedence. Set to `""` to omit it and fall back to the default image built into `envoy-gateway`. |
| global.images.envoyProxy.pullPolicy | string | `""` | Image pull policy for the managed Envoy Proxy data plane. Default behavior: IfNotPresent. |
| global.images.envoyProxy.pullSecrets | list | `[]` | Pull secrets for the managed Envoy Proxy data plane. |
| global.images.ratelimit.image | string | `"docker.io/envoyproxy/ratelimit:master"` | |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -951,6 +951,11 @@ spec:
MergeType controls how this EnvoyProxy merges with less specific configurations
in the hierarchy (EnvoyGateway defaults < GatewayClass < Gateway).
If unset, this EnvoyProxy completely replaces less specific settings.
When merging, if this EnvoyProxy sets envoyDeployment or envoyDaemonSet and a less
specific configuration sets the other, the less specific workload is converted to
the selected kind: its container, pod and name settings are kept, and settings that
only apply to the previous kind (including its patch, and envoyHpa when switching
to envoyDaemonSet) are dropped.
Note: this field has no effect when set in EnvoyGateway's default EnvoyProxySpec.
enum:
- Replace
Expand Down
26 changes: 15 additions & 11 deletions charts/gateway-helm/templates/_helpers.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -189,29 +189,33 @@ imagePullSecrets: {{ toYaml list }}

{{/*
The default Envoy Gateway configuration.
The envoy-gateway-config ConfigMap merges .Values.config.envoyGateway over this, so inline
settings (such as an Envoy Proxy image under config.envoyGateway.envoyProxy) take precedence.
*/}}
{{- define "eg.default-envoy-gateway-config" -}}
{{- if or .Values.global.images.envoyProxy.image .Values.config.envoyGateway.envoyProxy }}
{{- $envoyProxyBase := .Values.config.envoyGateway.envoyProxy | default dict }}
{{- $imageOverride := dict }}
{{- if .Values.global.images.envoyProxy.image }}
{{- $container := dict "image" (include "eg.envoyProxy.image" .) }}
{{- if .Values.global.images.envoyProxy.pullPolicy }}
{{- $_ := set $container "imagePullPolicy" .Values.global.images.envoyProxy.pullPolicy }}
{{- end }}
{{- $deployment := dict "container" $container }}
{{- $workload := dict "container" $container }}
{{- if or .Values.global.imagePullSecrets .Values.global.images.envoyProxy.pullSecrets }}
{{- $pullSecretsYaml := include "eg.envoyProxy.image.pullSecrets" . }}
{{- $pullSecrets := dict "imagePullSecrets" ($pullSecretsYaml | fromYaml).imagePullSecrets }}
{{- $_ := set $deployment "pod" $pullSecrets }}
{{- $_ := set $workload "pod" $pullSecrets }}
{{- end }}
{{- /* Target the workload kind the user configured; envoyDeployment and envoyDaemonSet are mutually exclusive. */}}
{{- $workloadKind := "envoyDeployment" }}
{{- $userKubernetes := dig "provider" "kubernetes" dict (.Values.config.envoyGateway.envoyProxy | default dict) }}
{{- if hasKey ($userKubernetes | default dict) "envoyDaemonSet" }}
{{- $workloadKind = "envoyDaemonSet" }}
{{- end }}
{{- $kubernetes := dict "envoyDeployment" $deployment }}
{{- $provider := dict "type" "Kubernetes" "kubernetes" $kubernetes }}
{{- $imageOverride = dict "provider" $provider }}
{{- end }}
{{- $merged := mustMergeOverwrite (dict) $envoyProxyBase $imageOverride }}
envoyProxy:
{{ toYaml $merged | indent 2 }}
provider:
type: Kubernetes
kubernetes:
{{ $workloadKind }}:
{{- toYaml $workload | nindent 8 }}
{{- end }}
provider:
type: Kubernetes
Expand Down
7 changes: 4 additions & 3 deletions charts/gateway-helm/values.tmpl.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,10 @@ global:
envoyProxy:
# -- Full image for the managed Envoy Proxy data plane.
# This updates the generated `envoyProxy` config and does not change the `envoy-gateway`
# control plane Deployment image. If not specified, the default image built into
# `envoy-gateway` is used.
image: ""
# control plane Deployment image. Defaults to the Envoy Proxy image this Envoy Gateway
# release is built for. An image set inline under `config.envoyGateway.envoyProxy` takes
# precedence. Set to `""` to omit it and fall back to the default image built into `envoy-gateway`.
image: ${EnvoyProxyImage}
# -- Image pull policy for the managed Envoy Proxy data plane.
# Default behavior: IfNotPresent.
pullPolicy: ""
Expand Down
52 changes: 51 additions & 1 deletion internal/gatewayapi/envoyproxy_merge.go
Original file line number Diff line number Diff line change
Expand Up @@ -90,9 +90,59 @@ func mergeEnvoyProxies(
return override, nil
}

merged, err := utils.Merge(*base, *override, mergeType)
merged, err := utils.Merge(*alignWorkloadKind(base, override), *override, mergeType)
if err != nil {
return nil, err
}
return &merged, nil
}

// alignWorkloadKind returns base with its Envoy workload converted to the kind
// that override selects, so a merge never yields both an envoyDeployment and an
// envoyDaemonSet. The container, pod and name settings carry over to the new
// kind; settings that only apply to the previous kind are dropped, including
// its patch and, when switching to a DaemonSet, envoyHpa.
func alignWorkloadKind(base, override *egv1a1.EnvoyProxy) *egv1a1.EnvoyProxy {
baseKube, overrideKube := kubernetesProvider(base), kubernetesProvider(override)
if baseKube == nil || overrideKube == nil {
return base
}

switch {
case overrideKube.EnvoyDaemonSet != nil && overrideKube.EnvoyDeployment == nil && baseKube.EnvoyDeployment != nil:
aligned := base.DeepCopy()
kube := aligned.Spec.Provider.Kubernetes
if kube.EnvoyDaemonSet == nil {
deployment := kube.EnvoyDeployment
kube.EnvoyDaemonSet = &egv1a1.KubernetesDaemonSetSpec{
Pod: deployment.Pod,
Container: deployment.Container,
Name: deployment.Name,
}
}
kube.EnvoyDeployment = nil
kube.EnvoyHpa = nil
return aligned
case overrideKube.EnvoyDeployment != nil && overrideKube.EnvoyDaemonSet == nil && baseKube.EnvoyDaemonSet != nil:
aligned := base.DeepCopy()
kube := aligned.Spec.Provider.Kubernetes
if kube.EnvoyDeployment == nil {
daemonSet := kube.EnvoyDaemonSet
kube.EnvoyDeployment = &egv1a1.KubernetesDeploymentSpec{
Pod: daemonSet.Pod,
Container: daemonSet.Container,
Name: daemonSet.Name,
}
}
kube.EnvoyDaemonSet = nil
return aligned
}
return base
}

func kubernetesProvider(ep *egv1a1.EnvoyProxy) *egv1a1.EnvoyProxyKubernetesProvider {
if ep.Spec.Provider == nil {
return nil
}
return ep.Spec.Provider.Kubernetes
}
182 changes: 182 additions & 0 deletions internal/gatewayapi/envoyproxy_merge_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import (

"github.com/google/go-cmp/cmp"
"github.com/stretchr/testify/require"
apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1"
gwapiv1 "sigs.k8s.io/gateway-api/apis/v1"

egv1a1 "github.com/envoyproxy/gateway/api/v1alpha1"
Expand Down Expand Up @@ -375,6 +376,159 @@ func TestMergeEnvoyProxyConfigs(t *testing.T) {
},
},
},
{
name: "gatewayclass StrategicMerge daemonset - converts default deployment and drops its patch and hpa",
defaultSpec: &egv1a1.EnvoyProxySpec{
Provider: &egv1a1.EnvoyProxyProvider{
Type: egv1a1.EnvoyProxyProviderTypeKubernetes,
Kubernetes: &egv1a1.EnvoyProxyKubernetesProvider{
EnvoyDeployment: &egv1a1.KubernetesDeploymentSpec{
Replicas: new(int32(2)),
Container: &egv1a1.KubernetesContainerSpec{Image: new("mirror.example/envoy:dev")},
Patch: &egv1a1.KubernetesPatchSpec{
Value: apiextensionsv1.JSON{Raw: []byte(`{"spec":{"replicas":3}}`)},
},
},
EnvoyHpa: &egv1a1.KubernetesHorizontalPodAutoscalerSpec{MaxReplicas: new(int32(5))},
},
},
},
gatewayClassProxy: &egv1a1.EnvoyProxy{
Spec: egv1a1.EnvoyProxySpec{
MergeType: new(egv1a1.StrategicMerge),
Provider: &egv1a1.EnvoyProxyProvider{
Type: egv1a1.EnvoyProxyProviderTypeKubernetes,
Kubernetes: &egv1a1.EnvoyProxyKubernetesProvider{
EnvoyDaemonSet: &egv1a1.KubernetesDaemonSetSpec{
Pod: &egv1a1.KubernetesPodSpec{NodeSelector: map[string]string{"role": "edge"}},
},
},
},
},
},
expectedSpec: &egv1a1.EnvoyProxySpec{
MergeType: new(egv1a1.StrategicMerge),
Provider: &egv1a1.EnvoyProxyProvider{
Type: egv1a1.EnvoyProxyProviderTypeKubernetes,
Kubernetes: &egv1a1.EnvoyProxyKubernetesProvider{
EnvoyDaemonSet: &egv1a1.KubernetesDaemonSetSpec{
Pod: &egv1a1.KubernetesPodSpec{NodeSelector: map[string]string{"role": "edge"}},
Container: &egv1a1.KubernetesContainerSpec{Image: new("mirror.example/envoy:dev")},
},
},
},
},
},
{
name: "gatewayclass JSONMerge daemonset - converts default deployment",
defaultSpec: &egv1a1.EnvoyProxySpec{
Provider: &egv1a1.EnvoyProxyProvider{
Type: egv1a1.EnvoyProxyProviderTypeKubernetes,
Kubernetes: &egv1a1.EnvoyProxyKubernetesProvider{
EnvoyDeployment: &egv1a1.KubernetesDeploymentSpec{
Container: &egv1a1.KubernetesContainerSpec{Image: new("mirror.example/envoy:dev")},
},
},
},
},
gatewayClassProxy: &egv1a1.EnvoyProxy{
Spec: egv1a1.EnvoyProxySpec{
MergeType: new(egv1a1.JSONMerge),
Provider: &egv1a1.EnvoyProxyProvider{
Type: egv1a1.EnvoyProxyProviderTypeKubernetes,
Kubernetes: &egv1a1.EnvoyProxyKubernetesProvider{
EnvoyDaemonSet: &egv1a1.KubernetesDaemonSetSpec{},
},
},
},
},
expectedSpec: &egv1a1.EnvoyProxySpec{
MergeType: new(egv1a1.JSONMerge),
Provider: &egv1a1.EnvoyProxyProvider{
Type: egv1a1.EnvoyProxyProviderTypeKubernetes,
Kubernetes: &egv1a1.EnvoyProxyKubernetesProvider{
EnvoyDaemonSet: &egv1a1.KubernetesDaemonSetSpec{
Container: &egv1a1.KubernetesContainerSpec{Image: new("mirror.example/envoy:dev")},
},
},
},
},
},
{
name: "gateway StrategicMerge deployment - converts default daemonset",
defaultSpec: &egv1a1.EnvoyProxySpec{
Provider: &egv1a1.EnvoyProxyProvider{
Type: egv1a1.EnvoyProxyProviderTypeKubernetes,
Kubernetes: &egv1a1.EnvoyProxyKubernetesProvider{
EnvoyDaemonSet: &egv1a1.KubernetesDaemonSetSpec{
Container: &egv1a1.KubernetesContainerSpec{Image: new("mirror.example/envoy:dev")},
},
},
},
},
gatewayProxy: &egv1a1.EnvoyProxy{
Spec: egv1a1.EnvoyProxySpec{
MergeType: new(egv1a1.StrategicMerge),
Provider: &egv1a1.EnvoyProxyProvider{
Type: egv1a1.EnvoyProxyProviderTypeKubernetes,
Kubernetes: &egv1a1.EnvoyProxyKubernetesProvider{
EnvoyDeployment: &egv1a1.KubernetesDeploymentSpec{Replicas: new(int32(3))},
},
},
},
},
expectedSpec: &egv1a1.EnvoyProxySpec{
MergeType: new(egv1a1.StrategicMerge),
Provider: &egv1a1.EnvoyProxyProvider{
Type: egv1a1.EnvoyProxyProviderTypeKubernetes,
Kubernetes: &egv1a1.EnvoyProxyKubernetesProvider{
EnvoyDeployment: &egv1a1.KubernetesDeploymentSpec{
Replicas: new(int32(3)),
Container: &egv1a1.KubernetesContainerSpec{Image: new("mirror.example/envoy:dev")},
},
},
},
},
},
{
name: "gateway StrategicMerge daemonset - converts gatewayclass deployment",
gatewayClassProxy: &egv1a1.EnvoyProxy{
Spec: egv1a1.EnvoyProxySpec{
Provider: &egv1a1.EnvoyProxyProvider{
Type: egv1a1.EnvoyProxyProviderTypeKubernetes,
Kubernetes: &egv1a1.EnvoyProxyKubernetesProvider{
EnvoyDeployment: &egv1a1.KubernetesDeploymentSpec{
Name: new("edge-proxy"),
Container: &egv1a1.KubernetesContainerSpec{Image: new("mirror.example/envoy:dev")},
},
},
},
},
},
gatewayProxy: &egv1a1.EnvoyProxy{
Spec: egv1a1.EnvoyProxySpec{
MergeType: new(egv1a1.StrategicMerge),
Provider: &egv1a1.EnvoyProxyProvider{
Type: egv1a1.EnvoyProxyProviderTypeKubernetes,
Kubernetes: &egv1a1.EnvoyProxyKubernetesProvider{
EnvoyDaemonSet: &egv1a1.KubernetesDaemonSetSpec{},
},
},
},
},
expectedSpec: &egv1a1.EnvoyProxySpec{
MergeType: new(egv1a1.StrategicMerge),
Provider: &egv1a1.EnvoyProxyProvider{
Type: egv1a1.EnvoyProxyProviderTypeKubernetes,
Kubernetes: &egv1a1.EnvoyProxyKubernetesProvider{
EnvoyDaemonSet: &egv1a1.KubernetesDaemonSetSpec{
Name: new("edge-proxy"),
Container: &egv1a1.KubernetesContainerSpec{Image: new("mirror.example/envoy:dev")},
},
},
},
},
},
}

for _, tc := range testCases {
Expand All @@ -398,3 +552,31 @@ func TestMergeEnvoyProxyConfigs(t *testing.T) {
})
}
}

func TestMergeEnvoyProxyConfigsWorkloadKindDoesNotMutateDefault(t *testing.T) {
defaultSpec := &egv1a1.EnvoyProxySpec{
Provider: &egv1a1.EnvoyProxyProvider{
Type: egv1a1.EnvoyProxyProviderTypeKubernetes,
Kubernetes: &egv1a1.EnvoyProxyKubernetesProvider{
EnvoyDeployment: &egv1a1.KubernetesDeploymentSpec{Replicas: new(int32(2))},
EnvoyHpa: &egv1a1.KubernetesHorizontalPodAutoscalerSpec{MaxReplicas: new(int32(5))},
},
},
}
original := defaultSpec.DeepCopy()
gatewayClassProxy := &egv1a1.EnvoyProxy{
Spec: egv1a1.EnvoyProxySpec{
MergeType: new(egv1a1.StrategicMerge),
Provider: &egv1a1.EnvoyProxyProvider{
Type: egv1a1.EnvoyProxyProviderTypeKubernetes,
Kubernetes: &egv1a1.EnvoyProxyKubernetesProvider{
EnvoyDaemonSet: &egv1a1.KubernetesDaemonSetSpec{},
},
},
},
}

_, err := MergeEnvoyProxyConfigs(defaultSpec, gatewayClassProxy, nil)
require.NoError(t, err)
require.Empty(t, cmp.Diff(original, defaultSpec))
}
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Because the gateway-helm chart now sets a default `global.images.envoyProxy.image`, `global.imageRegistry` and `global.imagePullSecrets` also apply to the managed Envoy Proxy image; mirror the Envoy Proxy image into that registry before upgrading, or set `global.images.envoyProxy.image: ""` to keep the previous behavior.
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Fixed a merged EnvoyProxy (with `mergeType` set) rendering both an Envoy Deployment and DaemonSet when it selects a different workload kind than a less specific configuration; the less specific workload is now converted to the selected kind, keeping its container, pod and name settings.
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
The gateway-helm chart now defaults `global.images.envoyProxy.image` to the Envoy Proxy image the release is built for, so the image appears in rendered manifests for image mirroring and air-gapped tooling; an image set inline under `config.envoyGateway.envoyProxy` now takes precedence over `global.images.envoyProxy.image`, matching the ratelimit and control plane images, and an inline `envoyDaemonSet` receives the global image instead of an extra `envoyDeployment`.
Loading
Loading