kadet-hub logokadet-hub

Kustomize JSON patch add: fields a Helm chart does not template

Kustomize JSON patch add and Kapitan's mutate for fields charts lack: appProtocol, trafficDistribution, topologySpreadConstraints, annotations, hostNetwork.

A chart can only set the fields its templates write. When a Kubernetes field you need is missing from them, the value you add to values.yaml is silently ignored, unless the chart has a values schema that rejects it. This post sets six such fields from open issues on the rendered manifests, first with a Kustomize patch over helm template, then with Kapitan’s mutate generator. Most of them are RFC 6902 JSON Patch operations, and mutate’s json6902 takes the same operations as Kustomize, so the two versions of each patch read almost the same.

Each patch was rendered from the chart version named in its section with Helm 4.3.1, Kustomize 5.8.3, and Kapitan 0.36.3 with mutate 0.1.0. In every case the Kustomize output and the mutate output set the same value and changed nothing else in the render. The mutate output was installed on kind clusters on Kubernetes 1.37.0 with three worker nodes, and each Kustomize output’s patched object passed a server-side dry run there. Charts from HTTP repositories matched the sha256 their index.yaml lists. The guide Patch a Helm chart compares the places such a patch can live.

The class for each chart pins its version, like this one for Vault, and the target adds mutate as a second compile entry on the chart’s output_path, as the guide shows:

parameters:
  kapitan:
    dependencies:
      - type: helm
        source: https://helm.releases.hashicorp.com
        chart_name: vault
        version: 0.34.1
        output_path: charts/vault
    compile:
      - input_type: helm
        input_paths: [charts/vault]
        output_path: vault
        helm_values_files: [values/vault.yaml]
        helm_params:
          name: vault
          namespace: vault

Vault Helm chart: Service appProtocol

hashicorp/vault-helm#1108, opened in April 2025 and at 9 thumbs up, asks for appProtocol: https on the Vault Service. Without it, Gateway API implementations such as GKE’s Gateway controller talk plain HTTP to a Vault that serves HTTPS, according to the issue and the GKE documentation it links. No maintainer has replied. Chart 0.34.1, released on 13 August 2026, writes the ports in server-service.yaml without it:

  ports:
    - name: {{ include "vault.scheme" . }}
      port: {{ .Values.server.service.port }}
      targetPort: {{ .Values.server.service.targetPort }}
      {{- if and (.Values.server.service.nodePort) (eq (.Values.server.service.type | toString) "NodePort") }}
      nodePort: {{ .Values.server.service.nodePort }}
      {{- end }}
    - name: https-internal
      port: 8201
      targetPort: 8201

The port takes its name from the scheme, https once global.tlsDisable is false, which this values file sets:

global:
  tlsDisable: false
resources:
  - helm.yaml
patches:
  - target:
      kind: Service
      name: vault
    patch: |-
      - op: test
        path: /spec/ports/0/name
        value: https
      - op: add
        path: /spec/ports/0/appProtocol
        value: https
parameters:
  mutations:
    vault-app-protocol:
      match: {apiVersion: v1, kind: Service, name: vault}
      json6902:
        - {op: test, path: /spec/ports/0/name, value: https}
        - {op: add, path: /spec/ports/0/appProtocol, value: https}

test makes both patches refuse to mark a plain HTTP port as https. With the chart’s default tlsDisable: true the first port is named http, and both stopped:

Error: testing value /spec/ports/0/name failed: test failed
mutate: mutation vault-app-protocol: vault/templates/server-service.yaml: json6902 test of '/spec/ports/0/name' failed

The Kustomize target name: vault matched the Service vault only, not vault-internal or vault-agent-injector-svc. The test applied the chart’s Services to kind, without the Vault server, and the stored Service carries the field:

https 8200 appProtocol=https

CloudNativePG cluster chart: topologySpreadConstraints

cloudnative-pg/charts#817, opened in February 2026, asks for topologySpreadConstraints on the Cluster that the cluster chart renders. The CloudNativePG operator supports the field, but the chart’s cluster.yaml only templates affinity. Pull request #818, opened the same day, adds it. It has had no review and now conflicts with the chart’s main branch. In its thread, a user pointed out that the label cnpg.io/cluster alone also matches pooler and Job pods, and suggested adding cnpg.io/podRole: instance to the selector.

On node-local storage that selector has a catch. CloudNativePG creates each instance’s volume through an initdb or join Job, whose pod gets the instance’s constraints but not the label cnpg.io/podRole, so the constraint does not count it. kind’s local-path volumes are bound to the node of the first pod that mounts them. In a run with that selector, the join Job of the third instance landed on the node of the second, the volume followed it, and the third instance stayed Pending:

0/4 nodes are available: 1 node(s) didn't match pod topology spread constraints, 1 node(s) had untolerated taint(s), 2 node(s) didn't match PersistentVolume's node affinity. preemption: 0/4 nodes are available: 1 No preemption victims found for incoming pod, 3 Preemption is not helpful for scheduling.

A selector that counts the Job pods too and leaves out only poolers spread all three instances. Until a chart release has the value, chart 0.9.0 takes this patch:

resources:
  - helm.yaml
patches:
  - target:
      group: postgresql.cnpg.io
      kind: Cluster
      name: pg-cluster
    patch: |-
      - op: add
        path: /spec/topologySpreadConstraints
        value:
          - maxSkew: 1
            topologyKey: kubernetes.io/hostname
            whenUnsatisfiable: DoNotSchedule
            labelSelector:
              matchLabels:
                cnpg.io/cluster: pg-cluster
              matchExpressions:
                - key: cnpg.io/podRole
                  operator: NotIn
                  values: [pooler]
parameters:
  mutations:
    cluster-topology-spread:
      match: {apiVersion: postgresql.cnpg.io/v1, kind: Cluster, name: pg-cluster}
      json6902:
        - op: add
          path: /spec/topologySpreadConstraints
          value:
            - maxSkew: 1
              topologyKey: kubernetes.io/hostname
              whenUnsatisfiable: DoNotSchedule
              labelSelector:
                matchLabels:
                  cnpg.io/cluster: pg-cluster
                matchExpressions:
                  - key: cnpg.io/podRole
                    operator: NotIn
                    values: [pooler]

With operator chart 0.29.1 on the cluster and the release named pg, the Cluster pg-cluster became Ready on a fresh cluster. Its three instance pods carry the constraint and run on three different workers:

pg-cluster-1 node=set-fields-worker2 ready=true spread=kubernetes.io/hostname
pg-cluster-2 node=set-fields-worker ready=true spread=kubernetes.io/hostname
pg-cluster-3 node=set-fields-worker3 ready=true spread=kubernetes.io/hostname

Harbor Helm chart: annotations on the core Deployment

goharbor/harbor-helm#2221, opened in August 2025, asks for annotations and labels on the components' Deployments, beyond the pod annotations the chart takes. One commenter needs an annotation on harbor-core for Reloader. A collaborator suggested a Helm hook, which the reporter cannot use with Argo CD. Three pull requests for the values were closed unmerged by the stale bot. In chart 1.19.4, released on 16 October 2026, core-dpl.yaml writes no annotations into the Deployment’s metadata, and core.podAnnotations, whose comment in values.yaml reads “Additional deployment annotations”, goes to the pod template only:

metadata:
  name: {{ template "harbor.core" . }}
  namespace: {{ .Release.Namespace | quote }}
  labels:
{{ include "harbor.labels" . | indent 4 }}
    component: core
    app.kubernetes.io/component: core

An annotation is a key in a map, which a strategic merge patch and mutate’s merge add without touching other keys:

resources:
  - helm.yaml
patches:
  - patch: |-
      apiVersion: apps/v1
      kind: Deployment
      metadata:
        name: harbor-core
        namespace: harbor
        annotations:
          reloader.stakater.com/auto: "true"
parameters:
  mutations:
    harbor-core-annotations:
      match: {apiVersion: apps/v1, kind: Deployment, name: harbor-core}
      merge:
        metadata:
          annotations:
            reloader.stakater.com/auto: "true"

Harbor generates secrets on every render, so two renders differ in their Secrets and in the Deployments’ checksum annotations. The comparison of the renders left those out. On kind, every Harbor Deployment became Available, and the stored harbor-core carries the annotation:

harbor-core annotations: {"deployment.kubernetes.io/revision":"1","reloader.stakater.com/auto":"true"}

ECK operator chart: ServiceMonitor labels and interval

elastic/cloud-on-k8s#7717, opened in April 2024, asks for labels and the interval on the operator’s ServiceMonitor, because a Prometheus from kube-prometheus-stack selects ServiceMonitors by its release label. A commenter confirmed in August 2026 that 3.5.0 ignores serviceMonitor.labels and keeps a second ServiceMonitor by hand. The chart has no values schema, so the key is silently dropped. In 3.5.0, released on 4 August 2026, service-monitor.yaml renders the ServiceMonitor only with metrics in secure mode, with the chart’s own labels and a fixed interval:

{{- if and .Values.config.metrics.secureMode.enabled .Values.serviceMonitor.enabled }}
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: {{ include "eck-operator.fullname" . }}
  namespace: {{ ternary .Values.serviceMonitor.namespace .Release.Namespace (not (and (.Values.serviceMonitor) (empty .Values.serviceMonitor.namespace))) }}
  labels: {{- include "eck-operator.labels" . | nindent 4 }}
spec:
  namespaceSelector:
    matchNames:
      - {{ .Release.Namespace }}
  selector:
    matchLabels:
      app.kubernetes.io/name: {{ include "eck-operator.name" . }}-metrics-service
      app.kubernetes.io/instance: {{ .Release.Name }}
  endpoints:
  - port: https
    path: /metrics
    scheme: https
    interval: 30s

These values turn the metrics endpoint and secure mode on:

config:
  metrics:
    port: "8080"
    secureMode:
      enabled: true
resources:
  - helm.yaml
patches:
  - target:
      group: monitoring.coreos.com
      kind: ServiceMonitor
      name: elastic-operator
    patch: |-
      - op: add
        path: /metadata/labels/release
        value: kube-prometheus-stack
      - op: test
        path: /spec/endpoints/0/interval
        value: 30s
      - op: replace
        path: /spec/endpoints/0/interval
        value: 60s
parameters:
  mutations:
    eck-servicemonitor:
      match: {apiVersion: monitoring.coreos.com/v1, kind: ServiceMonitor, name: elastic-operator}
      json6902:
        - {op: add, path: /metadata/labels/release, value: kube-prometheus-stack}
        - {op: test, path: /spec/endpoints/0/interval, value: 30s}
        - {op: replace, path: /spec/endpoints/0/interval, value: 60s}

add on /metadata/labels/release adds one key to the existing labels. The test of 30s stops the patch if a release changes the interval. On kind, with the ServiceMonitor CRD of kube-prometheus-stack 92.11.0, the operator became ready and the stored ServiceMonitor has both values:

release=kube-prometheus-stack interval=60s

Temporal Helm chart: Service trafficDistribution

temporalio/helm-charts#897, opened in April 2026, asks for spec.trafficDistribution on the server Services, at least the frontend. Its author sets the field today with a separate server-side apply after each install. No maintainer has replied. Chart 1.7.0, released on 12 September 2026, writes type, ports and selector in server-service.yaml and has no values schema, so server.frontend.service.trafficDistribution is ignored. The chart needs a database to render, and this values file names one that the test never contacted:

server:
  config:
    persistence:
      datastores:
        default:
          sql:
            pluginName: postgres12
            databaseName: temporal
            connectAddr: db.example.internal:5432
        visibility:
          sql:
            pluginName: postgres12
            databaseName: temporal_visibility
            connectAddr: db.example.internal:5432
resources:
  - helm.yaml
patches:
  - target:
      kind: Service
      name: temporal-frontend
    patch: |-
      - op: add
        path: /spec/trafficDistribution
        value: PreferSameZone
parameters:
  mutations:
    temporal-traffic-distribution:
      match: {apiVersion: v1, kind: Service, name: temporal-frontend}
      json6902:
        - {op: add, path: /spec/trafficDistribution, value: PreferSameZone}

PreferClose, the value in the issue, is the older name for PreferSameZone, and the Kubernetes documentation lists it as deprecated. The field is generally available since Kubernetes 1.33. The value PreferSameZone is enabled by default from 1.34, so a 1.33 cluster needs PreferClose. Without a database the Temporal servers cannot start, so the test applied the chart’s Services only, and the API server stored the field:

temporal-frontend trafficDistribution: PreferSameZone

MetalLB Helm chart: the speaker without host networking

metallb/metallb#3005, opened in April 2026, asks for hostNetwork: false on the speaker DaemonSet. The reporter runs FRR on every node next to MetalLB, and FRR does not accept iBGP peering with an address in its own network namespace, so the speaker has to run in the pod network and peer with the node’s FRR. Another user peers the speakers with Calico the same way. The maintainers asked for the use case. Pull request #3099 adds a speaker.hostNetwork value, and on it the maintainer fedepaol wrote “I think it makes sense to document this, mentioning that l2 won’t be able to work.” It is not merged. Chart 0.16.1 hard-codes the field in speaker.yaml:

      hostNetwork: true

By default the chart runs BGP in a separate frr-k8s DaemonSet, which keeps host networking whatever the speaker does. The test therefore uses MetalLB’s native mode, in which the speaker itself speaks BGP:

frrk8s:
  enabled: false
speaker:
  frr:
    enabled: false

Flipping hostNetwork alone is not enough in 0.16.1. The speaker serves its health endpoint on 127.0.0.1 inside its own network namespace (speaker/main.go), and the chart’s probes ask for 127.0.0.1, which the kubelet resolves in the node’s network namespace. With only the field changed, no speaker became ready on kind, and every probe was refused:

Readiness probe failed: Get "http://127.0.0.1:17472/readyz": dial tcp 127.0.0.1:17472: connect: connection refused
Liveness probe failed: Get "http://127.0.0.1:17472/healthz": dial tcp 127.0.0.1:17472: connect: connection refused

Pull request #3099 changes the field only, so it would leave the same probes in place. The patch below also removes both probes. The speaker then has no health check, and the kubelet restarts it only when the process exits:

resources:
  - helm.yaml
patches:
  - target:
      kind: DaemonSet
      name: metallb-speaker
    patch: |-
      - op: test
        path: /spec/template/spec/hostNetwork
        value: true
      - op: replace
        path: /spec/template/spec/hostNetwork
        value: false
      - op: test
        path: /spec/template/spec/containers/0/name
        value: speaker
      - op: remove
        path: /spec/template/spec/containers/0/livenessProbe
      - op: remove
        path: /spec/template/spec/containers/0/readinessProbe
parameters:
  mutations:
    metallb-speaker-pod-network:
      match: {apiVersion: apps/v1, kind: DaemonSet, name: metallb-speaker}
      json6902:
        - {op: test, path: /spec/template/spec/hostNetwork, value: true}
        - {op: replace, path: /spec/template/spec/hostNetwork, value: false}
        - {op: test, path: /spec/template/spec/containers/0/name, value: speaker}
        - {op: remove, path: /spec/template/spec/containers/0/livenessProbe}
        - {op: remove, path: /spec/template/spec/containers/0/readinessProbe}

On kind, a speaker ran on each of the four nodes with its own pod IP. Without a readiness probe, ready means that the container runs:

metallb-speaker-6nsjd hostNetwork=false pod IP is the node IP: no ready=true restarts=0
metallb-speaker-6rr28 hostNetwork=false pod IP is the node IP: no ready=true restarts=0
metallb-speaker-8z7rr hostNetwork=false pod IP is the node IP: no ready=true restarts=0
metallb-speaker-hc622 hostNetwork=false pod IP is the node IP: no ready=true restarts=0

L2 mode answers ARP requests for the service address on the node’s interface, which a speaker in the pod network does not have. With an L2 pool from the kind network and an nginx LoadBalancer Service, the test compared both settings from the host:

speaker in the pod network: the LoadBalancer address from the host: not reachable
speaker in the host network (plain chart): the LoadBalancer address from the host: reachable

The test did not set up a BGP peer, so it shows that the speakers run in the pod network, not that they announce routes over BGP from there.