Kustomize Best Practices (Part 2)

Part 1 of this series can be found here. Part 3 of this series can be found here.

Overlays can be a base for other overlays

Imagine that you have multiple instances of an application… for example, a green/blue deployment. In such a scenario, you may have some settings that are dev-specific, but common to both green and blue instances — and similarly for prod.

As stated in part 1 of this series, if something is the same between multiple overlays, consider putting it in your base — and so you don’t want to simply create a dev-green overlay and a dev-blue overlay and duplicate settings between them. In this case, however, the settings that are the same are only shared between all dev instances, but not prod… so create a dev-common overlay with a base of “base”, and use dev-common as the base for dev-green and dev-blue overlays (and similarly for prod) like so:

base
├── dev-common
│   ├── dev-green
│   └── dev-blue
└── prod-common
    ├── prod-green
    └── prod-blue

This allows you to specify/patch the common environment-specific configurations in the common overlay for that environment, while allowing green and blue specific configurations for each environment to be specified/patched in the environment specific overlays.

Consider disabling name suffix hashing for secrets and configMaps

By default, Kustomize will append a hash-suffix to configMaps and secrets created with generators. This means that multiple different versions of configMaps and secrets will accumulate, one with each change to the contents of the configMap or secret. If you don’t disable this name suffix hashing, or purge unused secrets/configMaps as necessary, you may run into ResourceQuota issues.

You can easily disable this behavior as follows:

generatorOptions:
 disableNameSuffixHash: true

See this article for details.

Test before patching list items with JSON6902 to make sure that item position within the list hasn’t changed

When using JSON6902 patching, you patch list elements by specifying the zero-based ordinal of the list element to patch. For example, you might use a patch like this:

- op: add
  path: /spec/ports/0/nodePort
  value: 34770

to patch a Service like this:

apiVersion: v1
kind: Service
metadata:
  name: myservice
spec:
  type: NodePort
  externalTrafficPolicy: Local
  ports:
  - name: http
    port: 8080
    protocol: TCP
  selector:
    mylabel: somevalue

However, if someone were to add another port to the ports list before the existing port definition like so:

apiVersion: v1
kind: Service
metadata:
  name: myservice
spec:
  type: NodePort
  externalTrafficPolicy: Local
  ports:
  - name: https
    port: 8443
    protocol: TCP
  - name: http
    port: 8080
    protocol: TCP
  selector:
    mylabel: somevalue

your patch would continue to function… but it would keep patching the first list item and it would not be correct. Unfortunately, this sort of scenario can be extremely difficult to debug!

You can use the test operation to help detect this sort of scenario. If you were to add the following to your patch:

- op: test
  path: /spec/ports/0/name
  value: http

Kustomize would check the name of the first port entry, and if it does not contain the value “http”, then an error will be thrown and the Kustomize operation will fail. This will generally be much preferred to silently patching the wrong list element!

Find more kustomize best practices in part 3.