Kustomize Best Practices (part 3)

Part 1 of this series can be found here. Part 2 of this series can be found here. Part 4 of this series can be found here.

Avoid naming conflicts when deploying multiple applications to the same Namespace with nameSuffix and namePrefix

In general, you should try to avoid deploying multiple applications (or application instances) to the same Namespace. To clarify, deploying different components of a single application instance in the same Namespace is often reasonable — but deploying multiple application instances in the same Namespace is often going to be more ill-advised.

However, sometimes we don’t have the the choices we might prefer. If you must deploy multiple applications to the same Namespace, you may be able to use namePrefix and/or nameSuffix to avoid naming conflicts. namePrefix and nameSuffix will automatically add a prefix or suffix (respectively) to all objects deployed through the kustomization.

For example, if you have the following base (/base):

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app.kubernetes.io/name: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: nginx
  template:
    metadata:
      labels:
        app.kubernetes.io/name: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.23.3
        ports:
        - containerPort: 80
---
kind: Service
metadata:
  name: nginx-service
spec:
  selector:
    app.kubernetes.io/name: nginx
  ports:
    - protocol: TCP
      port: 80

And following 2 application overlays:

../overlays/instance1

bases:
- ../base

namespace: my-shared-namespace

# content unique to application instance 1 goes here

../overlays/instance2

bases:
- ../base

namespace: my-shared-namespace

# content unique to application instance 2 goes here

Applying both of these overlays would result in a single Deployment named nginx-deployment and a single Service named nginx-service.

However, adding a nameSuffix as shown below will result in 2 Deployments (named nginx-deployment-instance1 and nginx-deployment-instance2) and 2 Services (named nginx-service-instance1 and nginx-service-instance2):

../overlays/instance1

bases:
- ../base

nameSuffix: -instance1
namespace: my-shared-namespace

# content unique to application instance 1 goes here

../overlays/instance2

bases:
- ../base

nameSuffix: -instance2
namespace: my-shared-namespace

# content unique to application instance 2 goes here

Mission accomplished? Unfortunately, probably not (yet). There can be a few snags that arise.

For example, if you use SealedSecrets and seal a secret called “mysecret” with the default options, and add it to the base, when you run kustomize the name of the secret would be changed to “mysecret-instance1” and “mysecret-instance2”. The SealedSecret, however, is (usually) tied to the name that it was generated with — and as a result, the unsealing process will fail and the corresponding Secret will not be unsealed. You could specify --scope namespace-wideto avoid this.

More commonly, however, the issue is that both of the Deployments will have the same labels and labelSelectors, and so will the Services. As a reult, Pods created by either Deployment (or more precisely by the ReplicaSet created by the Deployment) will be considered endpoints for both Services, which is usually not what you want. To avoid this, keep reading.

Ensure that multiple applications remain isolated in the same Namespace with commonLabels

The kustomize commonLabels feature when used in an kustomization adds the specified label to all labels and labelSelectors of all objects being deployed.

For example:

../overlays/instance1

bases:
- ../base

nameSuffix: -instance1
namespace: my-shared-namespace

commonLabels:
  app.kubernetes.io/instance: instance1

# content unique to application instance 1 goes here

generates the following yaml:

apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/instance: instance1 <-- here
  name: nginx-service-instance1
spec:
  ports:
  - port: 80
    protocol: TCP
  selector:
    app.kubernetes.io/instance: instance1 <-- and here
    app.kubernetes.io/name: nginx
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app.kubernetes.io/name: nginx
    app.kubernetes.io/instance: instance1 <-- and here
  name: nginx-deployment-instance1
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: nginx
      app.kubernetes.io/instance: instance1 <-- and here
  template:
    metadata:
      labels:
        app.kubernetes.io/name: nginx
        app.kubernetes.io/instance: instance1 <-- and here
    spec:
      containers:
      - image: nginx:1.23.3
        name: nginx
        ports:
        - containerPort: 80

As you can see, the commonLabels are added to both the labels and the labelSelectors, which (as long as the aggregate of the commonLabels values are unique between your overlays) will insure that your Services do not include Pods that they shouldn’t.

Find more kustomize best practices in part 4.