My Kubernetes Application Is Not Seeing ConfigMap or Secret Updates

Paul Dally
2 min readMay 17, 2022
Update stamp: https://www.onlygfx.com/update-stamp-png-transparent/, ConfigMap and Secret icons: https://github.com/kubernetes/community/blob/master/icons/visio/kubernetes-visio-stencil.vssx

Unlike ConfigMaps and Secrets consumed as environment variables (which are not updated automatically and require a Pod restart), mounted ConfigMaps and Secrets are updated automatically. So why isn’t your application seeing those updates? Two reasons are most common:

  • The ConfigMap or Secret is being used as a subPath volumeMount. These mounts are not updated automatically.
  • The application is coded to cache the ConfigMap or Secret at container startup and does not re-read the file whenever it changes.

If you are deploying a Pod directly (are you sure you really need to do this?), then you might have no other option but to delete and redeploy. For Pods created by Deployments, StatefulSets or DaemonSets, however, there are a handful of better solutions:

  1. Delete the Pods manually (perhaps one at a time) causing the Deployment /ReplicaSet (or Statefulset or DaemonSet). The restarted Pod will have the updated value when it starts, and if applicable, the application will cache that updated value. This is usually not ideal, because of the manual nature of the approach.
  2. If your Pod was created by a Deployment, StatefulSet or Daemonset, and you have permission, execute kubectl -n namespace rollout restart kind/name .
  3. If you don’t have permission to do a rollout restart, you can modify the Pod spec with a “trivial” change — for example, adding an environment variable which is not used by the application that you can change to force the rollout to occur. For example:
apiVersion: apps/v1
kind: Deployment
metadata:
name: helloworld-deployment
spec:
template:
spec:
containers:
- name: hello-world
image: helloworld-webserver:v1.0.0
env:
- name: FORCE_ROLLOUT
value: "1111"
...

Simply change the value of the FORCE_ROLLOUT environment variable to a different value than it previously was, and deploy this change along with the ConfigMap or Secret updates. Kubernetes will automatically terminate/recreate the Pods, adhering to any rollout strategies that might be applicable, and your application will be aware of your new ConfigMap and/or Secret values!

--

--

Paul Dally

AVP, IT Foundation Platforms Architecture at Sun Life Financial. Views & opinions expressed are my own, not necessarily those of Sun Life