What I Was Seeing
kubectl get pods -n production
NAME READY STATUS RESTARTS AGE
api-server-7d9f-xkp2n 0/1 ContainerCreating 0 8m8 minutes in ContainerCreating. No restarts. kubectl describe was the first stop.
Step 1: Always Start With Describe
kubectl describe pod api-server-7d9f-xkp2n -n productionScroll to the Events section at the bottom — this is where the actual error lives.
Cause 1: Volume Mount Failure (My Case)
My events showed:
Warning FailedMount 5m kubelet
MountVolume.SetUp failed for volume "config-secret" :
secret "app-config-v2" not found
The Deployment was referencing a Secret that didn't exist in the production namespace. The old secret was named app-config and I had updated the Deployment to use app-config-v2 without creating it first.
Fix:
# Check what secrets exist in the namespace
kubectl get secrets -n production
# Create the missing secret
kubectl create secret generic app-config-v2 \
--from-file=config.yaml=./config/production.yaml \
-n productionPod moved to Running within 30 seconds.
Cause 2: Image Pull Failure
Events show:
Warning Failed 2m kubelet
Failed to pull image "myregistry.io/api:v2.1.0":
rpc error: unauthorized: authentication required
This means the node can't authenticate to the image registry.
# Check if imagePullSecret is configured
kubectl get pod api-server-7d9f-xkp2n -n production -o yaml | grep imagePullSecrets -A 3
# Create the registry secret if missing
kubectl create secret docker-registry regcred \
--docker-server=myregistry.io \
--docker-username=your-user \
--docker-password=your-token \
-n production
# Reference it in your Deployment spec
# spec:
# imagePullSecrets:
# - name: regcredCause 3: CNI / Network Plugin Issue
Events show:
Warning NetworkNotReady 3m kubelet
runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady
message:Network plugin returns error: cni plugin not initialized
This is a node-level problem. The CNI plugin (Calico, Cilium, Flannel) is not running properly on that specific node.
# Check CNI pods on the affected node
kubectl get pods -n kube-system -o wide | grep <node-name>
# If CNI pod is crashing, check its logs
kubectl logs -n kube-system calico-node-xxxxx
# Quick fix: drain and uncordon the node
kubectl drain <node-name> --ignore-daemonsets --delete-emptydir-data
kubectl uncordon <node-name>Cause 4: ConfigMap Referenced But Missing
Same pattern as missing Secret:
Warning FailedMount 4m kubelet
MountVolume.SetUp failed for volume "app-config" :
configmap "app-config-v2" not found
kubectl get configmaps -n production
kubectl create configmap app-config-v2 --from-file=./config/ -n productionCause 5: PersistentVolumeClaim Not Bound
Warning FailedMount 6m kubelet
Unable to attach or mount volumes: unmounted volumes=[data],
unattached volumes=[data]: timed out waiting for the condition
kubectl get pvc -n production
# If STATUS shows Pending instead of Bound:
kubectl describe pvc data-pvc -n productionCommon reasons PVC is stuck Pending: no StorageClass defined, StorageClass doesn't exist, or PV provisioner not running.
The Debug Checklist
# 1. What do events say?
kubectl describe pod <pod-name> -n <namespace>
# 2. Is the image pullable?
kubectl get pod <pod-name> -o yaml | grep image:
# 3. Do all referenced Secrets/ConfigMaps exist?
kubectl get secrets,configmaps -n <namespace>
# 4. Are PVCs bound?
kubectl get pvc -n <namespace>
# 5. Is the node healthy?
kubectl get nodes
kubectl describe node <node-name> | grep -A 10 ConditionsContainerCreating almost always resolves to a missing dependency (Secret, ConfigMap, PVC) or a network/image issue. Describe output tells you exactly which one within seconds.