🎉 DevOps Interview Prep Bundle is live — 1000+ Q&A across 20 topicsGet it →
All Articles

Kubernetes StatefulSet Pod Not Starting: 6 Causes and Fixes

StatefulSet pods stuck in Pending, Init, or CrashLoopBackOff? This guide covers the 6 most common causes — PVC binding, headless service missing, init container failures, pod identity issues — with exact kubectl commands to diagnose and fix each.

Shubham4 min read
Share:Tweet

StatefulSets are harder to debug than Deployments because they have extra requirements: stable pod identity, ordered startup, per-pod PVCs, and a headless service. When a StatefulSet pod won't start, the cause is usually one of six things.

Quick Diagnosis

bash
# Check pod status
kubectl get pods -n <namespace> -l app=<your-statefulset>
 
# Check StatefulSet status
kubectl describe statefulset <name> -n <namespace>
 
# Check events
kubectl get events -n <namespace> --sort-by=.lastTimestamp | tail -20

Cause 1: PVC Not Bound

The most common cause. StatefulSet pods wait forever if their PVC is not bound.

Symptoms:

NAME       READY   STATUS    RESTARTS   AGE
postgres-0 0/1     Pending   0          10m

Diagnose:

bash
kubectl get pvc -n <namespace>
# Look for STATUS = Pending instead of Bound

Common reasons PVC stays Pending:

No default StorageClass:

bash
kubectl get storageclass
# If no StorageClass has (default) annotation, PVC will never bind
 
# Fix: set a default
kubectl patch storageclass gp2 -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'

WaitForFirstConsumer binding mode:

bash
kubectl describe pvc <pvc-name> -n <namespace>
# Events: waiting for first consumer to be created before binding
 
# This is normal — the PVC binds when the pod schedules
# If pod won't schedule, the PVC waits forever
# Check node selector or taints blocking pod placement
kubectl describe pod <pod-name> -n <namespace> | grep -A 5 "Events:"

StorageClass provisioner not running:

bash
kubectl get pods -n kube-system | grep provisioner
# If EBS CSI driver, check:
kubectl get pods -n kube-system | grep ebs-csi

Cause 2: Headless Service Missing or Wrong

StatefulSets require a headless service (clusterIP: None) for stable DNS names. If the service is missing or wrong, pods may not start or DNS won't work.

Check:

bash
kubectl get svc -n <namespace>
# Look for your headless service — CLUSTER-IP should be "None"
 
kubectl describe statefulset <name> -n <namespace> | grep serviceName
# This must match an existing headless service

Fix — create the headless service:

yaml
apiVersion: v1
kind: Service
metadata:
  name: postgres-headless
  namespace: production
spec:
  clusterIP: None          # This is what makes it headless
  selector:
    app: postgres
  ports:
  - port: 5432
    name: postgres
bash
kubectl apply -f headless-svc.yaml

Verify DNS works:

bash
# From another pod in the same namespace
kubectl run -it --rm debug --image=busybox --restart=Never -- nslookup postgres-0.postgres-headless.production.svc.cluster.local

Cause 3: Init Container Failing

StatefulSet pods with init containers that fail will stay in Init:0/1 or Init:CrashLoopBackOff.

Diagnose:

bash
kubectl describe pod postgres-0 -n <namespace>
# Look for init container status
 
kubectl logs postgres-0 -n <namespace> -c <init-container-name>
# Check init container logs

Common init container failures:

bash
# Permission fix init container failing
# Check if the target volume is read-only or wrong owner
 
# Database migration init container failing
# Check database connectivity from init container
kubectl exec -it postgres-0 -c <init-container> -- /bin/sh
# Try connecting to dependencies manually

Cause 4: Ordered Startup Blocking

StatefulSets start pods in order: pod-0 must be Running and Ready before pod-1 starts. If pod-0 has a readiness probe that never passes, pod-1 never starts.

Symptoms:

postgres-0   1/1     Running   0    5m
postgres-1   0/1     Init:0/1  0    5m   # Stuck waiting
postgres-2   0/1     Pending   0    0s   # Not even started

Wait — pod-0 shows Running above, so why is pod-1 stuck?

Check if pod-0 is actually Ready:

bash
kubectl get pod postgres-0 -n <namespace> -o jsonpath='{.status.conditions[?(@.type=="Ready")].status}'
# If False, pod-0 is Running but not Ready

Fix readiness probe:

yaml
readinessProbe:
  exec:
    command:
    - /bin/sh
    - -c
    - pg_isready -U postgres
  initialDelaySeconds: 30   # Give DB time to start
  periodSeconds: 10
  failureThreshold: 6        # Allow more failures before marking unready

Cause 5: Resource Insufficient on Node

StatefulSet pods can get stuck in Pending if no node has enough CPU/memory.

bash
kubectl describe pod postgres-0 -n <namespace> | grep -A 10 "Events:"
# Look for: Insufficient cpu, Insufficient memory
 
# Check node capacity
kubectl describe nodes | grep -A 5 "Allocated resources:"
 
# Fix: reduce resource requests or add nodes

StatefulSets and node affinity: if you have nodeAffinity or topologySpreadConstraints, check those too:

bash
kubectl describe pod postgres-0 -n <namespace> | grep -A 20 "Node-Selectors:"

Cause 6: PodManagementPolicy Blocking Parallel Operations

By default StatefulSets use OrderedReady — sequential startup. If you need parallel startup, you must explicitly set Parallel.

bash
kubectl get statefulset <name> -n <namespace> -o jsonpath='{.spec.podManagementPolicy}'

For databases, OrderedReady is usually correct. For stateless-ish workloads using StatefulSet just for stable identity, Parallel is faster:

yaml
spec:
  podManagementPolicy: Parallel   # Start all pods simultaneously

Full Diagnostic Checklist

bash
# 1. StatefulSet conditions
kubectl describe statefulset <name> -n <namespace>
 
# 2. Pod status for all replicas
kubectl get pods -n <namespace> -l app=<name> -o wide
 
# 3. PVC status
kubectl get pvc -n <namespace> | grep <name>
 
# 4. Headless service
kubectl get svc -n <namespace> | grep None
 
# 5. Events
kubectl get events -n <namespace> --sort-by=.lastTimestamp | grep <name>
 
# 6. Pod logs (including previous container)
kubectl logs <pod-name> -n <namespace> --previous
 
# 7. Init container logs
kubectl logs <pod-name> -n <namespace> -c <init-container-name>
 
# 8. Node resources
kubectl describe nodes | grep -E "Name:|Allocated"

Most Common Fix (It's Usually PVC)

In practice, 70% of StatefulSet startup issues come down to PVCs not binding. Check kubectl get pvc -n <namespace> first before debugging anything else.


More Kubernetes troubleshooting? Read our PVC stuck in Pending fix and Kubernetes rollout stuck fix.

🔧

Today I Fixed

Short real fixes from production — posted daily

Browse fixes
Newsletter

Stay ahead of the curve

Get the latest DevOps, Kubernetes, AWS, and AI/ML guides delivered straight to your inbox. No spam — just practical engineering content.

Related Articles

Comments