kubectl drain stuck — Cannot evict pod as it would violate the pod's disruption budget

Draining an EKS node for maintenance with kubectl drain <node> --ignore-daemonsets --delete-emptydir-data got stuck. The event on the pod showed: Cannot evict pod as it would violate the pod's disruption budget.

Root cause: The affected Deployment had a PodDisruptionBudget set to minAvailable: 1, but only 1 replica was running. Evicting it would bring available pods to 0, violating the PDB. Kubernetes correctly blocked the drain — but this left the node un-drainable until the situation was resolved.

Fix:

Identify which PDB is blocking:

kubectl get pdb -A
kubectl describe pdb my-app-pdb -n default

Check current replica count:

kubectl get deploy my-app -n default

Temporarily scale up to 2 replicas so eviction doesn't violate the PDB:

kubectl scale deploy my-app --replicas=2 -n default
 
# Wait for the new pod to be Running
kubectl get pods -n default -w

Now drain the node:

kubectl drain <node-name> --ignore-daemonsets --delete-emptydir-data

After the node is drained and maintenance is done, scale back down:

kubectl scale deploy my-app --replicas=1 -n default

If you need to bypass the PDB in an emergency (not recommended for production):

kubectl drain <node-name> --ignore-daemonsets --delete-emptydir-data --disable-eviction

Lesson: Before draining nodes, check that all PDB-protected Deployments have at least minAvailable + 1 replicas running — otherwise the drain will block indefinitely with no clear error in the drain output.