🎉 DevOps Interview Prep Bundle is live — 1000+ Q&A across 20 topicsGet it →
All Articles

Kubernetes HPA Not Scaling Down: Fix in 5 Minutes

HorizontalPodAutoscaler scales up fine but never scales back down, leaving you paying for pods you don't need? Here is exactly how to diagnose stabilization windows, metric server lag, and pod disruption budgets blocking scale-down.

Shubham4 min read
Share:Tweet

HPA scaling up is the behavior everyone tests; scaling down is the behavior everyone forgets to verify — and when it silently doesn't happen, you're just paying for idle capacity indefinitely with no error to alert you.

Step 1: Check What the HPA Actually Sees

bash
kubectl get hpa myapp -n production
# NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS
# myapp   Deployment/myapp   15%/50%   3         20        12    ← low utilization, still 12 replicas
 
kubectl describe hpa myapp -n production

If the target metric is well below threshold but replicas haven't dropped, the HPA has already decided to scale down — something else is blocking it, or it hasn't decided yet due to timing.

Cause 1: Stabilization Window Delaying Scale-Down (Most Common, and By Design)

bash
kubectl get hpa myapp -n production -o yaml | grep -A 10 behavior
yaml
spec:
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300    # Default 300s (5 min)
      policies:
        - type: Percent
          value: 10
          periodSeconds: 60

The default 5-minute stabilization window is intentional — it prevents flapping (scale down, then immediately scale back up on the next traffic blip). HPA looks at the maximum recommended replica count over the stabilization window, not the current value, so a brief dip in traffic doesn't trigger scale-down until utilization stays low for the full window.

bash
# Check recent HPA events to see if it's actively working through the window
kubectl describe hpa myapp -n production | grep -A 20 "Events:"

This usually isn't a bug — but if scale-down feels too slow for your workload:

yaml
spec:
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 60    # Shorter window if you're confident about traffic patterns
      policies:
        - type: Percent
          value: 25              # Scale down more aggressively per step
          periodSeconds: 60

Cause 2: PodDisruptionBudget Blocking Pod Termination

bash
kubectl get pdb -n production
# NAME         MIN AVAILABLE   MAX UNAVAILABLE   ALLOWED DISRUPTIONS
# myapp-pdb    12                                0    ← blocks ANY scale-down

If minAvailable is set to (or near) your current replica count, the HPA can decide to scale down but Kubernetes' eviction logic refuses because the PDB doesn't allow taking any pods away — this shows up as HPA "wanting" fewer replicas that never actually get removed.

bash
kubectl describe pdb myapp-pdb -n production

Fix — set a PDB that allows genuine headroom for scale-down:

yaml
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: myapp-pdb
spec:
  minAvailable: 3    # Not tied to current replica count — your actual floor
  selector:
    matchLabels:
      app: myapp

Cause 3: Metrics Server Reporting Stale or Missing Data

bash
kubectl top pods -n production -l app=myapp
# If this returns nothing or stale-looking numbers, metrics-server itself has a problem
 
kubectl get apiservice v1beta1.metrics.k8s.io -o yaml | grep -A 5 status

HPA behavior when metrics are unavailable is conservative by design — it will not scale down (or up) confidently without fresh metrics, and won't necessarily surface this as an obvious error.

bash
kubectl logs -n kube-system -l k8s-app=metrics-server --tail=50

Fix — restart or fix metrics-server if it's degraded:

bash
kubectl rollout restart deployment/metrics-server -n kube-system

Cause 4: Custom/External Metrics Source Not Reporting Correctly

If your HPA scales on a custom metric (queue depth, request rate from Prometheus Adapter) rather than CPU/memory, a stale or stuck metrics pipeline can pin the HPA's view of demand artificially high.

bash
kubectl get hpa myapp -n production -o jsonpath='{.spec.metrics}'
# Confirm which metric source is actually driving decisions
 
# For Prometheus Adapter-based custom metrics:
kubectl get --raw "/apis/custom.metrics.k8s.io/v1beta1/namespaces/production/pods/*/http_requests_per_second" | jq .

Fix — verify the underlying query and adapter are healthy:

bash
kubectl logs -n monitoring -l app=prometheus-adapter --tail=50

Cause 5: minReplicas Set Higher Than Actually Needed

bash
kubectl get hpa myapp -n production -o jsonpath='{.spec.minReplicas}'
# 12    ← if this is your minReplicas, the HPA is working correctly —
#         it literally cannot go below this floor

This isn't a bug at all — someone set a floor, intentionally or from a stale config copy-pasted between environments.

Fix — if the floor is outdated, lower it and let the HPA do its job:

bash
kubectl patch hpa myapp -n production --type merge -p '{"spec":{"minReplicas": 3}}'

Diagnostic Checklist

bash
kubectl describe hpa myapp -n production           # current targets, events, scaling decisions
kubectl get hpa myapp -n production -o yaml | grep -A 10 behavior   # stabilization window config
kubectl get pdb -n production                       # disruption budget blocking eviction
kubectl top pods -n production -l app=myapp          # confirm metrics are actually fresh
kubectl get hpa myapp -n production -o jsonpath='{.spec.minReplicas}'   # confirm the floor

More Kubernetes cost/scaling troubleshooting? Read our Kubernetes HPA not scaling fix and Kubernetes cost optimization strategies.

🔧

Today I Fixed

Short real fixes from production — posted daily

Browse fixes
Newsletter

Stay ahead of the curve

Get the latest DevOps, Kubernetes, AWS, and AI/ML guides delivered straight to your inbox. No spam — just practical engineering content.

Related Articles

Comments