🎉 DevOps Interview Prep Bundle is live — 1000+ Q&A across 20 topicsGet it →
All Articles

kubectl port-forward Connection Refused: Fix in 5 Minutes

kubectl port-forward starts fine but every connection gets refused or hangs? Here is exactly how to diagnose pod readiness, wrong target port, network policy, and CNI causes behind port-forward failures.

Shubham4 min read
Share:Tweet

port-forward connecting successfully at the kubectl level but every actual request failing is one of the more confusing Kubernetes errors, because the command itself doesn't error — it just silently proxies to nothing. Here is how to find the real cause.

Step 1: Confirm What port-forward Is Actually Doing

bash
kubectl port-forward pod/myapp-7f9c-abc12 8080:80 -n production
# Forwarding from 127.0.0.1:8080 -> 80
# Forwarding from [::1]:8080 -> 80
 
# In another terminal:
curl -v http://localhost:8080/health
# curl: (56) Recv failure: Connection reset by peer

port-forward reporting success just means it opened a tunnel — it does not verify anything is actually listening on the target port inside the pod. That's the gap that causes most of these failures.

Cause 1: Nothing Actually Listening on the Target Port Inside the Container

bash
# Check what the container is actually listening on
kubectl exec myapp-7f9c-abc12 -n production -- netstat -tlnp
# or if netstat isn't available:
kubectl exec myapp-7f9c-abc12 -n production -- ss -tlnp

If your app is listening on 8080 inside the container but you forwarded 80, every connection gets refused — port-forward maps to whatever port you tell it, it does not know what your app actually uses.

Fix — match the forward to the container's actual listening port:

bash
# Check the container spec for the real containerPort
kubectl get pod myapp-7f9c-abc12 -n production -o jsonpath='{.spec.containers[0].ports}'
 
kubectl port-forward pod/myapp-7f9c-abc12 8080:8080 -n production    # local:container

Cause 2: Pod Is Not Actually Ready

bash
kubectl get pod myapp-7f9c-abc12 -n production
# NAME              READY   STATUS    RESTARTS
# myapp-7f9c-abc12  0/1     Running   0    ← Running but not Ready

port-forward works against Running pods regardless of readiness — it does not check readiness probes. A pod that's up but still initializing (database connection pending, cache warming) will accept the TCP connection but refuse or hang on the actual request.

bash
kubectl describe pod myapp-7f9c-abc12 -n production | grep -A 5 "Readiness"
kubectl logs myapp-7f9c-abc12 -n production --tail=50

Fix — wait for readiness, or forward to a Service instead of a specific pod so you always hit a ready one:

bash
kubectl port-forward svc/myapp 8080:80 -n production
# Services only route to Ready endpoints — safer than targeting a pod directly

Cause 3: Forwarding to the Wrong Pod (Stale Pod Name After a Redeploy)

bash
# You copy-pasted a pod name from a previous session
kubectl port-forward pod/myapp-7f9c-abc12 8080:80 -n production
# error: unable to forward port because pod is not running. Current status=Terminated

Pod names change on every rollout — this is the most common "works yesterday, not today" cause.

Fix — get the current pod name, or better, forward to the Service/Deployment which always resolves correctly:

bash
kubectl get pods -n production -l app=myapp    # get the current pod name
# Or skip pod names entirely:
kubectl port-forward deployment/myapp 8080:80 -n production

Cause 4: NetworkPolicy Blocking the kube-apiserver's Path to the Pod

port-forward traffic goes through the kube-apiserver to the kubelet to the pod — it is not a direct connection from your laptop. A restrictive NetworkPolicy on the pod's namespace can block this path even though it looks like a local tunnel.

bash
kubectl get networkpolicy -n production
kubectl describe networkpolicy default-deny -n production
yaml
# A default-deny-all ingress policy blocks port-forward traffic too,
# since it counts as ingress to the pod
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny
spec:
  podSelector: {}
  policyTypes: ["Ingress"]
  # No ingress rules = deny all, including port-forward's path through kubelet

This is uncommon (kubelet-originated traffic is often exempted depending on CNI), but worth checking on CNIs that don't special-case it, particularly Calico with strict enforcement.

Cause 5: kubectl Version Skew With the Cluster

bash
kubectl version --short
# Client Version: v1.27.x
# Server Version: v1.30.x    ← more than 2 minor versions apart can cause subtle proxy issues

Fix — keep kubectl within the supported skew (one minor version either direction is safest):

bash
# Update kubectl to match server minor version
curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"

Diagnostic Checklist

bash
kubectl get pod POD_NAME -n NS                                    # confirm Running AND Ready
kubectl get pod POD_NAME -n NS -o jsonpath='{.spec.containers[0].ports}'   # confirm the real port
kubectl exec POD_NAME -n NS -- ss -tlnp                            # confirm something's listening
kubectl get networkpolicy -n NS                                    # check for blocking policies
kubectl port-forward svc/SERVICE_NAME LOCAL:SVC_PORT -n NS         # prefer Service over Pod

More Kubernetes troubleshooting? Read our Kubernetes Node NotReady fix and kubectl exec permission denied fix.

🔧

Today I Fixed

Short real fixes from production — posted daily

Browse fixes
Newsletter

Stay ahead of the curve

Get the latest DevOps, Kubernetes, AWS, and AI/ML guides delivered straight to your inbox. No spam — just practical engineering content.

Related Articles

Comments