🎉 DevOps Interview Prep Bundle is live — 1000+ Q&A across 20 topicsGet it →
All Articles

What is an Admission Webhook in Kubernetes Explained

Admission webhooks intercept every Kubernetes API request before it's persisted. Learn how mutating and validating webhooks work, with real examples from OPA, Istio, and custom webhooks.

DevOpsBoysJun 6, 20264 min read
Share:Tweet

Every time you run kubectl apply, your request doesn't go straight to etcd. It passes through a pipeline of admission controllers — and admission webhooks are custom plugins that let you intercept, modify, or reject any Kubernetes resource before it's created.


The Kubernetes Request Flow

kubectl apply
    ↓
API Server authentication + authorization
    ↓
Mutating Admission Webhooks  ← modify the resource
    ↓
Object Schema Validation
    ↓
Validating Admission Webhooks ← accept or reject
    ↓
Persisted to etcd
    ↓
Resource created

Two types. They run in this exact order:

  1. Mutating webhooks — can modify the incoming object (add labels, inject sidecars, set defaults)
  2. Validating webhooks — can only accept or reject, not modify

Real-World Examples

ToolTypeWhat it does
IstioMutatingAuto-injects the Envoy sidecar into every pod
OPA GatekeeperValidatingRejects pods that violate policies (no resource limits, etc.)
KyvernoBothMutates + validates based on ClusterPolicy rules
cert-managerMutatingInjects TLS certificates into pods
Vault AgentMutatingInjects Vault agent sidecar for secret retrieval

How Mutating Webhooks Work

When you deploy to a namespace with Istio, the Istio webhook mutates your Pod spec:

yaml
# You apply this:
apiVersion: v1
kind: Pod
spec:
  containers:
  - name: my-app
    image: myapp:latest
 
# Istio webhook transforms it to this:
apiVersion: v1
kind: Pod
spec:
  containers:
  - name: my-app
    image: myapp:latest
  - name: istio-proxy           # ← injected by mutating webhook
    image: docker.io/istio/proxyv2:1.20.0
    # ... full Envoy config
  initContainers:
  - name: istio-init            # ← also injected
    # ... iptables setup

You never wrote that — the webhook added it automatically.


How Validating Webhooks Work

OPA Gatekeeper's validating webhook rejects resources that violate policies:

bash
# Try to create a pod without resource limits:
kubectl apply -f pod-no-limits.yaml
 
# Error from server: admission webhook "validation.gatekeeper.sh" denied the request:
# [must-have-resource-limits] Container 'nginx' is missing CPU limits.
# Set resources.limits.cpu

The request never reached etcd. The webhook returned a rejection.


Webhook Configuration

Kubernetes knows about webhooks via MutatingWebhookConfiguration and ValidatingWebhookConfiguration objects:

yaml
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
  name: my-custom-validator
webhooks:
- name: validate-pods.mycompany.com
  clientConfig:
    service:
      name: my-webhook-service     # webhook server running in cluster
      namespace: webhook-system
      path: "/validate"
    caBundle: <base64-encoded-ca>
  rules:
  - operations: ["CREATE", "UPDATE"]
    apiGroups: [""]
    apiVersions: ["v1"]
    resources: ["pods"]
  admissionReviewVersions: ["v1"]
  sideEffects: None
  failurePolicy: Fail              # reject request if webhook is down
  namespaceSelector:
    matchLabels:
      webhook-enabled: "true"      # only apply to labeled namespaces

Building a Simple Custom Webhook

python
# webhook_server.py — minimal validating webhook
from flask import Flask, request, jsonify
import base64
import json
 
app = Flask(__name__)
 
 
@app.route("/validate", methods=["POST"])
def validate():
    review = request.json
    uid = review["request"]["uid"]
    resource = review["request"]["object"]
 
    # Rule: reject pods with 'latest' image tag
    containers = resource.get("spec", {}).get("containers", [])
    violations = []
 
    for container in containers:
        if container["image"].endswith(":latest") or ":" not in container["image"]:
            violations.append(
                f"Container '{container['name']}' uses ':latest' tag. "
                f"Pin to a specific version."
            )
 
    allowed = len(violations) == 0
 
    return jsonify({
        "apiVersion": "admission.k8s.io/v1",
        "kind": "AdmissionReview",
        "response": {
            "uid": uid,
            "allowed": allowed,
            "status": {
                "message": "; ".join(violations) if violations else "OK"
            }
        }
    })
 
 
if __name__ == "__main__":
    app.run(host="0.0.0.0", port=8443, ssl_context=("tls.crt", "tls.key"))

TLS is required. The API server only calls webhooks over HTTPS. Use cert-manager to manage the certificate.


Common Issues

Webhook blocking deployments

bash
# Check which webhooks exist
kubectl get mutatingwebhookconfigurations
kubectl get validatingwebhookconfigurations
 
# Temporarily bypass for debugging (never in prod)
kubectl delete validatingwebhookconfiguration my-webhook
 
# Or add a label to skip specific namespaces
kubectl label namespace kube-system admission-webhook=ignore

Webhook failure policy

yaml
failurePolicy: Fail   # request rejected if webhook is unreachable
failurePolicy: Ignore # request allowed if webhook is unreachable

Use Ignore only for non-critical webhooks. Defaulting to Fail means a crashed webhook blocks all new deployments.

Webhook timeout

yaml
timeoutSeconds: 5   # default, max 30

If your webhook takes >5 seconds, requests time out and fail (with failurePolicy: Fail). Keep webhook logic fast.


Tools That Use Admission Webhooks

  • Kyverno — policy-as-YAML, no Rego needed
  • OPA Gatekeeper — Rego-based policies
  • Polaris — Kubernetes best practices validation
  • Datree — Shift-left policy validation

Admission webhooks are the enforcement layer that makes Kubernetes policies actually stick — without them, ResourceQuotas and network policies are advisory at best.

🔧

Today I Fixed

Short real fixes from production — posted daily

Browse fixes
Newsletter

Stay ahead of the curve

Get the latest DevOps, Kubernetes, AWS, and AI/ML guides delivered straight to your inbox. No spam — just practical engineering content.

Related Articles

Comments