All Articles
github-actionskubernetescicd

How to Set Up GitHub Actions Self-Hosted Runners on Kubernetes from Scratch

Step-by-step guide to running GitHub Actions self-hosted runners on Kubernetes with auto-scaling. Save money, get more control, and speed up your CI/CD pipelines.

DevOpsBoysMar 20, 20266 min read
Share:Tweet

GitHub-hosted runners are convenient but limited — 2 CPU, 7GB RAM, shared with everyone, and they cost money at scale. Self-hosted runners give you full control: custom hardware, pre-installed tools, persistent caches, and auto-scaling based on demand.

This guide walks you through setting up self-hosted GitHub Actions runners on Kubernetes using the Actions Runner Controller (ARC) — the official way to run auto-scaling runners.

Why Self-Hosted Runners?

FeatureGitHub-HostedSelf-Hosted (K8s)
CPU/RAM2 CPU, 7GBYour choice
Cost (at scale)$0.008/minInfrastructure cost only
Disk space14GB freeYour choice
Cache persistenceNone (ephemeral)Persistent volumes
Private network accessNeeds tunnelingDirect access
Custom toolsInstall every runPre-baked in image
Queue timeVaries (shared pool)Minimal (dedicated)

For teams running 1000+ minutes/month, self-hosted runners on existing Kubernetes infrastructure can reduce CI/CD costs by 50-80%.

Architecture Overview

┌──────────────────────────────────────────────────┐
│                  GitHub.com                        │
│  ┌──────────┐  ┌──────────┐  ┌──────────┐        │
│  │ Workflow 1│  │ Workflow 2│  │ Workflow 3│        │
│  └─────┬────┘  └─────┬────┘  └─────┬────┘        │
└────────┼──────────────┼──────────────┼────────────┘
         │              │              │
         ▼              ▼              ▼
┌──────────────────────────────────────────────────┐
│            Kubernetes Cluster                      │
│  ┌────────────────────────────────────────┐       │
│  │   Actions Runner Controller (ARC)       │       │
│  │   - Watches for pending jobs           │       │
│  │   - Scales runners up/down             │       │
│  └────────────────────────────────────────┘       │
│                                                    │
│  ┌─────────┐  ┌─────────┐  ┌─────────┐           │
│  │ Runner 1 │  │ Runner 2 │  │ Runner 3 │  ← Pods │
│  └─────────┘  └─────────┘  └─────────┘           │
└──────────────────────────────────────────────────┘

ARC watches GitHub for queued workflow jobs, creates runner pods to handle them, and scales back down when jobs complete.

Prerequisites

  • A Kubernetes cluster (1.23+)
  • Helm 3
  • kubectl configured
  • A GitHub personal access token (PAT) or GitHub App

Step 1 — Create a GitHub App (Recommended)

A GitHub App is more secure than a PAT because it has granular permissions and doesn't expire.

  1. Go to GitHub → Settings → Developer Settings → GitHub Apps → New GitHub App
  2. Set the following:
    • Name: arc-runner-controller
    • Homepage URL: https://github.com/actions/actions-runner-controller
    • Uncheck "Webhook → Active"
  3. Set permissions:
    • Repository permissions:
      • Actions: Read
      • Administration: Read & Write
      • Metadata: Read
    • Organization permissions (if org-level runners):
      • Self-hosted runners: Read & Write
  4. Install the app on your organization or repositories
  5. Note the App ID and Installation ID
  6. Generate a Private Key (.pem file)

Step 2 — Install Actions Runner Controller

bash
# Add the Helm repo
helm repo add actions-runner-controller https://actions-runner-controller.github.io/actions-runner-controller
helm repo update
 
# Create namespace
kubectl create namespace arc-system
 
# Create secret with GitHub App credentials
kubectl create secret generic controller-manager \
  --namespace arc-system \
  --from-literal=github_app_id=YOUR_APP_ID \
  --from-literal=github_app_installation_id=YOUR_INSTALLATION_ID \
  --from-file=github_app_private_key=path/to/private-key.pem
 
# Install the controller
helm install arc actions-runner-controller/actions-runner-controller \
  --namespace arc-system \
  --set authSecret.create=false \
  --set authSecret.name=controller-manager

Verify it's running:

bash
kubectl get pods -n arc-system
 
# NAME                                              READY   STATUS    AGE
# arc-actions-runner-controller-xxx                  2/2     Running   1m

Step 3 — Create a Runner Deployment

Create a RunnerDeployment that defines your runners:

yaml
# runner-deployment.yaml
apiVersion: actions.summerwind.dev/v1alpha1
kind: RunnerDeployment
metadata:
  name: my-runners
  namespace: arc-system
spec:
  replicas: 2
  template:
    spec:
      repository: your-org/your-repo  # Or use organization: your-org
      labels:
      - self-hosted
      - linux
      - x64
      - custom-runner
      dockerEnabled: true
      resources:
        limits:
          cpu: "4"
          memory: "8Gi"
        requests:
          cpu: "2"
          memory: "4Gi"

Apply it:

bash
kubectl apply -f runner-deployment.yaml

Check runner pods:

bash
kubectl get runners -n arc-system
 
# NAME                    REPOSITORY          STATUS
# my-runners-abc12        your-org/your-repo  Running
# my-runners-def34        your-org/your-repo  Running

Verify in GitHub: go to Repository → Settings → Actions → Runners. You should see your self-hosted runners listed as "Idle."

Step 4 — Set Up Auto-Scaling

Replace the fixed-replica RunnerDeployment with a HorizontalRunnerAutoscaler:

yaml
# runner-autoscaler.yaml
apiVersion: actions.summerwind.dev/v1alpha1
kind: RunnerDeployment
metadata:
  name: my-runners
  namespace: arc-system
spec:
  template:
    spec:
      repository: your-org/your-repo
      labels:
      - self-hosted
      - linux
      - custom-runner
      dockerEnabled: true
      resources:
        limits:
          cpu: "4"
          memory: "8Gi"
        requests:
          cpu: "2"
          memory: "4Gi"
---
apiVersion: actions.summerwind.dev/v1alpha1
kind: HorizontalRunnerAutoscaler
metadata:
  name: my-runners-autoscaler
  namespace: arc-system
spec:
  scaleTargetRef:
    kind: RunnerDeployment
    name: my-runners
  minReplicas: 1
  maxReplicas: 10
  scaleDownDelaySecondsAfterScaleOut: 300
  metrics:
  - type: TotalNumberOfQueuedAndInProgressWorkflowRuns
    scaleUpThreshold: "0.75"
    scaleDownThreshold: "0.25"
    scaleUpFactor: "2"
    scaleDownFactor: "0.5"

This configuration:

  • Keeps at least 1 runner warm (min 1)
  • Scales up to 10 runners when jobs are queued
  • Waits 5 minutes before scaling down (prevents flapping)
  • Doubles runners when queue is 75%+ utilized
  • Halves runners when queue is below 25%

Step 5 — Build a Custom Runner Image

Pre-install your tools so every job starts faster:

dockerfile
# Dockerfile.runner
FROM summerwind/actions-runner:latest
 
# Install common DevOps tools
RUN sudo apt-get update && sudo apt-get install -y \
    curl \
    wget \
    jq \
    unzip \
    && sudo rm -rf /var/lib/apt/lists/*
 
# Install kubectl
RUN curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl" \
    && sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
 
# Install Helm
RUN curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash
 
# Install Terraform
RUN wget -O- https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp.gpg \
    && echo "deb [signed-by=/usr/share/keyrings/hashicorp.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list \
    && sudo apt-get update && sudo apt-get install -y terraform
 
# Install Node.js 20
RUN curl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash - \
    && sudo apt-get install -y nodejs
 
# Install AWS CLI
RUN curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" \
    && unzip awscliv2.zip \
    && sudo ./aws/install \
    && rm -rf awscliv2.zip aws/

Build and push:

bash
docker build -f Dockerfile.runner -t ghcr.io/your-org/custom-runner:latest .
docker push ghcr.io/your-org/custom-runner:latest

Update your RunnerDeployment to use the custom image:

yaml
spec:
  template:
    spec:
      image: ghcr.io/your-org/custom-runner:latest

Step 6 — Use Self-Hosted Runners in Workflows

Update your workflow to target self-hosted runners:

yaml
name: Build and Deploy
on:
  push:
    branches: [main]
 
jobs:
  build:
    runs-on: [self-hosted, linux, custom-runner]  # Match your labels
    steps:
    - uses: actions/checkout@v4
 
    - name: Build
      run: |
        # kubectl, helm, terraform are already installed!
        npm ci
        npm run build
        docker build -t my-app:${{ github.sha }} .

The runs-on labels must match the labels in your RunnerDeployment.

Step 7 — Add Persistent Caching

Use a PersistentVolumeClaim to cache npm, Docker layers, etc.:

yaml
apiVersion: actions.summerwind.dev/v1alpha1
kind: RunnerDeployment
metadata:
  name: my-runners
spec:
  template:
    spec:
      repository: your-org/your-repo
      labels:
      - self-hosted
      - linux
      volumeMounts:
      - name: cache
        mountPath: /home/runner/.cache
      - name: docker-cache
        mountPath: /var/lib/docker
      volumes:
      - name: cache
        persistentVolumeClaim:
          claimName: runner-cache
      - name: docker-cache
        persistentVolumeClaim:
          claimName: docker-cache
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: runner-cache
  namespace: arc-system
spec:
  accessModes: [ReadWriteMany]
  storageClassName: standard
  resources:
    requests:
      storage: 50Gi

With persistent caching, npm ci and Docker builds use cached layers from previous runs — reducing build times by 50-70%.

Step 8 — Security Hardening

Run as Non-Root

yaml
spec:
  template:
    spec:
      securityContext:
        runAsNonRoot: true
        runAsUser: 1000

Use Ephemeral Runners

For security-sensitive repos, use ephemeral runners that are destroyed after each job:

yaml
spec:
  template:
    spec:
      ephemeral: true  # New pod for every job

Network Policies

Restrict what runners can access:

yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: runner-network-policy
  namespace: arc-system
spec:
  podSelector:
    matchLabels:
      app: runner
  policyTypes:
  - Egress
  egress:
  - to: []  # Allow all egress (needed for GitHub API)
    ports:
    - protocol: TCP
      port: 443
  - to:
    - namespaceSelector:
        matchLabels:
          name: kube-system
    ports:
    - protocol: UDP
      port: 53

Monitoring

Monitor your runner fleet:

bash
# Check runner status
kubectl get runners -n arc-system
 
# Check autoscaler status
kubectl get hra -n arc-system
 
# View runner logs
kubectl logs -n arc-system -l app=runner --tail=50

Set up Prometheus metrics for the controller:

yaml
helm upgrade arc actions-runner-controller/actions-runner-controller \
  --namespace arc-system \
  --set metrics.serviceMonitor.enabled=true

Cost Comparison

For a team running 5000 CI/CD minutes per month:

SetupMonthly Cost
GitHub-hosted (Linux)$200 (at $0.008/min × 2x multiplier)
Self-hosted on existing K8s$30-50 (incremental resource cost)
Self-hosted on dedicated nodes$80-120 (2x t3.xlarge spot instances)

Savings: 40-75% depending on your setup.

Wrapping Up

Self-hosted runners on Kubernetes give you faster builds, lower costs, and full control over your CI/CD infrastructure. The setup takes about an hour, and ARC handles auto-scaling automatically.

Start with a simple RunnerDeployment for one repository, verify it works, then expand to org-level runners with auto-scaling.

Want to master GitHub Actions and Kubernetes for CI/CD? KodeKloud's courses cover pipeline design, Kubernetes operations, and automation patterns with hands-on labs. For a Kubernetes cluster to run your runners, DigitalOcean Kubernetes is affordable and supports all ARC features.

Back to all articles
Share:Tweet
Newsletter

Stay ahead of the curve

Get the latest DevOps, Kubernetes, AWS, and AI/ML guides delivered straight to your inbox. No spam — just practical engineering content.

Related Articles

Comments