Runners

Kubiya Runners are the execution engines that bring workflows to life. They are Kubernetes operators that orchestrate containers securely within your infrastructure, ensuring your data never leaves your environment while providing enterprise-grade workflow execution.

What is a Runner?

A Kubiya Runner is:

A Kubernetes operator that manages workflow execution
Deployed in your infrastructure (K8s cluster)
Connects to Kubiya platform for workflow coordination
Executes workflow steps as Kubernetes pods
Provides secure isolation between executions

Runner Architecture

Types of Runners

Local Runners

Deploy runners in your own infrastructure for complete control:

Your Infrastructure: Run in your Kubernetes cluster, VMs, or bare metal
Network Isolation: Keep sensitive data within your network
Custom Configuration: Tailor resources to your needs
Compliance: Meet regulatory requirements

Deployment Options

Helm Chart (Recommended)

helm repo add kubiya https://charts.kubiya.ai
helm install my-runner kubiya/kubiya-runner \
  --namespace kubiya \
  --create-namespace

See the Helm Chart documentation for detailed configuration options.

Kubernetes Manifest
- Create runner in Kubiya platform
- Download generated manifest
- Apply to your cluster
Docker
- For development/testing only
- Not recommended for production

Local runners require outbound HTTPS connectivity to Kubiya’s control plane for management and coordination. Your data and workflows remain within your infrastructure.

Hosted Runners

Managed by Kubiya for quick starts:

Zero Setup

Start executing workflows immediately without any infrastructure setup

Managed Updates

Automatic updates and maintenance handled by Kubiya

Elastic Scaling

Automatically scales based on workload without capacity planning

Development

Perfect for development, testing, and proof-of-concepts

How Runners Work

1. Connection & Authentication

The runner:

Authenticates with Kubiya using a secure token
Establishes a persistent connection
Registers its capabilities and constraints
Begins polling for assigned workflows

2. Workflow Execution

When a workflow is assigned:

# Runner creates pods like this for each step
apiVersion: v1
kind: Pod
metadata:
  name: workflow-${id}-step-${name}
  labels:
    kubiya.ai/workflow: ${workflow_id}
    kubiya.ai/step: ${step_name}
    kubiya.ai/runner: ${runner_name}
spec:
  containers:
  - name: step
    image: ${step.image}
    command: ${step.command}
    env: ${step.env}
    resources:
      requests:
        memory: ${step.resources.memory}
        cpu: ${step.resources.cpu}
  restartPolicy: Never
  serviceAccountName: kubiya-workflow

3. Resource Management

Runners manage:

Pod lifecycle: Creation, monitoring, cleanup
Resource limits: CPU, memory, GPU allocation
Storage: Volume mounts and persistent data
Networking: Service discovery and policies
Secrets: Secure injection of credentials

4. Security Model

Deployment Options

Helm Chart Installation

The recommended way to deploy runners:

# Add Kubiya Helm repository
helm repo add kubiya https://charts.kubiya.ai
helm repo update

# Install runner
helm install kubiya-runner kubiya/runner \
  --namespace kubiya \
  --create-namespace \
  --set runner.token=${RUNNER_TOKEN} \
  --set runner.name=production-runner

Configuration Options

# values.yaml
runner:
  # Runner identification
  name: production-runner
  token: ${RUNNER_TOKEN}
  
  # Resource allocation
  resources:
    requests:
      memory: "256Mi"
      cpu: "100m"
    limits:
      memory: "1Gi"
      cpu: "500m"
  
  # Workflow execution
  workflow:
    namespace: kubiya-workflows
    serviceAccount: kubiya-executor
    defaultTimeout: 30m
    maxConcurrent: 10
  
  # Security
  security:
    podSecurityStandard: restricted
    allowPrivileged: false
    runAsNonRoot: true
  
  # Networking
  network:
    dnsPolicy: ClusterFirst
    enableServiceLinks: false
  
  # Storage
  storage:
    workspaceSize: 10Gi
    storageClass: fast-ssd

kubectl Installation

For quick testing:

# Create namespace
kubectl create namespace kubiya

# Create secret with token
kubectl create secret generic kubiya-runner-token \
  --from-literal=token=${RUNNER_TOKEN} \
  -n kubiya

# Apply runner manifest
kubectl apply -f https://get.kubiya.ai/runner.yaml

Runner Capabilities

Container Orchestration

Runners can orchestrate any container:

# Python containers
step.analyze(
    image="python:3.11-slim",
    command=["python", "analyze.py"]
)

# Node.js applications
step.build(
    image="node:18-alpine",
    command=["npm", "run", "build"]
)

# Custom tools
step.security_scan(
    image="aquasec/trivy:latest",
    command=["trivy", "image", "myapp:latest"]
)

# Cloud CLIs
step.deploy(
    image="amazon/aws-cli:latest",
    command=["aws", "ecs", "update-service"]
)

Advanced Features

1. Sidecar Containers

Run multiple containers in a step:

step.database_migration(
    image="migrate:latest",
    sidecars=[
        {
            "name": "postgres",
            "image": "postgres:15",
            "env": {"POSTGRES_PASSWORD": "temp"}
        }
    ]
)

2. Init Containers

Prepare environment before main container:

step.process(
    image="processor:latest",
    init_containers=[
        {
            "name": "download-data",
            "image": "aws-cli:latest",
            "command": ["aws", "s3", "sync", "s3://data", "/data"]
        }
    ]
)

3. Volume Management

Share data between steps:

# Create volume
volume = workflow.create_volume("shared-data", size="5Gi")

# Write data
step.generate(
    image="generator:latest",
    volumes=[{"name": "data", "mount": "/output", "volume": volume}]
)

# Read data
step.process(
    image="processor:latest",
    volumes=[{"name": "data", "mount": "/input", "volume": volume}]
)

Monitoring & Observability

Runner Metrics

Exposed via Prometheus:

# Runner health
kubiya_runner_up{runner="production"} 1

# Workflow execution
kubiya_workflows_running{runner="production"} 5
kubiya_workflows_completed{runner="production",status="success"} 142
kubiya_workflows_completed{runner="production",status="failed"} 3

# Resource usage
kubiya_runner_cpu_usage{runner="production"} 0.45
kubiya_runner_memory_usage{runner="production"} 0.72

# Step execution times
kubiya_step_duration_seconds{step="build",percentile="p99"} 45.2

Logging

Structured logging with context:

{
  "timestamp": "2024-01-10T10:30:45Z",
  "level": "info",
  "runner": "production-runner",
  "workflow_id": "wf-123",
  "step_name": "deploy",
  "message": "Starting step execution",
  "image": "kubectl:latest",
  "namespace": "kubiya-workflows"
}

Health Checks

Built-in health endpoints:

# Liveness probe
curl http://runner:8080/healthz

# Readiness probe
curl http://runner:8080/ready

# Metrics
curl http://runner:8080/metrics

Security Best Practices

1. Network Isolation

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: kubiya-runner-policy
spec:
  podSelector:
    matchLabels:
      app: kubiya-runner
  policyTypes:
  - Ingress
  - Egress
  ingress:
  - from:
    - namespaceSelector:
        matchLabels:
          name: kubiya
  egress:
  - to:
    - namespaceSelector:
        matchLabels:
          name: kubiya
  - to:
    - namespaceSelector: {}
    ports:
    - protocol: TCP
      port: 443  # HTTPS only

2. RBAC Configuration

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: kubiya-runner
rules:
- apiGroups: [""]
  resources: ["pods", "pods/log"]
  verbs: ["create", "get", "list", "watch", "delete"]
- apiGroups: [""]
  resources: ["configmaps", "secrets"]
  verbs: ["get", "list"]
- apiGroups: [""]
  resources: ["persistentvolumeclaims"]
  verbs: ["create", "get", "delete"]

3. Pod Security Standards

apiVersion: v1
kind: Namespace
metadata:
  name: kubiya-workflows
  labels:
    pod-security.kubernetes.io/enforce: restricted
    pod-security.kubernetes.io/audit: restricted
    pod-security.kubernetes.io/warn: restricted

Troubleshooting

Common Issues

Runner not connecting

Workflows not executing

Performance issues

Next Steps

Deploy a Runner

Step-by-step runner deployment

Security Guide

Secure your runner deployment

Local Development

Set up local runners

Monitoring

Production monitoring setup

Introduction

Core Concepts

Workflows

MCP Integration

Full-Stack Agents

Streaming UI

Tutorials

Resources

​Runners

​What is a Runner?

​Runner Architecture

​Types of Runners

​Local Runners

​Deployment Options

​Hosted Runners

Zero Setup

Managed Updates

Elastic Scaling

Development

​How Runners Work

​1. Connection & Authentication

​2. Workflow Execution

​3. Resource Management

​4. Security Model

​Deployment Options

​Helm Chart Installation

​Configuration Options

​kubectl Installation

​Runner Capabilities

​Container Orchestration

​Advanced Features

​1. Sidecar Containers

​2. Init Containers

​3. Volume Management

​Monitoring & Observability

​Runner Metrics

​Logging

​Health Checks

​Security Best Practices

​1. Network Isolation

​2. RBAC Configuration

​3. Pod Security Standards

​Troubleshooting

​Common Issues

​Next Steps

Deploy a Runner

Security Guide

Local Development

Monitoring

Runners

What is a Runner?

Runner Architecture

Types of Runners

Local Runners

Deployment Options

Hosted Runners

How Runners Work

1. Connection & Authentication

2. Workflow Execution

3. Resource Management

4. Security Model

Deployment Options

Helm Chart Installation

Configuration Options

kubectl Installation

Runner Capabilities

Container Orchestration

Advanced Features

1. Sidecar Containers

2. Init Containers

3. Volume Management

Monitoring & Observability

Runner Metrics

Logging

Health Checks

Security Best Practices

1. Network Isolation

2. RBAC Configuration

3. Pod Security Standards

Troubleshooting

Common Issues

Next Steps