Workflow Execution

The Kubiya CLI supports executing workflows from multiple sources with comprehensive GitHub authentication, real-time tracking, and variable substitution.

What are Workflows?

Workflows are orchestrated sequences of tasks that:

Execute on Workers: Run distributed across your worker infrastructure
Use Tools: Leverage your tool ecosystem
Support Branching: Conditional logic and error handling
Track Progress: Real-time execution monitoring
Handle Failures: Automatic retries and fallback strategies

Quick Start

Execute Local Workflow

# Simple execution
kubiya workflow execute deployment.yaml

# With variables
kubiya workflow execute deployment.yaml \
  --var environment=production \
  --var version=v1.2.3

# With specific runner
kubiya workflow execute deployment.yaml \
  --runner production-runner

Execute from GitHub

# Repository shorthand
kubiya workflow execute myorg/deploy-workflows

# Specific file
kubiya workflow execute myorg/workflows/production/deploy.yaml

# Specific branch
kubiya workflow execute myorg/workflows/develop/staging.yaml --branch develop

# Full GitHub URL
kubiya workflow execute https://github.com/myorg/workflows/blob/main/deploy.yaml

Workflow Sources

Local Files

Execute workflows from your local filesystem.

YAML
JSON

# Execute YAML workflow
kubiya workflow execute deploy.yaml

# With relative path
kubiya workflow execute ./workflows/deploy.yaml

# With absolute path
kubiya workflow execute /home/user/workflows/deploy.yaml

Example Workflow:

# deploy.yaml
name: "Deploy Application"
description: "Deploy application to Kubernetes"

steps:
  - name: "build"
    type: "tool"
    tool: "docker-build"
    args:
      image: "myapp"
      tag: "{{.version}}"

  - name: "push"
    type: "tool"
    tool: "docker-push"
    args:
      image: "myapp:{{.version}}"
    depends_on:
      - "build"

  - name: "deploy"
    type: "tool"
    tool: "kubectl-apply"
    args:
      manifest: "k8s/deployment.yaml"
      namespace: "{{.environment}}"
    depends_on:
      - "push"

  - name: "health-check"
    type: "tool"
    tool: "health-check"
    args:
      url: "https://{{.environment}}.myapp.com/health"
    depends_on:
      - "deploy"

GitHub Repositories

Execute workflows from GitHub with automatic authentication.

GitHub Shorthand

# Owner/repository format
kubiya workflow execute myorg/deploy-workflows

# Specific file in repository
kubiya workflow execute myorg/workflows/deploy.yaml

# Specific directory
kubiya workflow execute myorg/workflows/production/

# With branch
kubiya workflow execute myorg/workflows/staging.yaml --branch develop

Full GitHub URLs

# Standard GitHub URL
kubiya workflow execute https://github.com/myorg/workflows/blob/main/deploy.yaml

# Raw GitHub URL
kubiya workflow execute https://raw.githubusercontent.com/myorg/workflows/main/deploy.yaml

# Private repository (uses GitHub integration)
kubiya workflow execute https://github.com/myorg/private-workflows/blob/main/deploy.yaml

GitHub Authentication

The CLI automatically uses GitHub tokens from Kubiya integrations for private repository access.

When Integration is Available:

📥 Cloning repository...
🔐 Using GitHub authentication from integrations
✅ Repository cloned with authentication
📄 Found workflow: deploy.yaml

When Integration is NOT Available:

📥 Cloning repository...
💡 For private repositories, set up GitHub integration at:
  • Composer App: https://compose.kubiya.ai
  • API: Use the integrations API
⚠️ Authenticated clone failed, trying public access...

Set up GitHub Integration:

# Via CLI
kubiya integration create github \
  --name "GitHub Integration" \
  --token "ghp_your_token_here" \
  --org "your-org"

# Via Composer
# Navigate to https://compose.kubiya.ai
# Go to Settings → Integrations → GitHub
# Click "Connect" and authorize

Raw URLs

Execute workflows from any HTTP/HTTPS endpoint.

# Direct workflow URL
kubiya workflow execute https://workflows.example.com/deploy.yaml

# With basic auth
kubiya workflow execute https://user:pass@workflows.internal/deploy.yaml

# S3 presigned URL
kubiya workflow execute https://bucket.s3.amazonaws.com/workflow.yaml?token=xyz

Execution Options

Variables

Pass variables to workflows for dynamic configuration.

# Single variable
kubiya workflow execute deploy.yaml --var environment=production

# Multiple variables
kubiya workflow execute deploy.yaml \
  --var environment=staging \
  --var version=v2.1.0 \
  --var replicas=3 \
  --var notify=true

# From environment
export ENVIRONMENT=production
export VERSION=v1.0.0
kubiya workflow execute deploy.yaml \
  --var environment=$ENVIRONMENT \
  --var version=$VERSION

# From file
kubiya workflow execute deploy.yaml \
  --var-file vars.json

vars.json:

{
  "environment": "production",
  "version": "v1.2.3",
  "replicas": 5,
  "enable_monitoring": true,
  "notify_slack": true
}

Runner Selection

Specify which runner should execute the workflow.

# Use specific runner
kubiya workflow execute deploy.yaml --runner k8s-runner

# Use default runner
kubiya workflow execute deploy.yaml  # Uses default runner

# Override runner for specific environment
kubiya workflow execute deploy.yaml \
  --var environment=production \
  --runner production-runner

Policy Validation

Workflows are validated against OPA policies before execution.

# Execute with policy checks (default)
kubiya workflow execute deploy.yaml

# Skip policy validation (not recommended)
kubiya workflow execute deploy.yaml --skip-policy-check

# Dry-run: validate without executing
kubiya workflow execute deploy.yaml --dry-run

Execution Options

# Verbose output
kubiya workflow execute deploy.yaml --verbose

# Save execution trace
kubiya workflow execute deploy.yaml --save-trace

# Set timeout
kubiya workflow execute deploy.yaml --timeout 3600s

# Async execution (don't wait for completion)
kubiya workflow execute deploy.yaml --async

Real-Time Tracking

The CLI provides real-time execution tracking with progress indicators.

Progress Display

🚀 Executing workflow: CI/CD Pipeline
Source: GitHub Repository
Repository: https://github.com/myorg/workflows.git
File: ci-cd/production.yaml
Runner: k8s-prod-runner
Variables:
  environment = production
  version = v1.2.3

🚀 Starting workflow execution...

📊 [████████████████████░] 4/5 steps completed

▶️ [4/5] 🔄 Running: Deploy Application
  ⏳ Running...
  ✅ Step completed in 45.2s
  📤 Output: Deployment successful, 3 pods updated

▶️ [5/5] 🔄 Running: Health Check
  ⏳ Running...
  ✅ Step completed in 12.1s
  📤 Output: All health checks passed

🎉 Workflow completed successfully!

📊 Workflow Execution Summary
Name: CI/CD Pipeline
Status: ✅ Completed
Duration: 2m 15.3s
Steps: 5/5 successful

┌─ ✅ Build Image (18.5s)
├─ ✅ Run Tests (22.1s)
├─ ✅ Security Scan (8.7s)
├─ ✅ Deploy Application (45.2s)
└─ ✅ Health Check (12.1s)

Execution ID: exec-abc123def456
View in Composer: https://compose.kubiya.ai/executions/exec-abc123def456

Execution Logs

# Follow execution logs
kubiya workflow logs <execution-id> --follow

# View specific step logs
kubiya workflow logs <execution-id> --step deploy

# Export logs
kubiya workflow logs <execution-id> > execution.log

# Filter logs
kubiya workflow logs <execution-id> --level ERROR
kubiya workflow logs <execution-id> --grep "deployment"

Workflow Definition

Basic Structure

name: "Workflow Name"
description: "Workflow description"
version: "1.0.0"

# Variables with defaults
variables:
  environment:
    type: "string"
    default: "staging"
    description: "Target environment"
  version:
    type: "string"
    required: true
    description: "Application version"

# Workflow steps
steps:
  - name: "step1"
    type: "tool"
    tool: "my-tool"
    args:
      arg1: "value1"
      arg2: "{{.version}}"

  - name: "step2"
    type: "tool"
    tool: "another-tool"
    depends_on:
      - "step1"

Step Types

Tool Execution
Conditional
Parallel
Error Handling

- name: "deploy"
  type: "tool"
  tool: "kubectl-apply"
  args:
    manifest: "deployment.yaml"
    namespace: "{{.environment}}"
  timeout: "300s"
  retry:
    max_attempts: 3
    backoff: "exponential"

Dependencies

steps:
  # Step 1: No dependencies
  - name: "build"
    type: "tool"
    tool: "docker-build"

  # Step 2: Depends on step 1
  - name: "test"
    type: "tool"
    tool: "run-tests"
    depends_on:
      - "build"

  # Step 3: Depends on step 2
  - name: "push"
    type: "tool"
    tool: "docker-push"
    depends_on:
      - "test"

  # Step 4: Depends on step 3
  - name: "deploy"
    type: "tool"
    tool: "kubectl-apply"
    depends_on:
      - "push"

  # Step 5: Parallel with step 4
  - name: "update-docs"
    type: "tool"
    tool: "update-documentation"
    depends_on:
      - "test"  # Can run as soon as tests pass

Variable Substitution

variables:
  environment: "staging"
  version: "v1.0.0"
  replicas: 3

steps:
  - name: "deploy"
    tool: "kubectl-apply"
    args:
      # Simple substitution
      namespace: "{{.environment}}"
      version: "{{.version}}"

      # Conditional
      replicas: "{{if eq .environment \"production\"}}5{{else}}{{.replicas}}{{end}}"

      # With default
      timeout: "{{.timeout | default \"300s\"}}"

      # Complex expression
      image: "myapp:{{.version}}-{{.environment}}"

CI/CD Integration

GitHub Actions

name: Deploy with Kubiya
on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Install Kubiya CLI
        run: |
          curl -fsSL https://raw.githubusercontent.com/kubiyabot/cli/main/install.sh | bash
          echo "$HOME/.kubiya/bin" >> $GITHUB_PATH

      - name: Execute Deployment
        env:
          KUBIYA_API_KEY: ${{ secrets.KUBIYA_API_KEY }}
        run: |
          kubiya workflow execute myorg/deployment-workflows/production.yaml \
            --var version=${{ github.sha }} \
            --var environment=production \
            --var git_ref=${{ github.ref }}

      - name: Check Deployment Status
        if: failure()
        run: |
          kubiya workflow logs ${{ steps.deploy.outputs.execution_id }}

GitLab CI

deploy:
  stage: deploy
  image: ubuntu:latest
  before_script:
    - curl -fsSL https://raw.githubusercontent.com/kubiyabot/cli/main/install.sh | bash
  script:
    - |
      kubiya workflow execute ./deployment/workflow.yaml \
        --var environment=${CI_ENVIRONMENT_NAME} \
        --var version=${CI_COMMIT_SHA} \
        --var pipeline_id=${CI_PIPELINE_ID}
  environment:
    name: production
  only:
    - main

Jenkins

pipeline {
  agent any

  environment {
    KUBIYA_API_KEY = credentials('kubiya-api-key')
  }

  stages {
    stage('Install CLI') {
      steps {
        sh 'curl -fsSL https://raw.githubusercontent.com/kubiyabot/cli/main/install.sh | bash'
      }
    }

    stage('Deploy') {
      steps {
        sh '''
          kubiya workflow execute deployment.yaml \
            --var environment=production \
            --var version=${BUILD_NUMBER} \
            --var git_commit=${GIT_COMMIT}
        '''
      }
    }
  }

  post {
    failure {
      sh 'kubiya workflow logs ${EXECUTION_ID} > failure.log'
      archiveArtifacts artifacts: 'failure.log'
    }
  }
}

Advanced Workflows

Multi-Environment

name: "Multi-Environment Deployment"
description: "Deploy to multiple environments sequentially"

variables:
  version:
    type: "string"
    required: true
  environments:
    type: "array"
    default: ["dev", "staging", "production"]

steps:
  - name: "build"
    type: "tool"
    tool: "docker-build"
    args:
      tag: "{{.version}}"

  - name: "deploy-dev"
    type: "tool"
    tool: "deploy-to-env"
    args:
      environment: "dev"
      version: "{{.version}}"
    depends_on: ["build"]

  - name: "test-dev"
    type: "tool"
    tool: "run-e2e-tests"
    args:
      environment: "dev"
    depends_on: ["deploy-dev"]

  - name: "deploy-staging"
    type: "tool"
    tool: "deploy-to-env"
    args:
      environment: "staging"
      version: "{{.version}}"
    depends_on: ["test-dev"]

  - name: "approval-prod"
    type: "approval"
    message: "Approve production deployment?"
    depends_on: ["deploy-staging"]

  - name: "deploy-production"
    type: "tool"
    tool: "deploy-to-env"
    args:
      environment: "production"
      version: "{{.version}}"
    depends_on: ["approval-prod"]

Blue-Green Deployment

name: "Blue-Green Deployment"

steps:
  - name: "deploy-green"
    type: "tool"
    tool: "kubectl-apply"
    args:
      manifest: "green-deployment.yaml"
      version: "{{.version}}"

  - name: "wait-green-ready"
    type: "tool"
    tool: "kubectl-wait"
    args:
      deployment: "myapp-green"
      condition: "available"
    depends_on: ["deploy-green"]

  - name: "test-green"
    type: "tool"
    tool: "run-smoke-tests"
    args:
      target: "green"
    depends_on: ["wait-green-ready"]

  - name: "switch-traffic"
    type: "tool"
    tool: "kubectl-patch-service"
    args:
      service: "myapp"
      selector: "version=green"
    depends_on: ["test-green"]

  - name: "monitor"
    type: "tool"
    tool: "monitor-metrics"
    args:
      duration: "300s"
    depends_on: ["switch-traffic"]

  - name: "cleanup-blue"
    type: "tool"
    tool: "kubectl-delete"
    args:
      deployment: "myapp-blue"
    depends_on: ["monitor"]
    on_error:
      - name: "rollback"
        tool: "kubectl-patch-service"
        args:
          service: "myapp"
          selector: "version=blue"

Troubleshooting

Execution Failures

# View detailed error logs
kubiya workflow logs <execution-id> --level ERROR

# Get execution status
kubiya workflow status <execution-id>

# Retry failed workflow
kubiya workflow retry <execution-id>

# Debug mode
kubiya workflow execute deploy.yaml --verbose --debug

GitHub Access Issues

# Check GitHub integration
kubiya integration list --type github

# Test repository access
git clone https://github.com/yourorg/workflows.git

# Use HTTPS with token
git clone https://oauth2:${GITHUB_TOKEN}@github.com/yourorg/workflows.git

# Set up integration
kubiya integration create github \
  --name "GitHub" \
  --token "ghp_your_token"

Policy Violations

# View policy details
kubiya policy describe <policy-id>

# List applicable policies
kubiya policy list --resource workflow

# Validate workflow against policies
kubiya workflow validate deploy.yaml

# Skip policy check (not recommended)
kubiya workflow execute deploy.yaml --skip-policy-check

Command Reference

# Execute workflow
kubiya workflow execute <source> [options]

# Options
--var key=value              Set variable
--var-file file.json         Load variables from file
--runner runner-name         Specify runner
--skip-policy-check          Skip policy validation
--dry-run                    Validate without executing
--verbose                    Enable verbose output
--save-trace                 Save execution trace
--timeout duration           Set timeout
--async                      Don't wait for completion

# View logs
kubiya workflow logs <execution-id> [options]
--follow                     Follow logs in real-time
--step step-name             Show logs for specific step
--level level                Filter by log level
--grep pattern               Filter by pattern

# Workflow management
kubiya workflow list         List recent executions
kubiya workflow status <id>  Get execution status
kubiya workflow retry <id>   Retry failed workflow
kubiya workflow cancel <id>  Cancel running workflow
kubiya workflow validate <file>  Validate workflow definition

Next Steps

Webhooks

Automate workflow execution with webhooks

Worker Management

Deploy workers to execute workflows

Resource Management

Manage tools and sources for workflows

Workflow DSL

Learn the Workflow DSL syntax

Getting Started

Core Operations

Advanced

​What are Workflows?

​Quick Start

​Execute Local Workflow

​Execute from GitHub

​Workflow Sources

​Local Files

​GitHub Repositories

​GitHub Shorthand

​Full GitHub URLs

​GitHub Authentication

​Raw URLs

​Execution Options

​Variables

​Runner Selection

​Policy Validation

​Execution Options

​Real-Time Tracking

​Progress Display

​Execution Logs

​Workflow Definition

​Basic Structure

​Step Types

​Dependencies

​Variable Substitution

​CI/CD Integration

​GitHub Actions

​GitLab CI

​Jenkins

​Advanced Workflows

​Multi-Environment

​Blue-Green Deployment

​Troubleshooting

​Execution Failures

​GitHub Access Issues

​Policy Violations

​Command Reference

​Next Steps

Webhooks

Worker Management

Resource Management

Workflow DSL

What are Workflows?

Quick Start

Execute Local Workflow

Execute from GitHub

Workflow Sources

Local Files

GitHub Repositories

GitHub Shorthand

Full GitHub URLs

GitHub Authentication

Raw URLs

Execution Options

Variables

Runner Selection

Policy Validation

Execution Options

Real-Time Tracking

Progress Display

Execution Logs

Workflow Definition

Basic Structure

Step Types

Dependencies

Variable Substitution

CI/CD Integration

GitHub Actions

GitLab CI

Jenkins

Advanced Workflows

Multi-Environment

Blue-Green Deployment

Troubleshooting

Execution Failures

GitHub Access Issues

Policy Violations

Command Reference

Next Steps