Configuration File

The Kubiya CLI uses a kubectl-style configuration file located at ~/.kubiya/config to manage contexts, authentication, and persistent settings. This provides a convenient way to manage multiple environments and configurations without repetitive environment variables or CLI flags.

Configuration File Location

~/.kubiya/config

The configuration file is created automatically when you first authenticate with the CLI. You can also create it manually.

Configuration Structure

The configuration file uses YAML format and follows a structure similar to kubectl:

apiVersion: v1
kind: Config
current-context: production

contexts:
  - name: production
    context:
      api-url: https://control-plane.kubiya.ai
      organization: my-org
      user: production-user

  - name: staging
    context:
      api-url: https://control-plane.kubiya.ai
      organization: my-org
      user: staging-user

users:
  - name: production-user
    user:
      token: kby_prod_api_key

  - name: staging-user
    user:
      token: kby_staging_api_key

organizations:
  - name: my-org
    organization:
      id: org-123456
      name: My Organization

Context Configuration

Basic Context

A context defines a complete configuration for connecting to the Kubiya Control Plane:

contexts:
  - name: production
    context:
      api-url: https://control-plane.kubiya.ai
      organization: my-org
      user: production-user

string

required

Unique identifier for the context

string

required

Control Plane API URL

string

required

Organization name reference (from organizations section)

string

required

User name reference (from users section)

Switching Contexts

# View current context
kubiya config current-context

# Switch to a different context
kubiya config use-context staging

# List all contexts
kubiya config get-contexts

Custom LLM Gateway Configuration

You can configure a local LiteLLM proxy gateway in your context for persistent, automatic configuration across all worker starts.

Configuration Options

Config File Path
Inline JSON Config
Ollama Local Models
Multi-Provider

Reference an external LiteLLM configuration file:

contexts:
  - name: production
    context:
      api-url: https://control-plane.kubiya.ai
      organization: my-org
      user: production-user
      litellm-proxy:
        enabled: true
        config-file: /Users/myuser/.kubiya/litellm_production_config.yaml

LiteLLM Config File (litellm_production_config.yaml):

model_list:
  - model_name: bedrock-claude-3-5-sonnet
    litellm_params:
      model: bedrock/anthropic.claude-3-5-sonnet-20240620-v1:0
      aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
      aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY
      aws_region_name: os.environ/AWS_REGION_NAME

litellm_settings:
  success_callback: ["langfuse"]
  drop_params: true

environment_variables:
  LANGFUSE_PUBLIC_KEY: "pk-lf-..."
  LANGFUSE_SECRET_KEY: "sk-lf-..."
  LANGFUSE_HOST: "https://cloud.langfuse.com"

Embed the LiteLLM configuration directly in the config file:

contexts:
  - name: staging
    context:
      api-url: https://control-plane.kubiya.ai
      organization: my-org
      user: staging-user
      litellm-proxy:
        enabled: true
        config-json: |
          {
            "model_list": [
              {
                "model_name": "gpt-4",
                "litellm_params": {
                  "model": "azure/gpt-4-staging",
                  "api_base": "https://staging-instance.openai.azure.com",
                  "api_key": "env:AZURE_STAGING_API_KEY"
                }
              }
            ]
          }

Configure Ollama for local model execution:

contexts:
  - name: local-dev
    context:
      api-url: https://control-plane.kubiya.ai
      organization: my-org
      user: dev-user
      litellm-proxy:
        enabled: true
        config-json: |
          {
            "model_list": [
              {
                "model_name": "llama3",
                "litellm_params": {
                  "model": "ollama/llama3",
                  "api_base": "http://localhost:11434"
                }
              },
              {
                "model_name": "codellama",
                "litellm_params": {
                  "model": "ollama/codellama",
                  "api_base": "http://localhost:11434"
                }
              }
            ]
          }

Make sure Ollama is running: ollama serve

Configure multiple LLM providers with fallback:

contexts:
  - name: production
    context:
      api-url: https://control-plane.kubiya.ai
      organization: my-org
      user: production-user
      litellm-proxy:
        enabled: true
        config-file: /Users/myuser/.kubiya/litellm_multi_provider.yaml

Multi-Provider Config (litellm_multi_provider.yaml):

model_list:
  # Primary: AWS Bedrock
  - model_name: claude-3-opus
    litellm_params:
      model: bedrock/anthropic.claude-3-opus-20240229-v1:0
      aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
      aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY
      aws_region_name: us-east-1

  # Fallback: Azure OpenAI
  - model_name: gpt-4
    litellm_params:
      model: azure/gpt-4
      api_base: https://your-instance.openai.azure.com
      api_key: os.environ/AZURE_API_KEY

  # Local development: Ollama
  - model_name: llama3
    litellm_params:
      model: ollama/llama3
      api_base: http://localhost:11434

litellm_settings:
  success_callback: ["langfuse"]
  drop_params: true

environment_variables:
  LANGFUSE_PUBLIC_KEY: "pk-lf-..."
  LANGFUSE_SECRET_KEY: "sk-lf-..."
  LANGFUSE_HOST: "https://cloud.langfuse.com"

LiteLLM Proxy Configuration Schema

boolean

required

Enable local LiteLLM proxy gateway for this context

string

Absolute path to LiteLLM configuration file (JSON or YAML). Mutually exclusive with config-json.

string

Inline JSON configuration string. Mutually exclusive with config-file.

Configuration Priority

When starting a worker, the LiteLLM proxy configuration respects this priority (highest to lowest):

CLI Flags: --enable-local-proxy with --proxy-config-file or --proxy-config-json
Environment Variables: KUBIYA_ENABLE_LOCAL_PROXY with KUBIYA_PROXY_CONFIG_FILE or KUBIYA_PROXY_CONFIG_JSON
Context Configuration: litellm-proxy section in current context (this file)
Control Plane Queue Settings: Configured via Composer UI or API
Control Plane LLM Gateway: Default centralized gateway (fallback)

Supported LLM Providers

LiteLLM supports 100+ LLM providers. Popular providers include:

AWS Bedrock

Claude, Llama, Mistral models on AWS

Azure OpenAI

GPT-4, GPT-3.5 via Azure

Ollama

Local open-source models

OpenAI

Direct OpenAI API access

Anthropic

Claude models directly

Google Vertex AI

Gemini and PaLM models

Full Provider List: See the complete list of supported providers including configuration examples for each.

Provider-Specific Documentation:

AWS Bedrock - IAM roles, session tokens, profiles
Ollama - Local model setup
Azure OpenAI - API versions, deployments
All Providers - Complete reference

Complete Configuration Example

Here’s a comprehensive example showing multiple contexts with different LiteLLM configurations:

apiVersion: v1
kind: Config
current-context: production

contexts:
  # Production: AWS Bedrock with Langfuse
  - name: production
    context:
      api-url: https://control-plane.kubiya.ai
      organization: acme-corp
      user: production-user
      litellm-proxy:
        enabled: true
        config-file: /Users/myuser/.kubiya/litellm_production.yaml

  # Staging: Azure OpenAI
  - name: staging
    context:
      api-url: https://control-plane.kubiya.ai
      organization: acme-corp
      user: staging-user
      litellm-proxy:
        enabled: true
        config-json: |
          {
            "model_list": [
              {
                "model_name": "gpt-4",
                "litellm_params": {
                  "model": "azure/gpt-4-staging",
                  "api_base": "https://staging.openai.azure.com",
                  "api_key": "env:AZURE_STAGING_API_KEY"
                }
              }
            ]
          }

  # Development: Ollama local models
  - name: development
    context:
      api-url: https://control-plane.kubiya.ai
      organization: acme-corp
      user: dev-user
      litellm-proxy:
        enabled: true
        config-json: |
          {
            "model_list": [
              {
                "model_name": "llama3",
                "litellm_params": {
                  "model": "ollama/llama3",
                  "api_base": "http://localhost:11434"
                }
              }
            ]
          }

  # Testing: Uses Control Plane gateway (no local proxy)
  - name: testing
    context:
      api-url: https://control-plane.kubiya.ai
      organization: acme-corp
      user: test-user
      # No litellm-proxy section = uses Control Plane gateway

users:
  - name: production-user
    user:
      token: kby_prod_key_here

  - name: staging-user
    user:
      token: kby_staging_key_here

  - name: dev-user
    user:
      token: kby_dev_key_here

  - name: test-user
    user:
      token: kby_test_key_here

organizations:
  - name: acme-corp
    organization:
      id: org-123456
      name: Acme Corporation

Using Context Configuration

Once configured, starting workers automatically uses the context settings:

# Switch to production context
kubiya config use-context production

# Start worker - automatically uses AWS Bedrock from context
kubiya worker start --queue-id=prod-queue --type=local

# Switch to development context
kubiya config use-context development

# Start worker - automatically uses Ollama from context
kubiya worker start --queue-id=dev-queue --type=local

# Override context with CLI flags (highest priority)
kubiya worker start \
  --queue-id=prod-queue \
  --type=local \
  --enable-local-proxy \
  --proxy-config-file=/path/to/different/config.yaml

Managing Configuration

View Current Configuration

# Show current context
kubiya config current-context

# Display full config file
kubiya config view

# Get specific context
kubiya config get-contexts production

Edit Configuration

# Edit config file directly
vim ~/.kubiya/config

# Or use your preferred editor
code ~/.kubiya/config
nano ~/.kubiya/config

Validate Configuration

# Test configuration by starting a worker in dry-run mode
kubiya worker start --queue-id=test --type=local --dry-run

# Check if LiteLLM config is valid
cat ~/.kubiya/litellm_production.yaml | python -c "import sys, yaml; yaml.safe_load(sys.stdin)"

Best Practices

Security

Store API keys in users section: Keep tokens in the users section, not inline in contexts

Use environment variables for secrets: Reference sensitive values via os.environ/VAR_NAME in LiteLLM configs

Set proper file permissions: chmod 600 ~/.kubiya/config to restrict access

Rotate tokens regularly: Update API keys at least quarterly

# Secure your config file
chmod 600 ~/.kubiya/config
chmod 600 ~/.kubiya/litellm_*.yaml

Organization

Separate configs by environment: Create distinct contexts for production, staging, and development

Use descriptive context names: Name contexts after their purpose (e.g., production-aws, staging-azure)

External config files: Store LiteLLM configs in separate files for easier management and version control

Document your setup: Add comments in YAML to explain custom configurations

Directory Structure

~/.kubiya/
├── config                          # Main configuration file
├── litellm_production.yaml         # Production LLM config
├── litellm_staging.yaml            # Staging LLM config
├── litellm_development.yaml        # Development LLM config
└── workers/
    ├── prod-queue/
    │   ├── venv/
    │   ├── logs/
    │   └── litellm_proxy.log       # Proxy logs (auto-generated)
    └── dev-queue/
        ├── venv/
        └── logs/

Troubleshooting

Configuration Not Loading

# Check if config file exists
ls -la ~/.kubiya/config

# Verify YAML syntax
cat ~/.kubiya/config | python -c "import sys, yaml; yaml.safe_load(sys.stdin)"

# Check current context
kubiya config current-context

# View full configuration
kubiya config view

LiteLLM Proxy Not Starting

# Verify config file path is absolute
echo $KUBIYA_PROXY_CONFIG_FILE

# Check LiteLLM config syntax
cat /path/to/litellm_config.yaml | python -c "import sys, yaml; yaml.safe_load(sys.stdin)"

# Test with CLI flag override
kubiya worker start \
  --queue-id=test-queue \
  --type=local \
  --enable-local-proxy \
  --proxy-config-file=/path/to/litellm_config.yaml

# Check worker logs
tail -f ~/.kubiya/workers/test-queue/logs/worker.log
tail -f ~/.kubiya/workers/test-queue/litellm_proxy.log

Context Switch Not Working

# List all contexts
kubiya config get-contexts

# Verify context exists
kubiya config get-contexts production

# Force switch
kubiya config use-context production --force

# Check environment variable override
echo $KUBIYA_CONTEXT

Migration from Environment Variables

If you’re currently using environment variables, you can migrate to the config file: Before (Environment Variables):

export KUBIYA_API_KEY="kby_prod_key"
export KUBIYA_ENABLE_LOCAL_PROXY=true
export KUBIYA_PROXY_CONFIG_FILE=/path/to/litellm_config.yaml

kubiya worker start --queue-id=prod-queue --type=local

After (Configuration File):

Create ~/.kubiya/config:

apiVersion: v1
kind: Config
current-context: production

contexts:
  - name: production
    context:
      api-url: https://control-plane.kubiya.ai
      organization: my-org
      user: production-user
      litellm-proxy:
        enabled: true
        config-file: /path/to/litellm_config.yaml

users:
  - name: production-user
    user:
      token: kby_prod_key

organizations:
  - name: my-org
    organization:
      id: org-123456
      name: My Organization

Start worker without environment variables:

kubiya worker start --queue-id=prod-queue --type=local

Benefits:

No need to set environment variables each time
Easy context switching for multiple environments
Configuration persists across sessions
Cleaner shell environment

Next Steps

Worker Management

Deploy workers with custom LLM configuration

Environment Variables

Override config with environment variables

Authentication

Set up authentication and API keys

LiteLLM Providers

Explore all supported LLM providers