Kubiya Stack Deployment Guide
Basic Steps
Verify that kubectl is installed and configured
Deploy a runner from the Kubiya App Portal
Connect to your Kubernetes cluster where the Kubiya Runner is installed
Step 1: Getting Okta Credentials:
Log in to Okta Administration
Navigate to https://{ORG}-admin.okta.com/admin/apps/active
Create a new app integration:
Select type: "API Services"
Choose an appropriate name and save
Edit general settings and disable "Require Demonstration Proof"
Configure API Access:
Under "API Scopes," grant:
Under "Admin Roles":
Select "READ ONLY ADMINISTRATOR"
Set up Client Credentials:
Click "Edit" under "Client Credentials"
Change to "Public Key/Private Key"
Click "Add Key" followed by "Generate New Key"
Under private key, switch to the PEM tab
Copy the content to a local file named private.pem
Step 2: Set Deploy Key for Source Code Repository
Before deploying, you need to prepare an SSH key to be used as the deploy key for accessing the source code repository.
Copy # Generate ssh key
ssh-keygen -t ed25519 -C "user@exmaple.com" -f /tmp/kubiya_deploy_key -N ""
# Print public key to add ass deploy key in git repo probider
cat /tmp/kubiya_deploy_key.pub
Configuring Deploy Key on GitHub and GitLab
GitHub
Navigate to Your Repository Settings:
Go to your repository on GitHub.
Click on Settings > Deploy keys (on the left sidebar).
Add the Deploy Key:
Provide a title (e.g., Kubiya Deploy Key).
Paste the contents of your public key (/tmp/kubiya_deploy_key.pub).
If the key needs write access, check Allow write access (only if necessary).
GitLab
Navigate to Your Repository Settings:
Go to your repository on GitLab.
Click on Settings > Repository > Deploy keys (under "Deploy Keys" section).
Add the Deploy Key:
Click on Expand next to the "Deploy Keys" section.
Provide a title (e.g., Kubiya Deploy Key).
Paste the contents of your public key (/tmp/kubiya_deploy_key.pub).
Check Write access if the key needs write permissions (only if necessary).
Deploying the Enforcer
Step 1: Set Environment Variables
First, set all required environment variables in your terminal:
Copy # Set your OKTA credentials
export OKTA_ORG_URL="your-token-endpoint-url" # example: kubiya.okta.com
export OKTA_CLIENT_ID="your-client-id"
export OKTA_PRIVATE_KEY_PATH="/path/to/your/private.pem"
export OPAL_POLICY_REPO_URL="
https://YOUR/REPO/HERE.git
" //
git@gitlab.com
:example/opa.git git@github.com:finebee/opa.git
export OPAL_POLICY_REPO_MAIN_BRANCH="YOUR_BRANCH" # example: main
export GIT_DEPLOY_KEY=$(cat /tmp/kubiya_deploy_key)
# Base64 encode the values
export OKTA_TOKEN_ENDPOINT_B64=$(echo -n "https://$OKTA_ORG_URL/oauth2/v1/token" | base64)
export OKTA_BASE_URL_B64=$(echo -n "https://$OKTA_ORG_URL" | base64)
export OKTA_CLIENT_ID_B64=$(echo -n "$OKTA_CLIENT_ID" | base64)
export PRIVATE_KEY_B64=$(cat "$OKTA_PRIVATE_KEY_PATH" | base64)
export OPAL_POLICY_REPO_URL_B64=$(echo -n "$OPAL_POLICY_REPO_URL" | base64)
export OPAL_POLICY_REPO_MAIN_BRANCH_B64=$(echo -n "$OPAL_POLICY_REPO_MAIN_BRANCH" | base64)
export GIT_DEPLOY_KEY_BS64=$(echo -n "$GIT_DEPLOY_KEY" | base64)
Step 2: Verify Environment Variables (Optional)
Copy #!/bin/bash
# Verify environment variables are set correctly
echo "Checking environment variables..."
echo "----------------------------------------"
# Check OKTA configuration
echo "OKTA Configuration:"
echo "OKTA_ORG_URL: ${OKTA_ORG_URL:-(not set)}"
echo "OKTA_CLIENT_ID: ${OKTA_CLIENT_ID:-(not set)}"
echo "OKTA_PRIVATE_KEY_PATH: ${OKTA_PRIVATE_KEY_PATH:-(not set)}"
echo
# Check Git configuration
echo "Git Configuration:"
echo "OPAL_POLICY_REPO_URL: ${OPAL_POLICY_REPO_URL:-(not set)}"
echo "OPAL_POLICY_REPO_MAIN_BRANCH: ${OPAL_POLICY_REPO_MAIN_BRANCH:-(not set)}"
echo "GIT_DEPLOY_KEY is set: $([[ -n "$GIT_DEPLOY_KEY" ]] && echo "✓" || echo "✗")"
echo
# Check Base64 encoded variables (showing first 10 characters only)
echo "Base64 Encoded Variables (first 10 chars):"
echo "OKTA Related:"
echo "- OKTA_TOKEN_ENDPOINT_B64: ${OKTA_TOKEN_ENDPOINT_B64:0:10}..."
echo "- OKTA_BASE_URL_B64: ${OKTA_BASE_URL_B64:0:10}..."
echo "- OKTA_CLIENT_ID_B64: ${OKTA_CLIENT_ID_B64:0:10}..."
echo "- PRIVATE_KEY_B64: ${PRIVATE_KEY_B64:0:10}..."
echo
echo "Git Related:"
echo "- OPAL_POLICY_REPO_URL_B64: ${OPAL_POLICY_REPO_URL_B64:0:10}..."
echo "- OPAL_POLICY_REPO_MAIN_BRANCH_B64: ${OPAL_POLICY_REPO_MAIN_BRANCH_B64:0:10}..."
echo "- GIT_DEPLOY_KEY_B64: ${GIT_DEPLOY_KEY_B64:0:10}..."
echo
# Verify required files exist
echo "File Verification:"
if [ -f "$OKTA_PRIVATE_KEY_PATH" ]; then
echo "✓ OKTA private key file exists"
else
echo "✗ OKTA private key file not found at $OKTA_PRIVATE_KEY_PATH"
fi
if [ -f "/tmp/kubiya_deploy_key" ]; then
echo "✓ Git deploy key file exists"
else
echo "✗ Git deploy key file not found at /tmp/kubiya_deploy_key"
fi
echo "----------------------------------------"
# Final verification
echo "Final Verification:"
if [ -n "$OKTA_ORG_URL" ] && [ -n "$OKTA_CLIENT_ID" ] && \
[ -n "$OKTA_PRIVATE_KEY_PATH" ] && [ -f "$OKTA_PRIVATE_KEY_PATH" ] && \
[ -n "$GIT_DEPLOY_KEY" ] && [ -f "/tmp/kubiya_deploy_key" ]; then
echo "✓ All required variables and files are present"
else
echo "✗ Some required variables or files are missing"
fi
Step 3: Deploy the Stack
Copy and run the following command to deploy the entire stack:
Copy kubectl apply -f - <<EOF
apiVersion: v1
kind: Secret
metadata:
name: opawatchdog-secrets
namespace: kubiya
type: Opaque
data:
POSTGRES_DB: cG9zdGdyZXM=
POSTGRES_USER: cG9zdGdyZXM=
POSTGRES_PASSWORD: cG9zdGdyZXM=
GIT_DEPLOY_KEY: $GIT_DEPLOY_KEY_BS64
OPAL_POLICY_REPO_URL: $OPAL_POLICY_REPO_URL_B64
OPAL_POLICY_REPO_MAIN_BRANCH: $OPAL_POLICY_REPO_MAIN_BRANCH_B64
OKTA_BASE_URL: $OKTA_BASE_URL_B64
OKTA_TOKEN_ENDPOINT: $OKTA_TOKEN_ENDPOINT_B64
OKTA_CLIENT_ID: $OKTA_CLIENT_ID_B64
private.pem: $PRIVATE_KEY_B64
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: enforcer
namespace: kubiya
spec:
replicas: 1
selector:
matchLabels:
app: enforcer
template:
metadata:
labels:
app: enforcer
spec:
containers:
- name: broadcast-kubiya
image: postgres:alpine
env:
- name: POSTGRES_DB
valueFrom:
secretKeyRef:
name: opawatchdog-secrets
key: POSTGRES_DB
- name: POSTGRES_USER
valueFrom:
secretKeyRef:
name: opawatchdog-secrets
key: POSTGRES_USER
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: opawatchdog-secrets
key: POSTGRES_PASSWORD
ports:
- containerPort: 5432
- name: opa-server-kubiya
image: permitio/opal-server:latest
env:
- name: OPAL_BROADCAST_URI
value: postgres://postgres:postgres@localhost:5432/postgres
- name: UVICORN_NUM_WORKERS
value: "4"
- name: OPAL_OPA_HEALTH_CHECK_POLICY_ENABLED
value: "true"
- name: OPAL_POLICY_REPO_SSH_KEY
valueFrom:
secretKeyRef:
name: opawatchdog-secrets
key: GIT_DEPLOY_KEY
- name: OPAL_POLICY_REPO_URL
valueFrom:
secretKeyRef:
name: opawatchdog-secrets
key: OPAL_POLICY_REPO_URL
- name: OPAL_POLICY_REPO_MAIN_BRANCH
valueFrom:
secretKeyRef:
name: opawatchdog-secrets
key: OPAL_POLICY_REPO_MAIN_BRANCH
- name: OPAL_POLICY_REPO_POLLING_INTERVAL
value: "30"
- name: OPAL_DATA_CONFIG_SOURCES
value: '{"config":{"entries":[{"url":"http://localhost:7002/policy-data","topics":["policy_data"],"dst_path":"/static"}]}}'
- name: OPAL_LOG_FORMAT_INCLUDE_PID
value: "true"
ports:
- containerPort: 7002
- name: opal-client-kubiya
image: permitio/opal-client:latest
env:
- name: OPAL_SERVER_URL
value: http://localhost:7002
- name: OPAL_LOG_FORMAT_INCLUDE_PID
value: "true"
- name: OPAL_INLINE_OPA_LOG_FORMAT
value: http
ports:
- containerPort: 7000
- containerPort: 8181
command: ["sh", "-c", "./wait-for.sh localhost:7002 --timeout=20 -- ./start.sh"]
- name: enforcer
image: ghcr.io/kubiyabot/opawatchdog
env:
- name: OKTA_BASE_URL
valueFrom:
secretKeyRef:
name: opawatchdog-secrets
key: OKTA_BASE_URL
- name: OKTA_TOKEN_ENDPOINT
valueFrom:
secretKeyRef:
name: opawatchdog-secrets
key: OKTA_TOKEN_ENDPOINT
- name: OKTA_CLIENT_ID
valueFrom:
secretKeyRef:
name: opawatchdog-secrets
key: OKTA_CLIENT_ID
- name: OKTA_PRIVATE_KEY
value: /etc/okta/private.pem
volumeMounts:
- name: private-key-volume
mountPath: /etc/okta/private.pem
subPath: private.pem
ports:
- containerPort: 5001
volumes:
- name: private-key-volume
secret:
secretName: opawatchdog-secrets
---
apiVersion: v1
kind: Service
metadata:
name: enforcer
namespace: kubiya
spec:
ports:
- name: enforcer
port: 5001
targetPort: 5001
selector:
app: enforcer
EOF
Step 4: Verify Deployment
Copy # Check if pods are running
kubectl get pods -n kubiya
# Check if service is created
kubectl get svc -n kubiya
# Check secrets (without revealing values)
kubectl get secrets -n kubiya
Step 5: Patch Tool-Manager deployment
Copy kubectl patch deployment tool-manager -n kubiya --type=json -p='[
{
"op": "add",
"path": "/spec/template/spec/containers/0/env/-",
"value": {
"name": "KUBIYA_AUTH_SERVER_URL",
"value": "http://enforcer.kubiya:5001"
}
}
]'
Step 6: Clean Up Environment Variables (Optional)
Copy echo "- OKTA_CLIENT_ID_B64"
echo "- OKTA_PRIVATE_KEY_PATH"
echo "- PRIVATE_KEY_B64"
echo "- OPAL_POLICY_REPO_URL"
echo "- OPAL_POLICY_REPO_URL_B64"
echo "- OPAL_POLICY_REPO_MAIN_BRANCH"
echo "- OPAL_POLICY_REPO_MAIN_BRANCH_B64"
# Verify cleanup
if [ -z "$OKTA_ORG_URL" ] && [ -z "$OKTA_TOKEN_ENDPOINT_B64" ] && [ -z "$OKTA_BASE_URL_B64" ]; then
echo "Cleanup completed successfully."
else
echo "Warning: Some variables may still be set. Please check your environment."
fi
Troubleshooting
If you encounter issues:
Check pod logs: kubectl logs <pod-name> -n kubiya
Check pod status: kubectl describe pod <pod-name> -n kubiya
Verify secrets are correctly created: kubectl describe secret opawatchdog-secrets -n kubiya
Security Notes
Keep all environment variables secure
Never commit sensitive credentials to version control
Regularly rotate credentials according to your security policies
Store private keys in a secure location
Follow the principle of least privilege when assigning permissions
Last updated 15 hours ago
