Semantic Search

Semantic Search uses vector embeddings to find nodes that are semantically similar to your query, even if they don’t contain exact keyword matches.

Overview

Semantic Search enables you to find relevant information in your context graph using natural language queries. Unlike keyword-based search, it understands meaning and context:

Meaning-Based: Finds conceptually related content, not just exact matches
Fast: Vector similarity search for quick results
Flexible: Works across different entity types and properties
Scored: Returns results with similarity scores for ranking

Semantic Search is ideal for finding information when you know what you’re looking for conceptually but don’t have exact keywords. It’s particularly useful for documentation, logs, and unstructured data.

Quick Start

from kubiya import ControlPlaneClient

# Initialize the client
client = ControlPlaneClient(api_key="your-api-key")

# Perform semantic search
results = client.graph.semantic_search(
    query="databases with high availability",
    limit=5
)

# Review results with similarity scores
for result in results:
    print(f"Node: {result['node_id']}")
    print(f"Content: {result['content']}")
    print(f"Similarity: {result['similarity_score']:.3f}")
    print(f"Source: {result['source']}")
    print("---")

Core Concepts

Vector Embeddings

Semantic Search converts your query and graph content into high-dimensional vectors (embeddings) that represent semantic meaning. Similar concepts have similar vectors, enabling meaning-based search.

Similarity Scores

Results include similarity scores (typically 0.0-1.0):

0.8-1.0: Highly relevant
0.6-0.8: Moderately relevant
0.4-0.6: Somewhat relevant
< 0.4: Low relevance

Dataset Integration

Semantic Search is powered by Kubiya’s cognitive memory system. Content indexed in cognitive datasets becomes searchable via semantic search.

Basic Usage

Simple Search

from kubiya import ControlPlaneClient

client = ControlPlaneClient(api_key="your-api-key")

# Search for relevant content
results = client.graph.semantic_search(
    query="production incidents last week",
    limit=10
)

print(f"Found {len(results)} relevant items")
for result in results:
    print(f"[{result['similarity_score']:.2f}] {result['content'][:100]}...")

Example Response

[
  {
    "node_id": "incident-2024-001",
    "content": "Production database outage in us-east-1 region affecting authentication service. Resolved after 45 minutes by scaling up read replicas.",
    "similarity_score": 0.87,
    "metadata": {
      "timestamp": "2024-12-05T14:30:00Z",
      "severity": "high",
      "service": "auth-db"
    }
  },
  {
    "node_id": "incident-2024-002",
    "content": "API gateway experiencing elevated error rates in production. Issue traced to rate limiting configuration.",
    "similarity_score": 0.82,
    "metadata": {
      "timestamp": "2024-12-07T09:15:00Z",
      "severity": "medium",
      "service": "api-gateway"
    }
  }
]

Search with Filters

from kubiya import ControlPlaneClient

client = ControlPlaneClient(api_key="your-api-key")

# Apply filters to narrow results
results = client.graph.semantic_search(
    query="kubernetes deployments",
    limit=20,
    filters={
        "labels": ["Deployment", "Kubernetes"],
        "properties": {
            "environment": "production",
            "status": "running"
        }
    }
)

print(f"Found {len(results)} production Kubernetes deployments")

Limit Results

from kubiya import ControlPlaneClient

client = ControlPlaneClient(api_key="your-api-key")

# Get top 3 most relevant results
top_results = client.graph.semantic_search(
    query="cost optimization opportunities",
    limit=3
)

for i, result in enumerate(top_results, 1):
    print(f"{i}. [{result['similarity_score']:.2f}] {result['content']}")

Practical Examples

1. Find Similar Resources

Find resources similar to a known resource:

from kubiya import ControlPlaneClient

def find_similar_resources(
    client: ControlPlaneClient,
    resource_description: str,
    limit: int = 5
):
    """Find resources similar to a description."""

    results = client.graph.semantic_search(
        query=resource_description,
        limit=limit
    )

    print(f"=== Similar Resources ===")
    for result in results:
        print(f"\nSimilarity: {result['similarity_score']:.2f}")
        print(f"Content: {result['content']}")

        if result.get('metadata'):
            print(f"Metadata: {result['metadata']}")

    return results

# Usage
client = ControlPlaneClient(api_key="your-api-key")

# Find resources similar to a description
similar = find_similar_resources(
    client,
    "Load balancer handling HTTPS traffic for web application",
    limit=5
)

2. Search Logs and Documentation

Search through operational logs and documentation:

from kubiya import ControlPlaneClient
from typing import List, Dict

def search_logs(
    client: ControlPlaneClient,
    query: str,
    min_score: float = 0.6
) -> List[Dict]:
    """Search logs with minimum similarity threshold."""

    results = client.graph.semantic_search(
        query=query,
        limit=50
    )

    # Filter by minimum score
    filtered = [r for r in results if r['similarity_score'] >= min_score]

    print(f"Found {len(filtered)} logs (score >= {min_score})")

    for result in filtered:
        timestamp = result.get('metadata', {}).get('timestamp', 'N/A')
        print(f"[{result['similarity_score']:.2f}] {timestamp}")
        print(f"  {result['content'][:150]}...")
        print()

    return filtered

# Usage
client = ControlPlaneClient(api_key="your-api-key")

# Search for error patterns
error_logs = search_logs(
    client,
    "connection timeout errors to database",
    min_score=0.7
)

# Search for successful deployments
deployment_logs = search_logs(
    client,
    "successful production deployments",
    min_score=0.6
)

Find incidents related to current issues:

from kubiya import ControlPlaneClient
from datetime import datetime

def find_related_incidents(
    client: ControlPlaneClient,
    incident_description: str,
    limit: int = 10
):
    """Find historical incidents related to current issue."""

    results = client.graph.semantic_search(
        query=incident_description,
        limit=limit,
        filters={
            "labels": ["Incident"],
            "properties": {
                "resolved": True
            }
        }
    )

    print(f"=== Related Historical Incidents ===")
    print(f"Query: {incident_description}\n")

    for i, result in enumerate(results, 1):
        print(f"{i}. Similarity: {result['similarity_score']:.2f}")
        print(f"   {result['content']}")

        if 'metadata' in result:
            resolution_time = result['metadata'].get('resolution_time', 'N/A')
            root_cause = result['metadata'].get('root_cause', 'N/A')
            print(f"   Resolution Time: {resolution_time}")
            print(f"   Root Cause: {root_cause}")

        print()

    return results

# Usage
client = ControlPlaneClient(api_key="your-api-key")

related = find_related_incidents(
    client,
    "API service returning 500 errors under high load",
    limit=5
)

# Extract patterns
if related:
    print("Common resolution patterns:")
    for result in related[:3]:
        if 'metadata' in result and 'resolution' in result['metadata']:
            print(f"  - {result['metadata']['resolution']}")

4. Knowledge Base Search

Search internal knowledge base and runbooks:

from kubiya import ControlPlaneClient

def search_knowledge_base(
    client: ControlPlaneClient,
    question: str
) -> Dict:
    """Search knowledge base for answers to questions."""

    results = client.graph.semantic_search(
        query=question,
        limit=5,
        filters={
            "labels": ["Documentation", "Runbook"]
        }
    )

    if not results:
        return {
            "answer": "No relevant documentation found",
            "sources": []
        }

    # Return top result as answer
    top_result = results[0]

    print(f"=== Knowledge Base Search ===")
    print(f"Question: {question}")
    print(f"Confidence: {top_result['similarity_score']:.2f}")
    print(f"\nAnswer:")
    print(top_result['content'])

    if len(results) > 1:
        print(f"\nRelated Articles:")
        for result in results[1:]:
            print(f"  [{result['similarity_score']:.2f}] {result['content'][:100]}...")

    return {
        "answer": top_result['content'],
        "confidence": top_result['similarity_score'],
        "sources": results
    }

# Usage
client = ControlPlaneClient(api_key="your-api-key")

# Search for procedures
answer = search_knowledge_base(
    client,
    "How do I roll back a production deployment?"
)

# Search for troubleshooting guides
answer = search_knowledge_base(
    client,
    "What to do when the database is slow?"
)

5. Content-Based Discovery

Discover resources based on content similarity:

from kubiya import ControlPlaneClient
from collections import defaultdict

def discover_by_content(
    client: ControlPlaneClient,
    theme: str,
    limit: int = 20
):
    """Discover resources related to a theme."""

    results = client.graph.semantic_search(
        query=theme,
        limit=limit
    )

    # Group by metadata categories
    by_category = defaultdict(list)

    for result in results:
        category = result.get('metadata', {}).get('category', 'uncategorized')
        by_category[category].append(result)

    print(f"=== Content Discovery: {theme} ===\n")

    for category, items in by_category.items():
        print(f"{category.upper()} ({len(items)} items)")
        for item in items[:3]:  # Top 3 per category
            print(f"  [{item['similarity_score']:.2f}] {item['content'][:80]}...")
        print()

    return {
        "theme": theme,
        "total_found": len(results),
        "by_category": dict(by_category)
    }

# Usage
client = ControlPlaneClient(api_key="your-api-key")

# Discover security-related content
security_content = discover_by_content(
    client,
    "security best practices and compliance requirements",
    limit=30
)

# Discover performance optimization content
perf_content = discover_by_content(
    client,
    "performance optimization and scalability patterns",
    limit=30
)

Error Handling

from kubiya import ControlPlaneClient
from kubiya.resources.exceptions import GraphError

client = ControlPlaneClient(api_key="your-api-key")

# Handle search errors
try:
    results = client.graph.semantic_search(
        query="test query",
        limit=10
    )
except GraphError as e:
    if "timeout" in str(e).lower():
        print(f"Search timed out: {e}")
    else:
        print(f"Semantic search failed: {e}")

# Handle empty results
results = client.graph.semantic_search(
    query="very specific query that matches nothing",
    limit=10
)

if not results:
    print("No relevant results found")
else:
    print(f"Found {len(results)} results")

# Handle low similarity scores
results = client.graph.semantic_search(query="query", limit=10)

high_quality = [r for r in results if r['similarity_score'] > 0.7]
if not high_quality:
    print("No high-confidence results found")
    print("Consider rephrasing your query or using intelligent search instead")

Best Practices

1. Use Descriptive Queries

# ❌ BAD - Too vague
results = client.graph.semantic_search(query="error", limit=10)

# ✅ GOOD - Specific and descriptive
results = client.graph.semantic_search(
    query="database connection timeout errors in production environment",
    limit=10
)

2. Filter Results by Score

results = client.graph.semantic_search(
    query="kubernetes deployment issues",
    limit=50
)

# Only use high-confidence results
relevant = [r for r in results if r['similarity_score'] > 0.7]

print(f"High confidence results: {len(relevant)}/{len(results)}")

3. Combine with Filters

# Use filters to narrow scope
results = client.graph.semantic_search(
    query="recent deployments",
    limit=20,
    filters={
        "labels": ["Deployment"],
        "properties": {
            "environment": "production",
            "status": "completed"
        }
    }
)

4. Handle Different Content Types

def search_with_fallback(client: ControlPlaneClient, query: str):
    """Search with fallback to intelligent search if no results."""

    # Try semantic search first
    results = client.graph.semantic_search(query=query, limit=10)

    if results and results[0]['similarity_score'] > 0.7:
        return {"type": "semantic", "results": results}

    # Fallback to intelligent search for complex queries
    print("Semantic search confidence low, using intelligent search...")
    intelligent_result = client.graph.intelligent_search(
        keywords=query,
        max_turns=5
    )

    return {"type": "intelligent", "result": intelligent_result}

API Reference

semantic_search()

def semantic_search(
    query: str,
    limit: int = 10,
    filters: Optional[Dict[str, Any]] = None
) -> List[Dict[str, Any]]

Parameters:

query (str): Natural language search query
limit (int): Maximum number of results (default 10)
filters (Optional[Dict]): Optional filters for labels and properties

Filters Structure:

{
    "labels": ["Label1", "Label2"],
    "properties": {
        "key1": "value1",
        "key2": "value2"
    }
}

Returns:

[
    {
        "node_id": str,
        "content": str,
        "similarity_score": float,
        "metadata": Dict
    }
]

Comparison with Other Search Methods

Semantic Search vs Intelligent Search

Feature	Semantic Search	Intelligent Search
Approach	Vector similarity	AI agent with tools
Speed	Fast (milliseconds)	Slower (seconds)
Depth	Single-step	Multi-turn reasoning
Use Case	Known concepts	Complex investigation
Results	Scored list	Natural language + data

When to use Semantic Search:

You know what you’re looking for conceptually
You need fast results
You want a ranked list of relevant content

When to use Intelligent Search:

You have complex, multi-part questions
You need reasoning across relationships
You want natural language explanations

Semantic Search vs Keyword Search

Semantic Search understands meaning, not just keywords:

# Keyword search might miss these
query = "resources wasting money"

# Semantic search finds:
# - "underutilized EC2 instances"
# - "overprovisioned databases"
# - "unused storage volumes"
# Even though they don't contain "wasting money"

Next Steps

Intelligent Search

AI-powered graph search with reasoning

Cognitive Memory

Store and recall context

Context Graph

Complete graph operations

Datasets

Manage cognitive datasets

Getting Started

Control Plane Client

Context Graph & Memory

Advanced

Resources

Semantic Search

Overview

Quick Start

Core Concepts

Vector Embeddings

Similarity Scores

Dataset Integration

Basic Usage

Simple Search

Search with Filters

Limit Results

Practical Examples

1. Find Similar Resources

2. Search Logs and Documentation

4. Knowledge Base Search

5. Content-Based Discovery

Error Handling

Best Practices

1. Use Descriptive Queries

2. Filter Results by Score

3. Combine with Filters

4. Handle Different Content Types

API Reference

semantic_search()

Comparison with Other Search Methods

Semantic Search vs Intelligent Search

Semantic Search vs Keyword Search

Next Steps

Intelligent Search

Cognitive Memory

Context Graph

Datasets

​Overview

​Quick Start

​Core Concepts

​Vector Embeddings

​Similarity Scores

​Dataset Integration

​Basic Usage

​Simple Search

​Search with Filters

​Limit Results

​Practical Examples

​1. Find Similar Resources

​2. Search Logs and Documentation

​3. Find Related Incidents

​4. Knowledge Base Search

​5. Content-Based Discovery

​Error Handling

​Best Practices

​1. Use Descriptive Queries

​2. Filter Results by Score

​3. Combine with Filters

​4. Handle Different Content Types

​API Reference

​semantic_search()

​Comparison with Other Search Methods

​Semantic Search vs Intelligent Search

​Semantic Search vs Keyword Search

​Next Steps

Intelligent Search

Cognitive Memory

Context Graph

Datasets

Overview

Quick Start

Core Concepts

Vector Embeddings

Similarity Scores

Dataset Integration

Basic Usage

Simple Search

Search with Filters

Limit Results

Practical Examples

1. Find Similar Resources

2. Search Logs and Documentation

3. Find Related Incidents

4. Knowledge Base Search

5. Content-Based Discovery

Error Handling

Best Practices

1. Use Descriptive Queries

2. Filter Results by Score

3. Combine with Filters

4. Handle Different Content Types

API Reference

semantic_search()

Comparison with Other Search Methods

Semantic Search vs Intelligent Search

Semantic Search vs Keyword Search

Next Steps