Knowledge Sources

Knowledge sources provide Kubiya agents with the context and information they need to deliver accurate, relevant responses and take appropriate actions. By connecting agents to knowledge sources, you can customize them with domain-specific information without additional training.

What Are Knowledge Sources?

Knowledge sources are collections of information that agents can reference during conversations. Unlike the general knowledge contained in foundation models, these sources provide:

• Organization-specific information: Internal documentation, policies, and procedures
• Technical context: System architecture, infrastructure details, and troubleshooting guides
• Domain expertise: Specialized knowledge relevant to your business or industry
• Current data: Up-to-date information that may not be in the model's training data

Types of Knowledge Sources

Kubiya supports several types of knowledge sources:

Document Repositories

• Documentation sites
• Wikis and knowledge bases
• PDF libraries
• Confluence spaces
• SharePoint sites

Code and Configuration

• GitHub/GitLab repositories
• Infrastructure as Code files
• Configuration management databases
• API specifications

Structured Data

• Databases
• CSV files
• JSON/YAML datasets
• Spreadsheets

Live Systems

• Service catalogs
• CMDB entries
• Asset management systems
• Monitoring dashboards

How Knowledge Sources Work

When an agent receives a query:

1. The agent analyzes the request to determine what information is needed
2. It searches relevant knowledge sources for context
3. Retrieved information is processed and integrated into the agent's reasoning
4. The agent generates a response based on both its core capabilities and the specific knowledge

flowchart LR
  User([User]) --> |Query| Agent[AI Agent]
  
  subgraph "Knowledge Processing"
    Agent --> |1. Analyze Query| QueryAnalysis[Query Analysis]
    QueryAnalysis --> |2. Determine Info Needs| SearchPlanning[Search Planning]
    SearchPlanning --> |3. Retrieve from Sources| Retrieval[Knowledge Retrieval]
    Retrieval --> |4. Process & Filter| Processing[Relevance Processing]
    Processing --> |5. Context Integration| Integration[Context Integration]
    Integration --> |6. Response Generation| Response[Response Generation]
  end
  
  subgraph "Knowledge Sources"
    Docs[(Documentation)]
    Code[(Code Repositories)]
    Data[(Structured Data)]
    Wiki[(Wikis/Confluence)]
    Live[(Live Systems)]
  end
  
  Retrieval --> |Document Query| Docs
  Retrieval --> |Code Search| Code
  Retrieval --> |Data Query| Data
  Retrieval --> |Wiki Search| Wiki
  Retrieval --> |System Query| Live
  
  Docs --> |Results| Processing
  Code --> |Results| Processing
  Data --> |Results| Processing
  Wiki --> |Results| Processing
  Live --> |Results| Processing
  
  Response --> |Answer with Citations| User
  
  classDef user fill:#f2f2f2,stroke:#333,stroke-width:2px
  classDef agent fill:#e6f7ff,stroke:#1890ff,stroke-width:2px
  classDef process fill:#f6ffed,stroke:#52c41a,stroke-width:2px
  classDef sources fill:#fff2e8,stroke:#fa541c,stroke-width:2px
  
  class User user
  class Agent agent
  class QueryAnalysis,SearchPlanning,Retrieval,Processing,Integration,Response process
  class Docs,Code,Data,Wiki,Live sources

Setting Up Knowledge Sources

To add a knowledge source to your Kubiya environment:

1. Identify valuable sources: Determine what information would help your agents
2. Prepare the content: Organize and format information for effective retrieval
3. Configure connections: Set up access to the source repositories
4. Index the content: Process the information for quick agent reference
5. Assign to agents: Connect relevant knowledge sources to specific agents

Knowledge Source Configuration

A typical knowledge source configuration includes:

name: infrastructure-documentation
description: "Infrastructure documentation and runbooks"
type: document-collection
source:
  type: confluence
  url: "https://company-wiki.atlassian.net"
  space: "INFRA"
authentication:
  type: api-token
  token_env: "CONFLUENCE_API_TOKEN"
refresh:
  schedule: "0 0 * * *"  # Daily at midnight
  on_change: true
processing:
  chunk_size: 1000
  overlap: 200
filters:
  - include: ["runbooks/*", "architecture/*"]
  - exclude: ["drafts/*", "archive/*"]

Knowledge Retrieval Methods

Kubiya uses several techniques to retrieve and utilize knowledge:

Semantic Search

Finds information based on meaning rather than exact keyword matches

Chunking

Breaks large documents into manageable pieces for more precise retrieval

Cross-referencing

Connects related information across different sources

Citation

Tracks the origin of information to provide references in responses

Best Practices for Knowledge Sources

For optimal knowledge integration:

• Keep content organized: Structure information logically for better retrieval
• Update regularly: Ensure information remains current and accurate
• Include context: Provide sufficient background for standalone understanding
• Balance coverage: Include enough detail without overwhelming the system
• Monitor usage: Track which sources are most helpful and refine accordingly

Common Knowledge Source Use Cases

Effective use cases for knowledge sources include:

• IT support: Technical documentation and troubleshooting guides
• HR assistance: Policy documents and employee handbooks
• Compliance: Regulations, standards, and procedure documents
• Product information: Specifications, FAQs, and user guides
• Customer support: Known issues, solutions, and service catalogs

Example: Enhancing Agent Responses

Without knowledge sources:

User: "How do I request access to the production database?"

Agent: "Generally, database access requests require approval from a database administrator. Would you like me to help you find your organization's process for this?"

With knowledge sources:

User: "How do I request access to the production database?"

Agent: "To request production database access, you'll need to:

1. Submit a ticket in ServiceNow using the 'Database Access Request' template
2. Select 'Production' environment and specify which databases you need
3. Include business justification and duration needed
4. Your request will be reviewed by the Database Team (SLA: 24 hours)
5. Once approved, credentials will be provisioned through Vault

Would you like me to initiate this request for you?"

Next Steps

• Learn about Agents that use knowledge sources
• Explore Tools that extend agent capabilities
• Connect your first knowledge source