Building a RAG Pipeline
Retrieval-Augmented Generation (RAG) combines the power of large language models with your own documents and data to provide accurate, contextual responses. This guide walks you through building a complete RAG pipeline using Kamiwaza's core services.
What You'll Build
By the end of this guide, you'll have:
- Document ingestion system that processes various file formats
- Embedding pipeline that converts text to vector representations
- Vector search system for finding relevant context
- LLM integration that generates responses using retrieved context
- Web interface for querying your documents
Prerequisites
Before starting, ensure you have:
- Kamiwaza installed and running (Installation Guide)
- At least 16GB of available RAM
- Sample documents (markdown format) to process
- Basic familiarity with Python (for SDK examples)
Architecture Overview
A RAG pipeline consists of four main components:
Step 1: Deploy Required Models
First, we'll deploy an embedding model for vectorizing text and a language model for generating responses.
Deploy an Embedding Model
The embedding model will be automatically loaded when you create an embedder - no manual deployment needed:
from kamiwaza_client import KamiwazaClient
client = KamiwazaClient(base_url="http://localhost:7777/api/")
# The embedding model will be automatically loaded when you create an embedder
# This happens seamlessly in the background
embedder = client.embedding.get_embedder(
model="BAAI/bge-base-en-v1.5",
provider_type="huggingface_embedding"
)
print("✅ Embedding model ready for use")
Deploy a Language Model
Deploy a language model using Kamiwaza for response generation:
from kamiwaza_client import KamiwazaClient
client = KamiwazaClient(base_url="http://localhost:7777/api/")
# Search for a suitable language model
model_repo = "Qwen/Qwen3-0.6B-GGUF" # Small efficient model
models = client.models.search_models(model_repo, exact=True)
print(f"Found model: {models[0]}")
# Download the model (this may take a few minutes)
print("Downloading model...")
client.models.initiate_model_download(model_repo)
client.models.wait_for_download(model_repo)
print("✅ Model download complete")
# Deploy the model
print("Deploying model...")
deployment_id = client.serving.deploy_model(repo_id=model_repo)
print(f"✅ Model deployed with ID: {deployment_id}")
# Get OpenAI-compatible client for the deployed model
openai_client = client.openai.get_client(repo_id=model_repo)
print("✅ OpenAI-compatible client ready")
Check Deployment Status
# List active deployments to verify
deployments = client.serving.list_active_deployments()
for deployment in deployments:
print(f"✅ {deployment.m_name} is {deployment.status}")
print(f" Endpoint: {deployment.endpoint}")
Step 2: Document Ingestion Pipeline
Now we'll create a pipeline to process documents, chunk them, and generate embeddings.
Document Processing Script
import os
from pathlib import Path
from typing import List, Dict
from kamiwaza_client import KamiwazaClient
class RAGPipeline:
def __init__(self, base_url="http://localhost:7777/api/"):
self.client = KamiwazaClient(base_url=base_url)
self.embedding_model = "BAAI/bge-base-en-v1.5" # Use a proven working model
self.collection_name = "documents"
# Initialize global embedder to prevent cleanup between operations
self.embedder = self.client.embedding.get_embedder(
model=self.embedding_model,
provider_type="huggingface_embedding"
)
print(f"✅ RAG Pipeline initialized with model: {self.embedding_model}")
def add_documents_to_catalog(self, filepaths: List[str]) -> List:
"""Add documents to the Kamiwaza catalog."""
datasets = []
for filepath in filepaths:
try:
# Create dataset for each file
dataset = self.client.catalog.create_dataset(
dataset_name=filepath,
platform="file",
environment="PROD",
description=f"RAG document: {Path(filepath).name}"
)
if dataset.urn:
datasets.append(dataset)
print(f"✅ Added to catalog: {Path(filepath).name}")
except Exception as e:
print(f"