The RAG Pipeline
Ingestion (one-time)
Retrieval (per query)
Hybrid merge, temporal decay, and MMR are optional. Without them, the pipeline is simply Vector Search → Reranker → Results. See Hybrid Search & Scoring for details.
Quick Start
Path Shorthand
For the quickest setup, pass a directory path directly to the Agent:Adding Documents
Theadd() method accepts strings, file paths, or URLs. The appropriate reader is selected automatically:
Searching
Document objects sorted by relevance.
Async Support
Every method has an async variant:Components
Each step in the pipeline is pluggable:Documents
The core data unit — text content with metadata and embeddings.
Readers
Read text, PDF, and web content into documents.
Chunkers
Split large documents into smaller, overlapping chunks.
Embedders
Generate vector embeddings with OpenAI, Voyage AI, Google, or Mistral.
Rerankers
Rerank search results for higher relevance with Cohere or SentenceTransformer.
Vector Databases
Store and search embeddings with seven backends: InMemory, PgVector, Qdrant, ChromaDb, MongoDb, RedisDB, and PineconeDb.
Hybrid Search
Combine vector + full-text search, MMR diversity, and temporal decay.
Agent Integration
Connect knowledge to agents via middleware or toolkits.
Knowledge Parameters
VectorDB
Vector database for storing and searching embeddings. Defaults to
InMemoryVectorDB.Embedder
Embedding provider for converting text to vectors.
Reranker
Optional reranker for improving search result relevance.
Chunker
Text chunker for splitting documents. When
None (default), documents are not chunked automatically. Pass a RecursiveChunker or TextChunker instance to enable chunking during add().List[Reader]
Document readers. Defaults include
TextReader, PDFReader, URLReader.bool
default:"true"
Automatically detect the correct reader based on the source.
FTSIndex
Full-text search index for hybrid vector + keyword search. See Hybrid Search.
HybridSearchConfig
Configuration for merging vector and full-text search results (weights, merge strategy).
TemporalDecay
Exponential score decay based on document age. See Hybrid Search.
MMRConfig
Maximal Marginal Relevance for diversity reranking. See Hybrid Search.