rag-echamor

RAG (Retrieval-Augmented Generation) assistant for Echamor data.

This project:

• Loads markdown knowledge documents from knowledge-base/
• Chunks and embeds them into a local Chroma database in preprocessed_db/
• Answers user questions with retrieval + reranking + LLM response generation
• Exposes an interactive chat UI with Gradio

Tech stack

• Python 3.11+
• uv for dependency and environment management
• ChromaDB for local vector storage
• OpenAI embeddings (text-embedding-3-large)
• LiteLLM for chat completions (openai/gpt-4.1-nano by default)
• Gradio for the chat interface

Project layout

• src/injest.py: ingestion pipeline (load docs, chunk, embed, index)
• src/answer.py: retrieval, reranking, query rewrite, final answer generation
• src/app.py: Gradio UI
• src/config.py: model/database/path configuration constants
• knowledge-base/: markdown source documents
• preprocessed_db/: persisted Chroma vector database
• notebooks/: exploratory notebook(s)

Prerequisites

1. Python 3.11+
2. uv installed
3. OpenAI API key

Setup

1. Install dependencies:

   uv sync

2. Create a .env file in the project root (or export env vars in your shell):

   OPENAI_API_KEY=your_key_here

Notes:

• python-dotenv is loaded by the app and ingestion scripts, so local .env values are picked up automatically.

Configuration

Main settings live in src/config.py:

• EMBEDDING_MODEL = "text-embedding-3-large"
• MODEL = "openai/gpt-4.1-nano"
• DB_NAME = "preprocessed_db"
• COLLECTION_NAME = "docs"
• KNOWLEDGE_BASE_PATH = "knowledge-base"

You can edit these constants to change models, collection name, or data paths.

Build the vector database (ingestion)

Run:

uv run python src/injest.py

What it does:

• Reads all markdown files recursively under knowledge-base/
• Splits content into chunks (LLM-assisted for larger files)
• Generates embeddings in batches
• Recreates the Chroma collection and stores all vectors/documents

Important ingestion knobs in src/injest.py:

• WORKERS (parallel chunking processes)
• MAX_EMBEDDING_BATCH_SIZE
• AVERAGE_CHUNK_SIZE, SINGLE_CHUNK_THRESHOLD

If you hit rate limits, lower WORKERS (the script already suggests setting it to 1).

Run the chat app

After ingestion:

uv run python src/app.py

Then open the Gradio URL shown in the terminal (the app also launches in your browser).

The UI includes:

• Chat conversation panel
• Retrieved context panel (source snippets used for answering)

Retrieval and answer flow

High-level pipeline in src/answer.py:

1. Rewrite user query to improve retrieval recall
2. Retrieve top chunks for both original and rewritten query
3. Merge results and rerank with the LLM
4. Keep top final chunks
5. Build a context-grounded system prompt and generate final answer

Using notebooks

Start JupyterLab:

uv run jupyter lab

Example notebook is available in notebooks/extract_data.ipynb.

Troubleshooting

• Missing API key: Ensure OPENAI_API_KEY is set in .env or shell environment.

• Empty or poor answers: Re-run ingestion after updating files in knowledge-base/.

• Rate limit or timeout errors during ingestion: Reduce WORKERS and retry.

• Unexpected retrieval behavior: Review model names and paths in src/config.py, then rebuild the vector DB.

Quick start summary

1. uv sync
2. Set OPENAI_API_KEY in .env
3. uv run python src/injest.py
4. uv run python src/app.py