psi-king

framework for building multi-modal first document retriever

PSI King - King of the Senses from Psychonauts 2

Overview

• psiking-core docs

Document / Node (TextNode, ImageNode, TableNode)

• a Document contains a list of nodes (document.nodes)
• each node can be one of the following types
  • TextNode
  • ImageNode
  • TableNode
• schemas are defined here
  • detailed descriptions are available here

Ingestion Pipeline

Document Ingestion Flow example:

• (Doc) Collection -> Extraction -> Transformation -> Index(?)
  • Extraction: read file into Document instance
  • Transformation: merging/chunking/filtering
  • Index: Embedding & inserting into searchable DB

Example

allganize-RAG-Evaluation-Dataset-KO PDF dataset

Reading PDF files and indexing into qdrant DB for retrieval ([examples/allganize-rag-evaluation])

• data: real-life pdf files from allganize-RAG-Evaluation-Dataset-KO
• models: jina-embeddings-v4 + Qdrant/BM42
• db: qdrant (dense + sparse)

Pipeline Overview

BeIR (scifact) Retrieval Benchmark

Indexing BeIR text retrieval benchmark ([./examples/beir])

• data: scifact data from BeIR: hflink
• models: bge-m3, Qdrant/BM42
• db: qdrant (dense + sparse)

Experiments

Korean sparse search with vectordb

• pgvector [docs] [experiments]
  • use mecab-ko + textsearch_ko to enable korean tsvector calculation
• qdrant [docs] [experiments]
  • build qdrant with cjk language support for korean tokenization

Acknowledgements

• A lot of the structure of this project was inspired by llama-index
  • https://github.com/run-llama/llama_index
• document parsing heavily utilizes docling
  • https://github.com/DS4SD/docling
• PSI King is a character from Psychonauts 2
  • https://www.doublefine.com/games/psychonauts-2

History of this framework's development is recorded below

• https://github.com/id4thomas/nlp_building_blocks/tree/main/projects/2024_11_arxiver_rag/experiments