This computational showcase isolates one audited result from a larger PCB-document experiment: replacing raw CLIP token vectors with a late-interaction-native ColPali encoder while holding the page-level MaxSim scorer, qrels, run format, and evaluation code fixed.
The benchmark contains 10 PCB documents, 101 pages, and 30 page-retrieval queries.
| Method | Recall@10 | nDCG@10 |
|---|---|---|
| CLIP full multi-vector | 0.1333 | 0.0807 |
| ColPali v1.3 full multi-vector | 0.9333 | 0.6832 |
| OCR BM25 | 0.8833 | 0.5241 |
These are point estimates from 30 queries with no confidence intervals. They demonstrate a large encoder-alignment difference in this pilot, but they do not establish statistical equivalence or superiority.
The exact aggregate source records are under results/source/, and the normalized public claim table is results/headline_claims.csv. Claim-level provenance is recorded in results/claim_provenance.csv.
scripts/retrieval/run_bm25.py: OCR BM25 baseline.scripts/retrieval/run_colpali_page_embeddings.py: ColPali page multi-vectors.scripts/retrieval/run_colpali_query_embeddings.py: ColPali query multi-vectors.scripts/retrieval/run_full_multivector_retrieval.py: shared dot-product/sum-MaxSim ranking path.scripts/evaluation/eval_full_multivector_run.py: page-level Recall, MRR, and nDCG evaluation.
The included synthetic test checks the MaxSim and evaluation primitives without downloading data or models. It does not reproduce the headline values.
Previously public analysis, preprocessing, retrieval, evaluation, and visualization utilities remain under scripts/ together with their historical configurations. They are retained to avoid unnecessary history loss and are documented-only; their presence does not publish the earlier CLIP-budget, Visual-RAG, or occlusion results as supported claims.
This is a narrow computational showcase, not a complete replication package.
- Source and derived document pages are excluded because the corpus contains third-party vendor material and one confidential drawing.
- The 101-page corpus therefore cannot be reconstructed from this repository alone.
- Qrels, query rows, OCR text, record-level runs, embeddings, model weights, logs, figures, notes, and paper tables are not distributed.
- Historical code for earlier token-budget, compression, Visual-RAG, and occlusion analyses is retained, but those analyses used the CLIP branch and were not rerun with ColPali; their outputs and claims remain outside this public claim surface.
- Full headline-result reproduction is
unavailablefrom this tree. Aggregate records areverified; the bounded three-test synthetic suite issmoke-testedin the isolated Python 3.11 environment.
See docs/reproducing.md, docs/data-and-models.md, docs/limitations.md, and docs/repository-scope.md.
Project code is distributed under the existing Apache License 2.0. Named dependencies, models, and datasets retain their own terms; no rights to those external assets are granted here.