pmock/localRAG
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add Docling corpus pipeline, retrieval evaluation and hybrid search with reranking

Browse Source 
- prepare_corpus.py: split double-page scans (split_spreads.py), re-OCR with
  OCRmyPDF/Tesseract only where the existing text layer is worse, Docling
  layout extraction with heading hierarchy, QA report per PDF
- docling_loader.py: cleanup (margin noise, split paragraphs, footnote marks,
  OCR comma/period repair), page mapping back to the original PDF, chunk
  headers with author, title and section
- local_rag.py: bge-m3 embeddings, configurable chunking and loader, index
  settings check (index_info.json), hybrid/union BM25 search with Reciprocal
  Rank Fusion and bge-reranker-v2-m3 cross-encoder; query instruction text
  removed after it lowered recall
- eval_retrieval.py: CLI options, same retrieval path as the chat, tolerant
  file name matching; goldsets and evaluation reports for the Kafka corpus
- grobid_test.py, grobid_qa.py, extract_docling.py, compare_extractors.py:
  extractor tests and quality checks
- .gitignore: corpus PDFs, vector stores and extractor output stay local
- server.py: uses ./vectorstore_docling_hdr

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Philipp Mock
2026-10-07 16:48:12 +02:00
co-authored by Claude Opus 5.5
parent 7d70169ee3
commit d54b3e5b8b
49 changed files with 5007 additions and 75766 deletions
Download Patch File Download Diff File Expand all files Collapse all files
eval/goldset.csv
+13
View File
@@ -0,0 +1,13 @@
frage,korpus,erwartete_quelle,erwartete_seite,anmerkung
"Which established assessments inspired the mathematical word problems used in the study?",bilder,dok1.pdf,8,"TIMSS und NEPS, Abschnitt Testing Material"
"How long did participation in the entire online study take on average?",bilder,dok1.pdf,10,"ca. 29 Minuten, Abschnitt Procedure"
"How were the decorative pictures presented in the dynamic condition?",bilder,dok1.pdf,9,"Slideshow, ein Bild pro Segment"
"Did dynamic decorative pictures perform better than static decorative pictures?",bilder,dok1.pdf,12,"Ergebnisteil, kein signifikanter Unterschied"
"Which scientific article did the participants read in the concept map experiment?",begriffskarten,dok2.PDF,6,"Eerland et al. 2011, Eiffel Tower"
"How many concepts were finally selected for the concept map?",begriffskarten,dok2.PDF,8,"24 Konzepte, sieben Teilbereiche"
"In which file format does the CoMap tool store nodes and connections?",begriffskarten,dok2.PDF,7,"JSON-Datei mit Knotenpositionen"
"What are the two hypotheses about gradually built-up concept maps?",begriffskarten,dok2.PDF,5,"H1 Wissenskonstruktion, H2 Faktenwissen"
"How were participants of the first smart speaker experiment recruited?",sprachassistent,dok3.pdf,5,"Prolific, wohnhaft in Deutschland"
"Which scale was used to measure media credibility of the smart speaker?",sprachassistent,dok3.pdf,6,"Flanagin und Metzger 2000, fuenf Items"
"What does the realism heuristic say about modality and trust?",sprachassistent,dok3.pdf,3,"MAIN-Modell, Sundar 2008"
"What was the average age of participants in the replication experiment?",sprachassistent,dok3.pdf,8,"28.2 Jahre, Experiment 2"
1 frage korpus erwartete_quelle erwartete_seite anmerkung
2 Which established assessments inspired the mathematical word problems used in the study? bilder dok1.pdf 8 TIMSS und NEPS, Abschnitt Testing Material
3 How long did participation in the entire online study take on average? bilder dok1.pdf 10 ca. 29 Minuten, Abschnitt Procedure
4 How were the decorative pictures presented in the dynamic condition? bilder dok1.pdf 9 Slideshow, ein Bild pro Segment
5 Did dynamic decorative pictures perform better than static decorative pictures? bilder dok1.pdf 12 Ergebnisteil, kein signifikanter Unterschied
6 Which scientific article did the participants read in the concept map experiment? begriffskarten dok2.PDF 6 Eerland et al. 2011, Eiffel Tower
7 How many concepts were finally selected for the concept map? begriffskarten dok2.PDF 8 24 Konzepte, sieben Teilbereiche
8 In which file format does the CoMap tool store nodes and connections? begriffskarten dok2.PDF 7 JSON-Datei mit Knotenpositionen
9 What are the two hypotheses about gradually built-up concept maps? begriffskarten dok2.PDF 5 H1 Wissenskonstruktion, H2 Faktenwissen
10 How were participants of the first smart speaker experiment recruited? sprachassistent dok3.pdf 5 Prolific, wohnhaft in Deutschland
11 Which scale was used to measure media credibility of the smart speaker? sprachassistent dok3.pdf 6 Flanagin und Metzger 2000, fuenf Items
12 What does the realism heuristic say about modality and trust? sprachassistent dok3.pdf 3 MAIN-Modell, Sundar 2008
13 What was the average age of participants in the replication experiment? sprachassistent dok3.pdf 8 28.2 Jahre, Experiment 2