Files
localRAG/eval/goldset.csv
T
Philipp MockandClaude Opus 5.5 d54b3e5b8b Add Docling corpus pipeline, retrieval evaluation and hybrid search with reranking
- prepare_corpus.py: split double-page scans (split_spreads.py), re-OCR with
  OCRmyPDF/Tesseract only where the existing text layer is worse, Docling
  layout extraction with heading hierarchy, QA report per PDF
- docling_loader.py: cleanup (margin noise, split paragraphs, footnote marks,
  OCR comma/period repair), page mapping back to the original PDF, chunk
  headers with author, title and section
- local_rag.py: bge-m3 embeddings, configurable chunking and loader, index
  settings check (index_info.json), hybrid/union BM25 search with Reciprocal
  Rank Fusion and bge-reranker-v2-m3 cross-encoder; query instruction text
  removed after it lowered recall
- eval_retrieval.py: CLI options, same retrieval path as the chat, tolerant
  file name matching; goldsets and evaluation reports for the Kafka corpus
- grobid_test.py, grobid_qa.py, extract_docling.py, compare_extractors.py:
  extractor tests and quality checks
- .gitignore: corpus PDFs, vector stores and extractor output stay local
- server.py: uses ./vectorstore_docling_hdr

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 16:48:12 +02:00

1.6 KiB

1fragekorpuserwartete_quelleerwartete_seiteanmerkung
2Which established assessments inspired the mathematical word problems used in the study?bilderdok1.pdf8TIMSS und NEPS, Abschnitt Testing Material
3How long did participation in the entire online study take on average?bilderdok1.pdf10ca. 29 Minuten, Abschnitt Procedure
4How were the decorative pictures presented in the dynamic condition?bilderdok1.pdf9Slideshow, ein Bild pro Segment
5Did dynamic decorative pictures perform better than static decorative pictures?bilderdok1.pdf12Ergebnisteil, kein signifikanter Unterschied
6Which scientific article did the participants read in the concept map experiment?begriffskartendok2.PDF6Eerland et al. 2011, Eiffel Tower
7How many concepts were finally selected for the concept map?begriffskartendok2.PDF824 Konzepte, sieben Teilbereiche
8In which file format does the CoMap tool store nodes and connections?begriffskartendok2.PDF7JSON-Datei mit Knotenpositionen
9What are the two hypotheses about gradually built-up concept maps?begriffskartendok2.PDF5H1 Wissenskonstruktion, H2 Faktenwissen
10How were participants of the first smart speaker experiment recruited?sprachassistentdok3.pdf5Prolific, wohnhaft in Deutschland
11Which scale was used to measure media credibility of the smart speaker?sprachassistentdok3.pdf6Flanagin und Metzger 2000, fuenf Items
12What does the realism heuristic say about modality and trust?sprachassistentdok3.pdf3MAIN-Modell, Sundar 2008
13What was the average age of participants in the replication experiment?sprachassistentdok3.pdf828.2 Jahre, Experiment 2