Philipp MockandClaude Opus 5.5
d54b3e5b8b
- prepare_corpus.py: split double-page scans (split_spreads.py), re-OCR with OCRmyPDF/Tesseract only where the existing text layer is worse, Docling layout extraction with heading hierarchy, QA report per PDF - docling_loader.py: cleanup (margin noise, split paragraphs, footnote marks, OCR comma/period repair), page mapping back to the original PDF, chunk headers with author, title and section - local_rag.py: bge-m3 embeddings, configurable chunking and loader, index settings check (index_info.json), hybrid/union BM25 search with Reciprocal Rank Fusion and bge-reranker-v2-m3 cross-encoder; query instruction text removed after it lowered recall - eval_retrieval.py: CLI options, same retrieval path as the chat, tolerant file name matching; goldsets and evaluation reports for the Kafka corpus - grobid_test.py, grobid_qa.py, extract_docling.py, compare_extractors.py: extractor tests and quality checks - .gitignore: corpus PDFs, vector stores and extractor output stay local - server.py: uses ./vectorstore_docling_hdr Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
18 lines
931 B
Bash
Executable File
18 lines
931 B
Bash
Executable File
#!/bin/bash
|
|
# Messreihe: 4 Indizes (2000/400) x 3 Suchvarianten mit Reranker x 2 Fragensätze
|
|
cd "$(dirname "$0")/.."
|
|
P=/Users/Philipp/opt/anaconda3/envs/docling/bin/python
|
|
until [ -f vectorstore_docling_2000_hdr/index_info.json ] && [ -f vectorstore_docling_2000/index_info.json ] && [ -f vectorstore_pypdf_2000_hdr/index_info.json ]; do sleep 30; done
|
|
for GS in human synthetic; do
|
|
[ $GS = human ] && G=eval/goldset_kafka_human.csv || G=eval/goldset_kafka.csv
|
|
for I in pypdf_2000 pypdf_2000_hdr docling_2000 docling_2000_hdr; do
|
|
for M in dense hybrid union; do
|
|
tag="m2000_${GS}_${I}_${M}"
|
|
caffeinate -i $P eval_retrieval.py --goldset $G --index ./vectorstore_$I --report eval/kafka_${tag}.txt --mode $M > eval/eval_${tag}.log 2>&1
|
|
printf "%-10s %-18s %-7s " "$GS" "$I" "$M"
|
|
grep -E '^ (Recall@|MRR)' eval/kafka_${tag}.txt | awk '{printf "%s %s ", $1, $2}'; echo
|
|
done
|
|
done
|
|
done
|
|
echo FERTIG
|