pmock/localRAG
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

- eval mit gpt-6-luna

Browse Source 
- new chat interface with personas
This commit is contained in:
Philipp Mock
2026-10-08 10:06:32 +02:00
parent e75716714e
commit 30fd55da21
17 changed files with 876 additions and 315 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+6 -6
View File
@@ -52,7 +52,7 @@ ollama serve
In another terminal, pull a model once (example):
```bash
ollama pull gpt-oss:20b
ollama pull gemma4:12b
```
The model name must match what you configure in `server.py` (see [Configuration reference](#configuration-reference)).
@@ -170,9 +170,9 @@ Each entry in `retrieved` has:
Every answer reports which chunks it was based on, in two places:
- **Browser:** open the developer console (Chrome/Edge: `F12` or `Cmd`+`Option`+`I`, Firefox: `F12`) before sending a question. Each answer logs a collapsed group `[RAG] Retrieved N chunk(s)` with the full text, source file and page of every chunk.
- **Server terminal:** the terminal running `uvicorn` prints the rephrased standalone query, the retrieval query, the L2 distances, a 150-character preview of each chunk, and the raw LLM response.
- **Server terminal:** the terminal running `uvicorn` prints the search queries the LLM sent to the corpus search tool (or that it did not search), the scores, a 150-character preview of each chunk, and the raw LLM response.
Use this when an answer looks wrong: if the right chunks were never retrieved, tune `RETRIEVAL_K`, `QUERY_ADDITIONAL_INSTRUCTIONS` or the chunk size; if the chunks were correct but the answer was not, tune `ANSWER_PROMPT` (see [Configuration reference](#configuration-reference)).
Use this when an answer looks wrong: if the right chunks were never retrieved, tune `RETRIEVAL_K`, `QUERY_ADDITIONAL_INSTRUCTIONS` or the chunk size; if the chunks were correct but the answer was not, or the LLM searched when it should not (or vice versa), tune `SYSTEM_PROMPT` (see [Configuration reference](#configuration-reference)).
---
@@ -194,7 +194,7 @@ It loads the existing vector store, lists the documents in it, and runs **one ha
2. **Chunk** – `RecursiveCharacterTextSplitter` (defaults in [local_rag.py](local_rag.py)).
3. **Embed** – Hugging Face `sentence-transformers/all-MiniLM-L6-v2`.
4. **Store** – FAISS; retrieval uses `similarity_search_with_score`.
5. **Query** – Optional rephrase with chat history, retrieval, then answer from the LLM.
5. **Query** – The corpus search is a tool the LLM may call (`search_kafka_corpus`). It decides per message whether to search, writes the query itself from the chat history, and answers from the excerpts it found; greetings or thanks are answered without a search.
---
@@ -227,8 +227,8 @@ Changing the embedding model or chunk settings requires **rebuilding** the vecto
|------|--------|
| Default number of chunks retrieved (`k`) | Module-level `RETRIEVAL_K` (overrides: pass `k=` to `query` / `query_with_history`) |
| Extra text appended only to the **FAISS query** (biases retrieval, not the final answer phrasing) | `QUERY_ADDITIONAL_INSTRUCTIONS` (concatenated to the search query before embedding) |
| **Rephrase** prompt (standalone question when there is chat history) | String `rephrase_prompt = f"""..."""` inside `query_with_history` |
| **Answer** prompt – opening instructions only | Module-level `ANSWER_PROMPT` (edit the role / style lines). The block from chat history through `Answer:` is built in `query_with_history` |
| **System** prompt – when to search, how to answer | Module-level `SYSTEM_PROMPT` |
| Search tool name, description, searches per message | `SEARCH_TOOL_NAME`, `SEARCH_TOOL_DESCRIPTION`, `MAX_TOOL_ROUNDS` |
### [add_pdfs.py](add_pdfs.py)