Add pinned conda environments for arm64 and x86

- envs/environment-docling.yml (arm64, Python 3.12) and envs/environment-local_rag.yml (x86_64, Python 3.10) with exact pip versions and create commands in the header - README and EINSTIEG: how to create the environments; x86 tested for chat, search and indexing (search about 2.3 s under Rosetta) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 07:49:17 +01:00
parent ce9a863a87
commit 9d7670f8ad
4 changed files with 313 additions and 6 deletions
@@ -9,11 +9,12 @@ Stand: Oktober 2026. Ausführliche englische Referenz: [README.md](README.md).
 | Umgebung | Architektur | Einsatz |
 |---|---|---|
 | `docling` | arm64 (Apple Silicon), Python 3.12, torch 2.14 | Standard. Auch für Docling (`prepare_corpus.py`). |
-| `local_rag` | x86_64 (Intel-Mac oder Rosetta), Python 3.10, torch 2.2.2 | Alternative für x86. Chat und Suche getestet; Indexbau sollte gehen (gleiches Embedding), ungetestet. Docling nicht. |
+| `local_rag` | x86_64 (Intel-Mac oder Rosetta), Python 3.10, torch 2.2.2 | Alternative für x86. Chat, Suche und Indexbau getestet; Docling läuft hier nicht. |

 - **Warum bge-m3 auf x86 nur über safetensors lädt:** Für macOS x86 gibt es höchstens torch 2.2.2. `transformers` lädt `.bin`-Gewichte aber erst ab torch 2.6, und bge-m3 hat im Hauptzweig nur `pytorch_model.bin`.
 - **Die Lösung im Code:** `local_rag.py` lädt deshalb die offizielle safetensors-Konvertierung (`EMBEDDING_REVISION`). Die Gewichte sind dieselben, die Vektoren in beiden Umgebungen identisch, und dieselben Indizes funktionieren in beiden.
- **Geschwindigkeit unter Rosetta:** Spürbar langsamer. Im Test brauchte die Suche gut 5 statt gut 1 Sekunde, der Serverstart war dagegen nicht langsamer.
+- **Neu anlegen:** Mit festen Versionen aus `envs/environment-docling.yml` bzw. `envs/environment-local_rag.yml`. Die Befehle stehen jeweils im Kopf der Datei.
+- **Geschwindigkeit unter Rosetta:** Spürbar langsamer. Im Test brauchte die Suche etwa 2,3 statt gut 1 Sekunde, der Serverstart war dagegen nicht langsamer.

 **Voraussetzungen (einmalig)**
 - Ollama mit dem Modell: `ollama pull gemma4:12b`.
@@ -11,7 +11,7 @@ A short German quick start is in [EINSTIEG.md](EINSTIEG.md).
 | Conda env | Architecture | Use |
 |---|---|---|
 | `docling` | arm64 (Apple Silicon), Python 3.12, torch 2.14 | Default. Also needed for Docling (`prepare_corpus.py`). |
-| `local_rag` | x86_64 (Intel Mac or Rosetta), Python 3.10, torch 2.2.2 | Alternative for x86. Chat and search work; Docling does not. Search is slower under Rosetta (about 5 s instead of about 1 s). |
+| `local_rag` | x86_64 (Intel Mac or Rosetta), Python 3.10, torch 2.2.2 | Alternative for x86. Chat, search and indexing tested; Docling does not run. Search is slower under Rosetta (about 2.3 s instead of about 1 s). |

 **bge-m3 on x86**

@@ -20,15 +20,17 @@ A short German quick start is in [EINSTIEG.md](EINSTIEG.md).

 **New environment**

+[envs/](envs/) holds one file per environment with exact versions, exported from the working environments. The header of each file has the exact commands.
+
 ```bash
-conda create -n local_rag python=3.10 -y
+CONDA_SUBDIR=osx-arm64 conda env create -f envs/environment-docling.yml
 ```

 ```bash
-conda activate local_rag && pip install -r requirements.txt
+CONDA_SUBDIR=osx-64 conda env create -f envs/environment-local_rag.yml
 ```

-For the arm64 env on Apple Silicon, create it with `CONDA_SUBDIR=osx-arm64` so that conda does not fall back to x86 packages.
+`CONDA_SUBDIR` forces the architecture; without it, conda on Apple Silicon may pick the wrong one. [requirements.txt](requirements.txt) lists only the direct dependencies, without versions.

 **Ollama**

@@ -0,0 +1,190 @@
+# Conda environment "docling" (arm64, Python 3.12.11), exact package versions.
+# Generated 2026-10-26 from the working environment.
+# Create on Apple Silicon (arm64 packages enforced):
+#   CONDA_SUBDIR=osx-arm64 conda env create -f envs/environment-docling.yml
+#   conda activate docling && conda config --env --set subdir osx-arm64
+# OCR for prepare_corpus.py also needs Tesseract and Ghostscript (Homebrew:
+# brew install tesseract ghostscript) and deu.traineddata from tesseract-ocr/tessdata_best in ./tessdata.
+name: docling
+channels:
+  - defaults
+dependencies:
+  - python=3.12.11
+  - pip
+  - pip:
+      - accelerate==1.15.0
+      - aiohappyeyeballs==2.7.1
+      - aiohttp==3.14.4
+      - aiosignal==1.4.0
+      - annotated-doc==0.0.5
+      - annotated-types==0.8.0
+      - antlr4-python3-runtime==4.9.3
+      - anyio==4.15.1
+      - attrs==26.1.0
+      - beautifulsoup4==4.15.0
+      - certifi==2026.7.22
+      - cffi==2.1.1
+      - charset-normalizer==3.5.2
+      - click==8.5.0
+      - cloudpickle==3.1.2
+      - colorlog==6.12.0
+      - cryptography==50.0.2
+      - defusedxml==0.7.1
+      - dill==0.4.1
+      - distro==1.9.0
+      - doclang==0.7.3
+      - docling-core==2.99.0
+      - docling-ibm-models==4.0.3
+      - docling-parse==7.22.2
+      - docling-slim==2.134.0
+      - docling==2.134.0
+      - et_xmlfile==2.0.0
+      - faiss-cpu==1.15.1
+      - Faker==40.41.0
+      - fastapi==0.142.2
+      - filelock==4.0.12
+      - filetype==1.2.0
+      - fonttools==4.66.1
+      - fpdf2==2.8.9
+      - frozenlist==1.8.0
+      - fsspec==2026.9.0
+      - h11==0.16.0
+      - hf-xet==1.6.0
+      - httpcore2==2.13.1
+      - httpcore==1.0.9
+      - httptools==0.8.0
+      - httpx-sse==0.4.3
+      - httpx2==2.13.1
+      - httpx==0.28.1
+      - huggingface_hub==1.33.0
+      - idna==3.20
+      - img2pdf==0.6.3
+      - Jinja2==3.1.6
+      - jiter==0.17.0
+      - joblib==1.6.0
+      - jsonpatch==1.33
+      - jsonpointer==3.1.1
+      - jsonref==1.1.0
+      - jsonschema-specifications==2025.9.1
+      - jsonschema==4.26.0
+      - langchain-classic==1.0.8
+      - langchain-community==0.4.2
+      - langchain-core==1.6.7
+      - langchain-huggingface==1.2.2
+      - langchain-ollama==1.1.0
+      - langchain-openai==1.6.7
+      - langchain-protocol==0.0.19
+      - langchain-text-splitters==1.1.3
+      - langchain==1.4.3
+      - langcodes==3.5.1
+      - langgraph-checkpoint==4.2.0
+      - langgraph-prebuilt==1.1.0
+      - langgraph-sdk==0.4.6
+      - langgraph==1.2.14
+      - langsmith==0.14.4
+      - latex2mathml==3.81.1
+      - lxml==6.1.3
+      - mail-parser==4.8.0
+      - markdown-it-py==4.2.0
+      - marko==2.2.4
+      - MarkupSafe==3.0.4
+      - mdurl==0.1.2
+      - mpire==2.10.2
+      - mpmath==1.3.0
+      - multidict==6.9.1
+      - multiprocess==0.70.19
+      - narwhals==2.26.0
+      - networkx==3.7
+      - numpy==2.5.3
+      - ocrmac==1.0.1
+      - ocrmypdf==17.13.0
+      - olefile==0.47
+      - ollama==0.6.3
+      - omegaconf==2.3.1
+      - openai==3.26.0
+      - opencv-python==5.0.0.93
+      - openpyxl==3.1.5
+      - opentelemetry-api==1.45.1
+      - orjson==3.12.0
+      - ormsgpack==1.12.2
+      - packaging==26.3
+      - pandas==3.0.6
+      - pdfminer.six==20260107
+      - pikepdf==10.16.0
+      - pillow==12.3.0
+      - pluggy==1.6.0
+      - polyfactory==3.3.0
+      - propcache==0.5.4
+      - psutil==7.2.2
+      - pyclipper==1.4.0
+      - pycparser==3.0
+      - pydantic-settings==2.15.0
+      - pydantic==2.13.5
+      - pydantic_core==2.46.5
+      - Pygments==2.21.0
+      - pylatexenc==2.11
+      - pyobjc-core==12.2.2
+      - pyobjc-framework-Cocoa==12.2.2
+      - pyobjc-framework-CoreML==12.2.2
+      - pyobjc-framework-Quartz==12.2.2
+      - pyobjc-framework-Vision==12.2.2
+      - pypdf==6.19.0
+      - pypdfium2==5.14.0
+      - python-dateutil==2.9.0.post0
+      - python-docx==1.2.0
+      - python-dotenv==1.2.4
+      - python-oxmsg==0.0.2
+      - python-pptx==1.0.2
+      - PyYAML==6.0.3
+      - rank-bm25==0.2.2
+      - rapidocr==3.9.2
+      - referencing==0.37.0
+      - regex==2026.9.29
+      - requests-toolbelt==1.0.0
+      - requests==2.34.2
+      - rich==15.0.0
+      - rpds-py==2026.9.1
+      - rtree==1.4.1
+      - safetensors==0.8.0
+      - scikit-learn==1.9.1
+      - scipy==1.18.1
+      - semchunk==3.2.5
+      - sentence-transformers==6.1.0
+      - shapely==2.1.2
+      - shellingham==1.5.4
+      - six==1.17.0
+      - sniffio==1.3.1
+      - snowballstemmer==3.1.1
+      - soupsieve==2.10
+      - SQLAlchemy==2.1.3
+      - starlette==1.7.0
+      - sympy==1.14.0
+      - tabulate==0.10.0
+      - tenacity==9.1.4
+      - threadpoolctl==3.7.0
+      - tiktoken==0.14.0
+      - tokenizers==0.23.2
+      - torch==2.14.1
+      - torchvision==0.29.1
+      - tqdm==4.70.1
+      - transformers==5.18.0
+      - tree-sitter-c==0.24.2
+      - tree-sitter-javascript==0.25.0
+      - tree-sitter-python==0.25.0
+      - tree-sitter-typescript==0.23.2
+      - tree-sitter==0.26.0
+      - truststore==0.10.4
+      - typer==0.26.8
+      - typing-inspection==0.4.4
+      - typing_extensions==4.16.0
+      - uharfbuzz==0.56.2
+      - urllib3==2.8.0
+      - uuid_utils==0.17.1
+      - uvicorn==0.54.0
+      - uvloop==0.23.0
+      - watchfiles==1.3.0
+      - websockets==16.1.1
+      - xlsxwriter==3.2.9
+      - xxhash==4.0.1
+      - yarl==1.25.1
+      - zstandard==0.25.0
@@ -0,0 +1,114 @@
+# Conda environment "local_rag" (x86_64, Python 3.10.18), exact package versions.
+# Generated 2026-10-26 from the working environment.
+# Alternative for x86_64 (Intel Mac or Rosetta). torch 2.2.2 is the last version for
+# macOS x86; bge-m3 is therefore loaded as safetensors (EMBEDDING_REVISION in local_rag.py).
+# Docling does not run here.
+# Create (on Apple Silicon, force x86 packages):
+#   CONDA_SUBDIR=osx-64 conda env create -f envs/environment-local_rag.yml
+#   conda activate local_rag && conda config --env --set subdir osx-64
+name: local_rag
+channels:
+  - defaults
+dependencies:
+  - python=3.10.18
+  - pip
+  - pip:
+      - aiohappyeyeballs==2.6.1
+      - aiohttp==3.13.3
+      - aiosignal==1.4.0
+      - annotated-doc==0.0.4
+      - annotated-types==0.7.0
+      - anyio==4.12.1
+      - async-timeout==4.0.3
+      - attrs==25.4.0
+      - certifi==2026.1.4
+      - charset-normalizer==3.4.4
+      - click==8.3.1
+      - dataclasses-json==0.6.7
+      - distro==1.9.0
+      - exceptiongroup==1.3.1
+      - faiss-cpu==1.13.2
+      - fastapi==0.128.0
+      - filelock==3.20.3
+      - frozenlist==1.8.0
+      - fsspec==2026.1.0
+      - greenlet==3.3.1
+      - h11==0.16.0
+      - hf-xet==1.2.0
+      - httpcore==1.0.9
+      - httptools==0.7.1
+      - httpx-sse==0.4.3
+      - httpx==0.28.1
+      - huggingface-hub==0.36.0
+      - idna==3.11
+      - Jinja2==3.1.6
+      - jiter==0.13.0
+      - joblib==1.5.3
+      - jsonpatch==1.33
+      - jsonpointer==3.0.0
+      - langchain-classic==1.0.1
+      - langchain-community==0.4.1
+      - langchain-core==1.2.11
+      - langchain-huggingface==1.2.0
+      - langchain-ollama==1.0.1
+      - langchain-openai==1.1.9
+      - langchain-text-splitters==1.1.0
+      - langchain==1.2.7
+      - langgraph-checkpoint==4.0.0
+      - langgraph-prebuilt==1.0.7
+      - langgraph-sdk==0.3.3
+      - langgraph==1.0.7
+      - langsmith==0.6.4
+      - MarkupSafe==3.0.3
+      - marshmallow==3.26.2
+      - mpmath==1.3.0
+      - multidict==6.7.1
+      - mypy_extensions==1.1.0
+      - networkx==3.4.2
+      - numpy==1.26.4
+      - ollama==0.6.1
+      - openai==2.20.0
+      - orjson==3.11.5
+      - ormsgpack==1.12.2
+      - packaging==25.0
+      - propcache==0.4.1
+      - pydantic-settings==2.12.0
+      - pydantic==2.12.5
+      - pydantic_core==2.41.5
+      - pypdf==6.6.2
+      - python-dotenv==1.2.1
+      - PyYAML==6.0.3
+      - rank-bm25==0.2.2
+      - regex==2026.1.15
+      - requests-toolbelt==1.0.0
+      - requests==2.32.5
+      - safetensors==0.7.0
+      - scikit-learn==1.7.2
+      - scipy==1.15.3
+      - sentence-transformers==5.2.1
+      - shellingham==1.5.4
+      - sniffio==1.3.1
+      - snowballstemmer==3.1.1
+      - SQLAlchemy==2.0.46
+      - starlette==0.50.0
+      - sympy==1.14.0
+      - tenacity==9.1.2
+      - threadpoolctl==3.6.0
+      - tiktoken==0.12.0
+      - tokenizers==0.22.2
+      - torch==2.2.2
+      - tqdm==4.67.1
+      - transformers==4.57.6
+      - typer-slim==0.21.1
+      - typing-inspect==0.9.0
+      - typing-inspection==0.4.2
+      - typing_extensions==4.15.0
+      - urllib3==2.6.3
+      - uuid_utils==0.14.0
+      - uvicorn==0.40.0
+      - uvloop==0.22.1
+      - watchfiles==1.1.1
+      - websockets==16.0
+      - xxhash==3.6.0
+      - yarl==1.22.0
+      - zstandard==0.25.0