From 2a34c42270ca9d412fd35bcc5398cd1158bf148f Mon Sep 17 00:00:00 2001 From: Kaloyan Danchev Date: Mon, 29 Jun 2026 22:14:41 +0300 Subject: [PATCH] =?UTF-8?q?docs(services):=20correct=20Ollama=20iGPU=20ent?= =?UTF-8?q?ry=20=E2=80=94=20remove=20flash-attn/q8=5F0,=20note=20large-pro?= =?UTF-8?q?mpt=20crash?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The flash-attention + q8_0 KV cache settings were removed after they (and the gfx1103 ROCm path generally) crash on large prompts. Document that the 780M iGPU is direct/interactive-chat only, not an agent backend (Hermes ~15.5k-token prompt crashed every call; reverted to cloud model). Co-Authored-By: Claude Opus 4.8 (1M context) --- docs/03-SERVICES-OTHER.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/03-SERVICES-OTHER.md b/docs/03-SERVICES-OTHER.md index 3dc0263..40e2f5e 100644 --- a/docs/03-SERVICES-OTHER.md +++ b/docs/03-SERVICES-OTHER.md @@ -510,13 +510,13 @@ Non-critical services that enhance functionality but don't affect core network o |---------|-------| | `HSA_OVERRIDE_GFX_VERSION` | 11.0.0 (report 780M as gfx1100) | | `OLLAMA_IGPU_ENABLE` | 1 (Ollama 0.30+ drops iGPUs otherwise) | -| `OLLAMA_FLASH_ATTENTION` | 1 | -| `OLLAMA_KV_CACHE_TYPE` | q8_0 | | devices | `/dev/kfd`, `/dev/dri` | | group_add | `18` (video) | **Verified:** Ornith 1.0 9B GGUF runs 100% on GPU (33/33 layers), ~11 tok/s. iGPU is RAM-bandwidth bound (shared GTT, ~23 GiB addressable). 9B fits; 35B RAM-tight (no swap); 397B not feasible. +**⚠️ Limitation:** large prompts (~10k+ tokens) crash the iGPU (`ROCm unspecified launch failure` — gfx1103 rocBLAS). Tried flash-attn off, gfx1100/gfx1102, `num_batch=128` — all crash, so `OLLAMA_FLASH_ATTENTION`/`OLLAMA_KV_CACHE_TYPE=q8_0` were removed. Good for direct/interactive chat only, **not** an agent backend with big system prompts (tried as Hermes' model with a ~15.5k-token prompt — crashed every call, reverted to cloud). + --- ## Stopped/Disabled Services