diff --git a/docs/wip/LOCAL-AI-STACK.md b/docs/wip/LOCAL-AI-STACK.md new file mode 100644 index 0000000..30861ae --- /dev/null +++ b/docs/wip/LOCAL-AI-STACK.md @@ -0,0 +1,150 @@ +# WIP: Local AI Stack on Unraid + +**Status:** Planning +**Created:** 2026-01-25 + +--- + +## Overview + +Deploy a hardware-accelerated local AI stack on Unraid (192.168.31.2), initially on N100 (Intel iGPU) with future migration path to N5 Air (AMD 780M iGPU). + +--- + +## Phase 1: Local AI Configuration (Current N100) + +### 1. Hardware-Accelerated AI Stack + +Install via Community Applications (Apps) tab: + +| App | Purpose | Configuration | +|-----|---------|---------------| +| Intel GPU Top | N100 QuickSync iGPU support | Required for GPU acceleration | +| Ollama | LLM runtime | Extra Parameters: `--device /dev/dri` | +| Open WebUI | Chat interface | `OLLAMA_BASE_URL=http://192.168.31.2:11434` | + +**Ollama Docker Template:** +``` +Extra Parameters: --device /dev/dri +Port: 11434 +``` + +**Open WebUI Docker Template:** +``` +OLLAMA_BASE_URL=http://192.168.31.2:11434 +Port: 3080 (or available port) +``` + +**PWA Setup:** Open WebUI on phone/tablet → "Add to Home Screen" for native experience. + +### 2. SSH Bridge & Terminal Agent (Aider) + +**SSH Key Setup on Unraid:** +```bash +# Create directory +mkdir -p /mnt/user/appdata/ssh_keys + +# Generate AI agent key +ssh-keygen -t ed25519 -f /mnt/user/appdata/ssh_keys/ai_agent -N "" + +# Deploy to MikroTik (via existing key) +# The AI can then manage MikroTik remotely +``` + +**Aider Configuration:** +```bash +export OLLAMA_API_BASE=http://192.168.31.2:11434 +aider --model ollama_chat/qwen2.5-coder:14b +``` + +### 3. Sanitized Knowledge Base + +Upload `topology.md` to Open WebUI → Workspace → Knowledge section. + +--- + +## Phase 2: Hardware Migration (N100 → N5 Air) + +### 1. Clean Break (On N100) + +1. Stop all Docker containers +2. Stop Array: Main tab → Stop +3. Disable Auto-Start: Settings → Disk Settings → Enable Auto-Start: No +4. Uninstall "Intel GPU Top" plugin +5. Shutdown N100 + +### 2. N5 Air BIOS Configuration + +Move Unraid USB and drives to N5 Air. Boot to BIOS (F2/Del): + +| Setting | Value | Purpose | +|---------|-------|---------| +| SVM Mode / AMD-V | Enabled | Virtualization | +| UMA Frame Buffer Size | 8GB-16GB | RAM allocation for Radeon 780M | +| IOMMU | Enabled | Device passthrough | + +### 3. N5 Air Integration + +1. Boot Unraid on N5 Air +2. Install "AMD GPU Top" from Apps +3. Update Ollama Docker: + ``` + Extra Parameters: --device /dev/dri + Environment: HSA_OVERRIDE_GFX_VERSION=11.0.0 + ``` +4. CPU Pinning: Settings → CPU Pinning → Assign 8-12 threads to Ollama + +--- + +## Phase 3: Network Topology for AI + +```markdown +## Network Map +- **Gateway/DNS:** 192.168.31.1 (MikroTik hAP ax³) +- **Unraid Server:** 192.168.31.2 (Docker host, AI stack) +- **AdGuard DNS:** 192.168.31.4 (macvlan on Unraid) +- **MikroTik AdGuard:** 172.17.0.2 (container, primary DNS) +- **MikroTik Tailscale:** 172.17.0.3 (container, VPN) + +## AI-Manageable Hosts +| Host | IP | SSH Port | Key | +|------|-----|----------|-----| +| Unraid | 192.168.31.2 | 422 | id_ed25519_unraid | +| MikroTik | 192.168.31.1 | 2222 | mikrotik_key | + +## Services +| Service | URL | +|---------|-----| +| Gitea | https://git.xtrm-lab.org | +| Woodpecker CI | https://ci.xtrm-lab.org | +| AdGuard (MikroTik) | http://192.168.31.1:3000 | +| AdGuard (Unraid) | http://192.168.31.4 | +| Ollama API | http://192.168.31.2:11434 | +| Open WebUI | http://192.168.31.2:3080 | + +## Operational Protocol +1. Use SSH keys for all remote commands +2. Verify container status before changes: `docker ps` or `/container print` +3. Never output raw passwords or credentials +4. Document all infrastructure changes in git repo +``` + +--- + +## Tasks + +- [ ] Install Intel GPU Top plugin on Unraid +- [ ] Deploy Ollama with `--device /dev/dri` +- [ ] Configure Open WebUI with Ollama endpoint +- [ ] Generate AI agent SSH key +- [ ] Deploy key to MikroTik for remote management +- [ ] Install Aider on workstation +- [ ] Create and upload topology.md to Open WebUI +- [ ] Test AI queries against infrastructure +- [ ] (Future) Migrate to N5 Air hardware + +## Notes + +- Current infrastructure repo: https://git.xtrm-lab.org/jazzymc/infrastructure +- MikroTik containers use bridge network 172.17.0.0/24 +- Unraid SSH on non-standard port 422