Add WIP: Local AI Stack planning document
All checks were successful
ci/woodpecker/push/woodpecker Pipeline was successful
All checks were successful
ci/woodpecker/push/woodpecker Pipeline was successful
- Phase 1: N100 Intel iGPU setup (Ollama, Open WebUI, Aider) - Phase 2: Future migration to N5 Air AMD 780M - Phase 3: Network topology for AI management - Customized for current infrastructure (192.168.31.x)
This commit is contained in:
150
docs/wip/LOCAL-AI-STACK.md
Normal file
150
docs/wip/LOCAL-AI-STACK.md
Normal file
@@ -0,0 +1,150 @@
|
||||
# WIP: Local AI Stack on Unraid
|
||||
|
||||
**Status:** Planning
|
||||
**Created:** 2026-01-25
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Deploy a hardware-accelerated local AI stack on Unraid (192.168.31.2), initially on N100 (Intel iGPU) with future migration path to N5 Air (AMD 780M iGPU).
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Local AI Configuration (Current N100)
|
||||
|
||||
### 1. Hardware-Accelerated AI Stack
|
||||
|
||||
Install via Community Applications (Apps) tab:
|
||||
|
||||
| App | Purpose | Configuration |
|
||||
|-----|---------|---------------|
|
||||
| Intel GPU Top | N100 QuickSync iGPU support | Required for GPU acceleration |
|
||||
| Ollama | LLM runtime | Extra Parameters: `--device /dev/dri` |
|
||||
| Open WebUI | Chat interface | `OLLAMA_BASE_URL=http://192.168.31.2:11434` |
|
||||
|
||||
**Ollama Docker Template:**
|
||||
```
|
||||
Extra Parameters: --device /dev/dri
|
||||
Port: 11434
|
||||
```
|
||||
|
||||
**Open WebUI Docker Template:**
|
||||
```
|
||||
OLLAMA_BASE_URL=http://192.168.31.2:11434
|
||||
Port: 3080 (or available port)
|
||||
```
|
||||
|
||||
**PWA Setup:** Open WebUI on phone/tablet → "Add to Home Screen" for native experience.
|
||||
|
||||
### 2. SSH Bridge & Terminal Agent (Aider)
|
||||
|
||||
**SSH Key Setup on Unraid:**
|
||||
```bash
|
||||
# Create directory
|
||||
mkdir -p /mnt/user/appdata/ssh_keys
|
||||
|
||||
# Generate AI agent key
|
||||
ssh-keygen -t ed25519 -f /mnt/user/appdata/ssh_keys/ai_agent -N ""
|
||||
|
||||
# Deploy to MikroTik (via existing key)
|
||||
# The AI can then manage MikroTik remotely
|
||||
```
|
||||
|
||||
**Aider Configuration:**
|
||||
```bash
|
||||
export OLLAMA_API_BASE=http://192.168.31.2:11434
|
||||
aider --model ollama_chat/qwen2.5-coder:14b
|
||||
```
|
||||
|
||||
### 3. Sanitized Knowledge Base
|
||||
|
||||
Upload `topology.md` to Open WebUI → Workspace → Knowledge section.
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Hardware Migration (N100 → N5 Air)
|
||||
|
||||
### 1. Clean Break (On N100)
|
||||
|
||||
1. Stop all Docker containers
|
||||
2. Stop Array: Main tab → Stop
|
||||
3. Disable Auto-Start: Settings → Disk Settings → Enable Auto-Start: No
|
||||
4. Uninstall "Intel GPU Top" plugin
|
||||
5. Shutdown N100
|
||||
|
||||
### 2. N5 Air BIOS Configuration
|
||||
|
||||
Move Unraid USB and drives to N5 Air. Boot to BIOS (F2/Del):
|
||||
|
||||
| Setting | Value | Purpose |
|
||||
|---------|-------|---------|
|
||||
| SVM Mode / AMD-V | Enabled | Virtualization |
|
||||
| UMA Frame Buffer Size | 8GB-16GB | RAM allocation for Radeon 780M |
|
||||
| IOMMU | Enabled | Device passthrough |
|
||||
|
||||
### 3. N5 Air Integration
|
||||
|
||||
1. Boot Unraid on N5 Air
|
||||
2. Install "AMD GPU Top" from Apps
|
||||
3. Update Ollama Docker:
|
||||
```
|
||||
Extra Parameters: --device /dev/dri
|
||||
Environment: HSA_OVERRIDE_GFX_VERSION=11.0.0
|
||||
```
|
||||
4. CPU Pinning: Settings → CPU Pinning → Assign 8-12 threads to Ollama
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: Network Topology for AI
|
||||
|
||||
```markdown
|
||||
## Network Map
|
||||
- **Gateway/DNS:** 192.168.31.1 (MikroTik hAP ax³)
|
||||
- **Unraid Server:** 192.168.31.2 (Docker host, AI stack)
|
||||
- **AdGuard DNS:** 192.168.31.4 (macvlan on Unraid)
|
||||
- **MikroTik AdGuard:** 172.17.0.2 (container, primary DNS)
|
||||
- **MikroTik Tailscale:** 172.17.0.3 (container, VPN)
|
||||
|
||||
## AI-Manageable Hosts
|
||||
| Host | IP | SSH Port | Key |
|
||||
|------|-----|----------|-----|
|
||||
| Unraid | 192.168.31.2 | 422 | id_ed25519_unraid |
|
||||
| MikroTik | 192.168.31.1 | 2222 | mikrotik_key |
|
||||
|
||||
## Services
|
||||
| Service | URL |
|
||||
|---------|-----|
|
||||
| Gitea | https://git.xtrm-lab.org |
|
||||
| Woodpecker CI | https://ci.xtrm-lab.org |
|
||||
| AdGuard (MikroTik) | http://192.168.31.1:3000 |
|
||||
| AdGuard (Unraid) | http://192.168.31.4 |
|
||||
| Ollama API | http://192.168.31.2:11434 |
|
||||
| Open WebUI | http://192.168.31.2:3080 |
|
||||
|
||||
## Operational Protocol
|
||||
1. Use SSH keys for all remote commands
|
||||
2. Verify container status before changes: `docker ps` or `/container print`
|
||||
3. Never output raw passwords or credentials
|
||||
4. Document all infrastructure changes in git repo
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Tasks
|
||||
|
||||
- [ ] Install Intel GPU Top plugin on Unraid
|
||||
- [ ] Deploy Ollama with `--device /dev/dri`
|
||||
- [ ] Configure Open WebUI with Ollama endpoint
|
||||
- [ ] Generate AI agent SSH key
|
||||
- [ ] Deploy key to MikroTik for remote management
|
||||
- [ ] Install Aider on workstation
|
||||
- [ ] Create and upload topology.md to Open WebUI
|
||||
- [ ] Test AI queries against infrastructure
|
||||
- [ ] (Future) Migrate to N5 Air hardware
|
||||
|
||||
## Notes
|
||||
|
||||
- Current infrastructure repo: https://git.xtrm-lab.org/jazzymc/infrastructure
|
||||
- MikroTik containers use bridge network 172.17.0.0/24
|
||||
- Unraid SSH on non-standard port 422
|
||||
Reference in New Issue
Block a user