infrastructure/docs/wip/LOCAL-AI-STACK.md

WIP: Local AI Stack on Unraid

Status: Planning
Created: 2026-01-25

---

Overview

Deploy a hardware-accelerated local AI stack on Unraid (192.168.31.2), initially on N100 (Intel iGPU) with future migration path to N5 Air (AMD 780M iGPU).

---

Phase 1: Local AI Configuration (Current N100)

1. Hardware-Accelerated AI Stack

Install via Community Applications (Apps) tab:

App	Purpose	Configuration
Intel GPU Top	N100 QuickSync iGPU support	Required for GPU acceleration
Ollama	LLM runtime	Extra Parameters: --device /dev/dri
Open WebUI	Chat interface	OLLAMA_BASE_URL=http://192.168.31.2:11434

Ollama Docker Template:

Extra Parameters: --device /dev/dri
Port: 11434

Open WebUI Docker Template:

OLLAMA_BASE_URL=http://192.168.31.2:11434
Port: 3080 (or available port)

PWA Setup: Open WebUI on phone/tablet → "Add to Home Screen" for native experience.

2. SSH Bridge & Terminal Agent (Aider)

SSH Key Setup on Unraid:

# Create directory
mkdir -p /mnt/user/appdata/ssh_keys

# Generate AI agent key
ssh-keygen -t ed25519 -f /mnt/user/appdata/ssh_keys/ai_agent -N ""

# Deploy to MikroTik (via existing key)
# The AI can then manage MikroTik remotely

Aider Configuration:

export OLLAMA_API_BASE=http://192.168.31.2:11434
aider --model ollama_chat/qwen2.5-coder:14b

3. Sanitized Knowledge Base

Upload topology.md to Open WebUI → Workspace → Knowledge section.

---

Phase 2: Hardware Migration (N100 → N5 Air)

1. Clean Break (On N100)

1. Stop all Docker containers
2. Stop Array: Main tab → Stop
3. Disable Auto-Start: Settings → Disk Settings → Enable Auto-Start: No
4. Uninstall "Intel GPU Top" plugin
5. Shutdown N100

2. N5 Air BIOS Configuration

Move Unraid USB and drives to N5 Air. Boot to BIOS (F2/Del):

Setting	Value	Purpose
SVM Mode / AMD-V	Enabled	Virtualization
UMA Frame Buffer Size	8GB-16GB	RAM allocation for Radeon 780M
IOMMU	Enabled	Device passthrough

3. N5 Air Integration

1. Boot Unraid on N5 Air
2. Install "AMD GPU Top" from Apps
3. Update Ollama Docker:

   Extra Parameters: --device /dev/dri
   Environment: HSA_OVERRIDE_GFX_VERSION=11.0.0
   

4. CPU Pinning: Settings → CPU Pinning → Assign 8-12 threads to Ollama

---

Phase 3: Network Topology for AI

## Network Map
- **Gateway/DNS:** 192.168.31.1 (MikroTik hAP ax³)
- **Unraid Server:** 192.168.31.2 (Docker host, AI stack)
- **AdGuard DNS:** 192.168.31.4 (macvlan on Unraid)
- **MikroTik AdGuard:** 172.17.0.2 (container, primary DNS)
- **MikroTik Tailscale:** 172.17.0.3 (container, VPN)

## AI-Manageable Hosts
| Host | IP | SSH Port | Key |
|------|-----|----------|-----|
| Unraid | 192.168.31.2 | 422 | id_ed25519_unraid |
| MikroTik | 192.168.31.1 | 2222 | mikrotik_key |

## Services
| Service | URL |
|---------|-----|
| Gitea | https://git.xtrm-lab.org |
| Woodpecker CI | https://ci.xtrm-lab.org |
| AdGuard (MikroTik) | http://192.168.31.1:3000 |
| AdGuard (Unraid) | http://192.168.31.4 |
| Ollama API | http://192.168.31.2:11434 |
| Open WebUI | http://192.168.31.2:3080 |

## Operational Protocol
1. Use SSH keys for all remote commands
2. Verify container status before changes: `docker ps` or `/container print`
3. Never output raw passwords or credentials
4. Document all infrastructure changes in git repo

---

Tasks

• Install Intel GPU Top plugin on Unraid
• Deploy Ollama with --device /dev/dri
• Configure Open WebUI with Ollama endpoint
• Generate AI agent SSH key
• Deploy key to MikroTik for remote management
• Install Aider on workstation
• Create and upload topology.md to Open WebUI
• Test AI queries against infrastructure
• (Future) Migrate to N5 Air hardware

Notes

• Current infrastructure repo: https://git.xtrm-lab.org/jazzymc/infrastructure
• MikroTik containers use bridge network 172.17.0.0/24
• Unraid SSH on non-standard port 422