diff --git a/docs/00-CURRENT-STATE.md b/docs/00-CURRENT-STATE.md new file mode 100644 index 0000000..5ba2c88 --- /dev/null +++ b/docs/00-CURRENT-STATE.md @@ -0,0 +1,265 @@ +# Infrastructure Upgrade Proposal: xtrm-lab.org (v2) + +## Current Infrastructure State + +**Document Updated:** 2026-01-18 +**Target Domain:** xtrm-lab.org + +--- + +## Network Topology + +### MikroTik hAP ax³ Router (192.168.31.1) + +| Parameter | Value | +|-----------|-------| +| RouterOS Version | 7.20.6 (stable) | +| WAN IP (Static) | 62.73.120.142 | +| LAN Subnet | 192.168.31.0/24 | +| Docker Bridge | 172.17.0.0/24 | +| SSH Access | `ssh -i /root/.ssh/mikrotik_key -p 2222 unraid@192.168.31.1` | + +**Interfaces:** +- `ether1` - WAN (62.73.120.142/23) +- `bridge` - LAN (192.168.31.1/24) +- `docker-bridge` - Container network (172.17.0.1/24) +- `back-to-home-vpn` - WireGuard VPN (192.168.216.1/24) + +**Running Containers on MikroTik:** +| Container | IP | Purpose | +|-----------|-----|---------| +| pihole:latest | 172.17.0.2 | DNS sinkhole (Pi-hole v6) | +| unbound:latest | 172.17.0.3 | Recursive DNS resolver | + +### Unraid Server (192.168.31.2) + +**Tailscale IP:** 100.100.208.70 + +**Key Services:** + +| Service | Container Name | Port(s) | Network | External URL | +|---------|---------------|---------|---------|--------------| +| Portainer | portainer | 9002→9000, 9444→9443 | bridge | http://100.100.208.70:9002 (Tailscale) | +| Pi-hole | binhex-official-pihole | 53, 80, 67 | br0 (192.168.31.4) | ph1.xtrm-lab.org | +| Unbound | unbound | 53 | br0 (192.168.31.5) | - | +| Traefik | traefik | 8001→80, 44301→443 | dockerproxy | traefik.xtrm-lab.org | +| Authentik | authentik | 9000, 9443 | dockerproxy | auth.xtrm-lab.org | +| Authentik Worker | authentik-worker | - | authentik | - | +| Vaultwarden | vaultwarden | 4743→80 | bridge | vault.xtrm-lab.org | +| Plex | plex | 32400 | host | plex.xtrm-lab.org | +| Home Assistant | HomeAssistant_inabox | 8123 | host (192.168.31.15) | ha.xtrm-lab.org | +| Transmission | transmission | 9091, 51413 | bridge | - | +| Nextcloud | Nextcloud | 8666→80 | bridge | - | +| PostgreSQL | postgresql17 | 5432 | bridge | - | +| Redis | Redis | 6379 | bridge | - | +| Uptime Kuma | UptimeKuma | 3001 | bridge | - | +| NetAlertX | NetAlertX | 20211 | host | netalert.xtrm-lab.org | +| UrBackup | UrBackup | 55414 | host | urbackup.xtrm-lab.org | +| Homarr | homarr | 10004→7575 | bridge | - | +| Nebula Sync | nebula-sync | - | - | Pi-hole sync | +| DoH Server | DoH-Server | 8053 | dockerproxy | doh.xtrm-lab.org | +| stunnel DoT | stunnel-dot | 853 | bridge | dns.xtrm-lab.org:853 | +| Pangolin | pangolin | 3003→3001, 3004→3002 | bridge | Fossorial controller | +| Gitea | gitea | 3005→3000, 2222→22 | dockerproxy | git.xtrm-lab.org | +| Woodpecker Server | woodpecker-server | 8008→8000 | dockerproxy | ci.xtrm-lab.org | +| Woodpecker Agent | woodpecker-agent | - | dockerproxy | - | +| RustDesk ID | rustdesk-hbbs | 21115-21116, 21118-21119 | bridge | rustdesk.xtrm-lab.org | +| RustDesk Relay | rustdesk-hbbr | 21117 | bridge | rustdesk.xtrm-lab.org | + +--- + +## Current NAT/Port Forwarding (MikroTik) + +| Rule | Protocol | WAN Port | Destination | Purpose | +|------|----------|----------|-------------|---------| +| Forward HTTP | TCP | 80 | 192.168.31.2:8001 | Traefik HTTP | +| Forward HTTPS | TCP | 443 | 192.168.31.2:44301 | Traefik HTTPS | +| Plex | TCP | 32400 | 192.168.31.2:32400 | Plex Media Server | +| Transmission | TCP/UDP | 51413 | 192.168.31.2:51413 | BitTorrent | +| DoT | TCP | 853 | 172.17.0.2:853 | DNS over TLS | +| DoH | TCP/UDP | 5443 | 172.17.0.2:443 | DNS over HTTPS | +| DNS Force | UDP/TCP | 53 | 172.17.0.2:53 | Force LAN DNS to Pi-hole | +| RustDesk NAT Test | TCP | 21115 | 192.168.31.2:21115 | RustDesk NAT Test | +| RustDesk ID TCP | TCP | 21116 | 192.168.31.2:21116 | RustDesk ID Server | +| RustDesk ID UDP | UDP | 21116 | 192.168.31.2:21116 | RustDesk ID Server | +| RustDesk Relay | TCP | 21117 | 192.168.31.2:21117 | RustDesk Relay | + +--- + +## Current WireGuard Configuration + +**Interface:** `back-to-home-vpn` +- Listen Port: 59188 +- Address: 192.168.216.1/24 +- Public Key: `3e+p++SJ6f5EURt6WCKApOLMQHWpURm/vn/0s9+EKzs=` + +**Existing Peers:** +1. hAP ax³ (secondary device) +2. Kaloyan's S25 Ultra (mobile) +3. Additional peer (unnamed) + +--- + +## Traefik Configuration + +**Entry Points:** +- HTTP (:80) → Redirects to HTTPS +- HTTPS (:443) + +**Certificate Resolver:** Cloudflare DNS Challenge +- Email: admin@xtrm-lab.org +- DNS Provider: Cloudflare + +**Existing Middlewares:** +- `default-headers` - Security headers (HSTS, XSS protection, etc.) +- `authentik-forward-auth` - Forward auth to Authentik (configured but not applied) +- `pihole1-redirect` / `pihole2-redirect` - Redirect root to /admin/ + +--- + +## Authentik Configuration + +| Parameter | Value | +|-----------|-------| +| Version | 2025.8.1 | +| URL | auth.xtrm-lab.org | +| PostgreSQL Host | postgresql17 | +| Database | authentik_db | +| Redis Host | redis | +| Network | dockerproxy | + +**Status:** Deployed but not yet integrated with services + +--- + +## Portainer Configuration (Phase 6) + +| Parameter | Value | +|-----------|-------| +| Version | CE Latest | +| HTTP Port | 9002 | +| HTTPS Port | 9444 | +| Data Path | /mnt/user/appdata/portainer | +| Tailscale URL | http://100.100.208.70:9002 | +| Local URL | http://192.168.31.2:9002 | + +**Status:** Deployed, awaiting initial setup and MikroTik connection (Phase 6.2/6.3) + +--- + +## DNS Architecture + +``` + ┌─────────────────────────────────────┐ + │ Internet │ + └───────────────┬─────────────────────┘ + │ + ┌───────────────▼─────────────────────┐ + │ MikroTik hAP ax³ (192.168.31.1) │ + │ WAN: 62.73.120.142 │ + └───────────────┬─────────────────────┘ + │ + ┌────────────────────────┼────────────────────────┐ + │ │ │ + ▼ ▼ ▼ +┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ +│ Pi-hole (Router) │ │ Unraid Server │ │ LAN Devices │ +│ 172.17.0.2 │ │ 192.168.31.2 │ │ 192.168.31.x │ +│ Primary DNS │ │ │ │ │ +└────────┬─────────┘ └────────┬─────────┘ └──────────────────┘ + │ │ + ▼ ▼ +┌──────────────────┐ ┌──────────────────┐ +│ Unbound (Router) │ │ Unbound (Unraid) │ +│ 172.17.0.3 │ │ 192.168.31.5 │ +│ Recursive DNS │ │ Recursive DNS │ +└──────────────────┘ └──────────────────┘ + │ + ▼ + ┌──────────────────┐ + │ Pi-hole (Unraid) │ + │ 192.168.31.4 │ + │ Secondary DNS │ + └──────────────────┘ +``` + +--- + +## Service Interruption Risk Assessment + +| Phase | Component | Interruption Risk | Mitigation | +|-------|-----------|-------------------|------------| +| 1 | Tailscale Integration | LOW | Add-on service, no changes to existing | +| 1 | DoH Endpoint | LOW | New endpoint, existing DNS unaffected | +| 2 | Pangolin/Gerbil | MEDIUM | New containers, may conflict with WG port 51820 | +| 2 | Newt Connector | LOW | Outbound only | +| 3 | Authentik Forward Auth | HIGH | Will gate all services - test thoroughly | +| 4 | Sunshine/Moonlight | LOW | New service, Tailscale-only access | +| 5 | RustDesk | MEDIUM | New ports required on MikroTik | +| 6 | Portainer | LOW | Management tool only, no service impact | + +--- + +## Ports Required for Full Implementation + +### New MikroTik Port Forwards Needed: + +| Service | Protocol | Port(s) | Destination | Phase | +|---------|----------|---------|-------------|-------| +| WireGuard (Fossorial) | UDP | 51820 | 192.168.31.2:51820 | 2 | +| RustDesk ID TCP | TCP | 21115-21117 | 192.168.31.2:21115-21117 | 5 | +| RustDesk Relay | TCP | 21118-21119 | 192.168.31.2:21118-21119 | 5 | +| RustDesk NAT | UDP | 21116 | 192.168.31.2:21116 | 5 | + +--- + +## Next Steps + +Proceed to individual phase documents: +1. [Phase 1: Global DNS Portability](./01-PHASE1-DNS-PORTABILITY.md) +2. [Phase 2: Fossorial Tunnel Stack](./02-PHASE2-FOSSORIAL-STACK.md) +3. [Phase 3: Identity & Zero Trust](./03-PHASE3-AUTHENTIK-ZEROTRUST.md) +4. [Phase 4: Remote Gaming](./04-PHASE4-REMOTE-GAMING.md) +5. [Phase 5: RustDesk Setup](./05-PHASE5-RUSTDESK.md) +6. [Phase 6: Portainer Management](./06-PHASE6-PORTAINER-MANAGEMENT.md) + +--- + +## Completed Infrastructure Tasks + +### Static IP Assignment for Critical Services + +**Status:** COMPLETED (2026-01-18) +**Priority:** High +**Reason:** Critical services should have static IPs outside DHCP/dynamic lease range to prevent IP conflicts and ensure reliable inter-container communication. + +#### dockerproxy Network (172.18.0.0/16) +Static IP range: 172.18.0.2 - 172.18.0.50 + +| Service | Static IP | +|---------|-----------| +| dockersocket | 172.18.0.2 | +| traefik | 172.18.0.3 | +| authentik | 172.18.0.11 | +| authentik-worker | 172.18.0.12 | +| postgresql17 | 172.18.0.13 | +| Redis | 172.18.0.14 | +| vaultwarden | 172.18.0.15 | + +#### bridge Network (172.17.0.0/16) +Static IP range: 172.17.0.2 - 172.17.0.50 + +| Service | Static IP | +|---------|-----------| +| portainer | 172.17.0.2 | +| rustdesk-hbbs | 172.17.0.3 | +| rustdesk-hbbr | 172.17.0.4 | + +#### Implementation Steps +1. [x] Update Docker network IPAM config to reserve static range +2. [x] Recreate critical containers with --ip flag or docker-compose static IP +3. [x] Update any hardcoded references to old IPs +4. [x] Test inter-container connectivity +5. [x] Document final IP assignments + +**Note:** IPs assigned via `docker network connect --ip`. To persist across container recreation, update Unraid Docker templates or use docker-compose. diff --git a/docs/01-PHASE1-DNS-PORTABILITY.md b/docs/01-PHASE1-DNS-PORTABILITY.md new file mode 100644 index 0000000..75ee104 --- /dev/null +++ b/docs/01-PHASE1-DNS-PORTABILITY.md @@ -0,0 +1,208 @@ +# Phase 1: Global DNS Portability + +## Status: ✅ COMPLETED (2026-01-18) + +Pi-hole ad-blocking works on all devices via Tailscale MagicDNS and DoH/DoT endpoints. + +--- + +## Tailscale Configuration + +| Parameter | Value | +|-----------|-------| +| Unraid Tailscale IP | 100.100.208.70 | +| Hostname | xtrm-unraid | +| Subnet Route | 192.168.31.0/24 (advertised & approved) | +| Global DNS | Pi-hole via MagicDNS | +| Override Local DNS | Enabled | + +### Connected Devices +| Device | Tailscale IP | Status | +|--------|--------------|--------| +| xtrm-unraid | 100.100.208.70 | Online | +| kaloyans-macbook-air | 100.68.118.59 | Active | +| mikrotik-tailscale-1 | 100.75.93.123 | Online | +| samsung-sm-s938b | 100.111.64.56 | Offline | + +--- + +## DNS Services + +### Pi-hole Instances + +| Instance | Location | IP | Web UI | Status | +|----------|----------|-----|--------|--------| +| Primary | MikroTik Container | 172.17.0.2 | ph2.xtrm-lab.org | ✅ Running | +| Secondary | Unraid (macvlan br0) | 192.168.31.4 | ph1.xtrm-lab.org | ✅ Running | + +**Sync:** nebula-sync (healthy) syncs Unraid → MikroTik every 5 minutes + +### Unbound Instances (Recursive DNS) + +| Instance | Location | IP | Status | +|----------|----------|-----|--------| +| Primary | MikroTik Container | 172.17.0.3 | ✅ Running | +| Secondary | Unraid (macvlan br0) | 192.168.31.5 | ✅ Running | + +### Pi-hole Upstream Configuration + +**Unraid Pi-hole (192.168.31.4):** +``` +upstreams = ["172.17.0.3#53", "192.168.31.5#53"] +``` + +--- + +## DoH Endpoint (DNS over HTTPS) + +| Parameter | Value | +|-----------|-------| +| URL | `https://doh.xtrm-lab.org/dns-query` | +| Container | DoH-Server (ghcr.io/ich777/doh-server) | +| Listen Port | 8053 | +| Upstream DNS | udp:192.168.31.1:53 | +| Network | dockerproxy | + +**Traefik Route (dynamic.yml):** +```yaml +doh-secure: + rule: "Host(`doh.xtrm-lab.org`)" + entryPoints: [https] + tls: + certResolver: cloudflare + service: doh # → http://DoH-Server:8053 +``` + +**Test:** +```bash +curl -H 'accept: application/dns-json' 'https://doh.xtrm-lab.org/dns-query?name=google.com&type=A' +``` + +--- + +## DoT Endpoint (DNS over TLS) + +| Parameter | Value | +|-----------|-------| +| Hostname | doh.xtrm-lab.org:853 | +| Container | stunnel-dot (dweomer/stunnel) | +| Accept Port | 853 | +| Forward To | 192.168.31.4:53 (Unraid Pi-hole) | + +**MikroTik NAT:** WAN:853 → 192.168.31.2:853 + +**Android Private DNS:** Settings → Private DNS → `doh.xtrm-lab.org` + +--- + +## DNS Architecture + +``` + ┌─────────────────────────────────────┐ + │ External Clients │ + └──────────────┬──────────────────────┘ + │ + ┌─────────────────────────┼─────────────────────────┐ + │ │ │ + ▼ ▼ ▼ + ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ + │ Tailscale │ │ DoH │ │ DoT │ + │ MagicDNS │ │ doh.xtrm-lab.org│ │ :853 │ + │ 100.100.100.100 │ │ (Traefik→DoH) │ │ (stunnel) │ + └────────┬────────┘ └────────┬────────┘ └────────┬────────┘ + │ │ │ + └────────────────────────┼────────────────────────┘ + │ + ▼ + ┌─────────────────────────────────────────────────────────────────────┐ + │ LAN (192.168.31.0/24) │ + │ │ + │ ┌─────────────────────────┐ ┌─────────────────────────┐ │ + │ │ MikroTik Pi-hole │ │ Unraid Pi-hole │ │ + │ │ 172.17.0.2 │ │ 192.168.31.4 │ │ + │ │ (NAT forced for LAN) │ │ (Direct access allowed) │ │ + │ └───────────┬─────────────┘ └───────────┬─────────────┘ │ + │ │ │ │ + │ ▼ ▼ │ + │ ┌─────────────────────────┐ ┌─────────────────────────┐ │ + │ │ MikroTik Unbound │ │ Unraid Unbound │ │ + │ │ 172.17.0.3 (recursive) │ │ 192.168.31.5 (recursive)│ │ + │ └─────────────────────────┘ └─────────────────────────┘ │ + └─────────────────────────────────────────────────────────────────────┘ +``` + +--- + +## DHCP DNS Servers (MikroTik) + +| Priority | Server | Notes | +|----------|--------|-------| +| Primary | 192.168.31.1 | MikroTik (NAT forces to Pi-hole 172.17.0.2) | +| Secondary | 192.168.31.4 | Unraid Pi-hole (direct, for failover) | + +--- + +## MikroTik DNS NAT Rules + +| Rule | Action | Description | +|------|--------|-------------| +| 3 | ACCEPT | Traffic TO 192.168.31.4:53 (allows Unraid Pi-hole) | +| 7 | DST-NAT | Force LAN DNS to 172.17.0.2 (MikroTik Pi-hole) | +| 21 | DST-NAT | DoT WAN:853 → 192.168.31.2:853 | + +--- + +## Failover Behavior + +| Scenario | Behavior | +|----------|----------| +| MikroTik Pi-hole down | Clients use secondary DNS (192.168.31.4) | +| MikroTik Unbound down | Pi-holes use Unraid Unbound (192.168.31.5) | +| Unraid down | MikroTik services continue independently | + +--- + +## Client Configuration + +### macOS (DoH) +- Firefox/Chrome: Settings → Security → Custom DNS → `https://doh.xtrm-lab.org/dns-query` +- System-wide: Install DNS profile with DoH URL + +### Android (DoT) +- Settings → Network → Private DNS → `doh.xtrm-lab.org` + +### Tailscale Clients +- Automatic via MagicDNS (no configuration needed) + +--- + +## Verification Commands + +```bash +# Test DoH endpoint +curl -H 'accept: application/dns-json' 'https://doh.xtrm-lab.org/dns-query?name=google.com&type=A' + +# Test ad-blocking via DoH +curl -H 'accept: application/dns-json' 'https://doh.xtrm-lab.org/dns-query?name=ads.google.com&type=A' +# Expected: 0.0.0.0 + +# Test Pi-holes directly +dig +short google.com @172.17.0.2 # MikroTik Pi-hole +dig +short google.com @192.168.31.4 # Unraid Pi-hole + +# Test Unbound directly +dig +short google.com @172.17.0.3 # MikroTik Unbound +dig +short google.com @192.168.31.5 # Unraid Unbound + +# Check Tailscale ad-blocking +dig +short ads.google.com @100.100.100.100 +# Expected: 0.0.0.0 +``` + +--- + +## Known Issues + +| Issue | Status | Notes | +|-------|--------|-------| +| Certificate renewal failing | ⚠️ Open | Cloudflare API token needs Zone:DNS:Edit permission. Certs expire Feb 11, 2026. | diff --git a/docs/02-PHASE2-FOSSORIAL-STACK.md b/docs/02-PHASE2-FOSSORIAL-STACK.md new file mode 100644 index 0000000..6da91c2 --- /dev/null +++ b/docs/02-PHASE2-FOSSORIAL-STACK.md @@ -0,0 +1,391 @@ +# Phase 2: The "Fossorial" Tunnel Stack (Pangolin, Gerbil, Newt) + +## Goal +Deploy the Fossorial tunnel stack using your MikroTik's static IP (62.73.120.142) to host tunnels without requiring a VPS, with MikroTik container fallback for resilience. + +--- + +## What is Fossorial? + +Fossorial is a self-hosted tunnel solution consisting of: + +| Component | Purpose | Role | +|-----------|---------|------| +| **Pangolin** | Central controller/dashboard | Manages tunnels, provides web UI | +| **Gerbil** | WireGuard manager | Handles WireGuard peer configuration | +| **Newt** | Tunnel connector | Lightweight agent that "dials out" to establish tunnels | + +**Why Fossorial over plain WireGuard?** +- Automatic peer management +- Web-based tunnel configuration +- Self-healing connections via Newt +- Easier certificate/identity management + +--- + +## Current WireGuard State (MikroTik) + +``` +Interface: back-to-home-vpn +├── Listen Port: 59188 (non-standard - good!) +├── Address: 192.168.216.1/24 +├── Public Key: 3e+p++SJ6f5EURt6WCKApOLMQHWpURm/vn/0s9+EKzs= +└── Peers: 3 configured +``` + +**Port 51820 Status:** NOT in use - available for Fossorial + +--- + +## Architecture Overview + +``` + Internet + │ + ┌────────────▼────────────┐ + │ MikroTik (62.73.120.142)│ + │ Port Forward: │ + │ UDP 51820 → Unraid │ + │ TCP 443 → Traefik │ + └────────────┬────────────┘ + │ + ┌──────────────────┼──────────────────┐ + │ │ │ + ▼ ▼ ▼ + ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ + │ Pangolin │ │ Gerbil │ │ Newt │ + │ (Controller) │ │ (WG Manager) │ │ (Connector) │ + │ :3000 web UI │ │ :51820 WG │ │ Outbound only │ + │ │ │ :8080 API │ │ │ + └────────┬────────┘ └────────┬────────┘ └────────┬────────┘ + │ │ │ + └───────────────────┴───────────────────┘ + Internal Network + 192.168.31.0/24 +``` + +--- + +## Implementation Steps + +### Step 2.1: Create Docker Network for Fossorial + +```bash +docker network create --driver bridge fossorial +``` + +--- + +### Step 2.2: Deploy Pangolin (Controller) + +**Unraid Docker Template:** + +```xml + + + pangolin + fossoriumtech/pangolin:latest + https://hub.docker.com/r/fossoriumtech/pangolin + fossorial + sh + false + Pangolin - Fossorial tunnel controller and dashboard + Network:VPN + https://pangolin.xtrm-lab.org + --restart unless-stopped + + + 3000 + + + /mnt/user/appdata/pangolin/data + /mnt/user/appdata/pangolin/config + + + https://pangolin.xtrm-lab.org + GENERATE_A_SECURE_32_CHAR_KEY + + + true + Host(`pangolin.xtrm-lab.org`) + https + cloudflare + default-headers@file + 3000 + dockerproxy + + + https://pangolin.xtrm-lab.org + Pangolin Dashboard + http + + + /app/data + +``` + +**Generate Secret Key:** +```bash +openssl rand -hex 32 +``` + +--- + +### Step 2.3: Deploy Gerbil (WireGuard Manager) + +**Unraid Docker Template:** + +```xml + + + gerbil + fossoriumtech/gerbil:latest + https://hub.docker.com/r/fossoriumtech/gerbil + fossorial + sh + true + Gerbil - Fossorial WireGuard manager + Network:VPN + --cap-add=NET_ADMIN --cap-add=SYS_MODULE --sysctl net.ipv4.ip_forward=1 --sysctl net.ipv4.conf.all.src_valid_mark=1 + + + 51820 + 8080 + + + /mnt/user/appdata/gerbil/wireguard + /mnt/user/appdata/gerbil/data + + + http://pangolin:3000 + 62.73.120.142 + 51820 + wg0 + SAME_AS_PANGOLIN_SECRET + + + http://192.168.31.2:8080/health + Gerbil WireGuard + http + +``` + +--- + +### Step 2.4: Deploy Newt (Connector) + +**Unraid Docker Template:** + +```xml + + + newt + fossoriumtech/newt:latest + https://hub.docker.com/r/fossoriumtech/newt + fossorial + sh + false + Newt - Fossorial tunnel connector (dials out to establish tunnels) + Network:VPN + --restart unless-stopped + + + /mnt/user/appdata/newt/data + + + https://pangolin.xtrm-lab.org + 62.73.120.142:51820 + GENERATE_VIA_PANGOLIN_UI + unraid-local + + + docker + Newt Connector + docker + +``` + +--- + +### Step 2.5: MikroTik Port Forward for WireGuard + +**Add NAT rule for Fossorial WireGuard:** + +```routeros +# Connect via SSH +ssh -i /root/.ssh/mikrotik_key -p 2222 unraid@192.168.31.1 + +# Add port forward +/ip/firewall/nat add chain=dstnat \ + action=dst-nat \ + to-addresses=192.168.31.2 \ + to-ports=51820 \ + protocol=udp \ + dst-address=62.73.120.142 \ + dst-port=51820 \ + comment="Fossorial WireGuard" + +# Add firewall rule to allow +/ip/firewall/filter add chain=forward \ + action=accept \ + protocol=udp \ + dst-address=192.168.31.2 \ + dst-port=51820 \ + comment="Allow Fossorial WireGuard" \ + place-before=14 +``` + +**Verification:** +```routeros +/ip/firewall/nat print where comment~"Fossorial" +``` + +--- + +### Step 2.6: Connect Networks (fossorial ↔ dockerproxy) + +Pangolin needs to be accessible via Traefik. Either: + +**Option A: Connect Pangolin to both networks** +```bash +docker network connect dockerproxy pangolin +``` + +**Option B: Use Traefik external routing in dynamic.yml** +```yaml +# Add to /mnt/user/appdata/traefik/dynamic.yml +http: + routers: + pangolin-secure: + rule: "Host(`pangolin.xtrm-lab.org`)" + entryPoints: + - https + middlewares: + - default-headers + tls: + certResolver: cloudflare + service: pangolin + + services: + pangolin: + loadBalancer: + servers: + - url: "http://192.168.31.2:3000" +``` + +--- + +### Step 2.7: MikroTik Container Fallback (Optional) + +Deploy a lightweight Gerbil instance on MikroTik for resilience: + +**Prerequisites:** +- USB storage connected to MikroTik (already present: `usb1`) +- Container mode enabled + +**MikroTik Commands:** + +```routeros +# Create container for Gerbil fallback +/container/config set registry-url=https://registry-1.docker.io tmpdir=usb1/tmp + +# Pull gerbil image +/container add \ + remote-image=fossoriumtech/gerbil:latest \ + interface=docker-bridge \ + root-dir=usb1/gerbil \ + start-on-boot=yes \ + comment="Fossorial Gerbil Fallback" + +# Configure environment +/container/envs add name=gerbil-env key=GERBIL_PUBLIC_IP value="62.73.120.142" +/container/envs add name=gerbil-env key=GERBIL_PUBLIC_PORT value="51821" +/container/envs add name=gerbil-env key=GERBIL_MODE value="standalone" +``` + +**Note:** MikroTik containers have limited resources. This is a fallback for critical services only (Pi-hole access, Authentik). + +--- + +## Service Interruption Assessment + +| Action | Risk | Impact | Mitigation | +|--------|------|--------|------------| +| Deploy Pangolin/Gerbil/Newt | NONE | New containers | - | +| Port forward 51820 | LOW | New port, existing WG on 59188 unaffected | - | +| Connect fossorial network | LOW | Container networking | Test connectivity | +| MikroTik container | MEDIUM | Router resources | Monitor CPU/memory | + +**Existing WireGuard (back-to-home-vpn) Impact:** NONE +- Uses port 59188, not 51820 +- Completely separate interface + +--- + +## Verification Checklist + +- [ ] All three containers running: `docker ps | grep -E "pangolin|gerbil|newt"` +- [ ] Pangolin web UI accessible: https://pangolin.xtrm-lab.org +- [ ] Gerbil API responding: `curl http://192.168.31.2:8080/health` +- [ ] MikroTik NAT rule in place: `ssh ... "/ip/firewall/nat print"` +- [ ] External WireGuard test: Connect from external network to 62.73.120.142:51820 +- [ ] Newt connected in Pangolin dashboard + +--- + +## Initial Pangolin Setup + +1. Navigate to https://pangolin.xtrm-lab.org +2. Create admin account +3. Add Gerbil node: + - Name: `unraid-gerbil` + - API URL: `http://gerbil:8080` + - API Key: (same as GERBIL_API_KEY) +4. Create a tunnel: + - Name: `home-services` + - Assign to Gerbil node +5. Generate Newt API key in Pangolin UI +6. Update Newt container with the API key + +--- + +## Rollback Procedure + +1. **Stop containers:** + ```bash + docker stop newt gerbil pangolin + docker rm newt gerbil pangolin + ``` + +2. **Remove MikroTik NAT:** + ```routeros + /ip/firewall/nat remove [find comment="Fossorial WireGuard"] + ``` + +3. **Remove network:** + ```bash + docker network rm fossorial + ``` + +4. **Clean up data (if desired):** + ```bash + rm -rf /mnt/user/appdata/pangolin /mnt/user/appdata/gerbil /mnt/user/appdata/newt + ``` + +--- + +## Files Modified + +| File/System | Change | Backup Required | +|-------------|--------|-----------------| +| MikroTik NAT | Add UDP 51820 forward | N/A (can remove) | +| /mnt/user/appdata/traefik/dynamic.yml | Add pangolin route | YES | +| New directories created | /mnt/user/appdata/pangolin,gerbil,newt | N/A | + +--- + +## Dependencies for Next Phase + +Phase 3 (Authentik) can now use Fossorial tunnels to: +- Expose Authentik externally without Cloudflare dependency +- Create secure tunnels for mobile OIDC authentication diff --git a/docs/03-PHASE3-AUTHENTIK-ZEROTRUST.md b/docs/03-PHASE3-AUTHENTIK-ZEROTRUST.md new file mode 100644 index 0000000..69da74c --- /dev/null +++ b/docs/03-PHASE3-AUTHENTIK-ZEROTRUST.md @@ -0,0 +1,196 @@ +# Phase 3: Identity & Zero Trust (Authentik) + +## Status: ✅ COMPLETED + +**Last Verified:** 2026-01-18 + +--- + +## Goal +Gate every `*.xtrm-lab.org` service behind OIDC authentication using Authentik, implementing Zero Trust access control. + +--- + +## Current Authentik State + +| Parameter | Value | +|-----------|-------| +| Container | authentik (+ authentik-worker) | +| Version | 2025.8.1 | +| Network | dockerproxy | +| Ports | 9000 (HTTP), 9443 (HTTPS) | +| URL | https://auth.xtrm-lab.org | +| PostgreSQL | postgresql17 (authentik_db) | +| Redis | redis | +| Status | ✅ Running (healthy) | + +--- + +## Verified Configuration + +### Users + +| Username | Name | Status | +|----------|------|--------| +| akadmin | authentik Default Admin | Active | +| admin | Admin User | Active | +| jazzymc | Kaloyan Danchev | Active | + +### Groups + +| Group Name | Purpose | +|------------|---------| +| authentik Admins | Administrative access | +| authentik Read-only | Read-only access | + +### Outpost + +| Name | Type | Status | +|------|------|--------| +| authentik Embedded Outpost | proxy | ✅ Running | + +### Applications + +| Application | Slug | +|-------------|------| +| XTRM-Lab Protected Services | xtrm-lab-protected | +| Actual Budget | actual-budget | + +### Proxy Provider + +| External Host | Mode | +|---------------|------| +| https://auth.xtrm-lab.org | forward_domain | + +### 2FA Status + +| Type | Count | +|------|-------| +| TOTP Devices | 2 | +| WebAuthn Devices | 0 | + +--- + +## Services Protected by Authentik Forward Auth + +The following services require Authentik authentication: + +| Service | Domain | +|---------|--------| +| n8n | n8n.xtrm-lab.org | +| Traefik Dashboard | traefik.xtrm-lab.org | +| NetAlertX | netalert.xtrm-lab.org | +| UrBackup | urbackup.xtrm-lab.org | +| Pi-hole 1 | ph1.xtrm-lab.org | +| Pi-hole 2 | ph2.xtrm-lab.org | +| Unimus | unimus.xtrm-lab.org | +| Homarr | xtrm-lab.org | +| Uptime Kuma | uptime.xtrm-lab.org | +| Transmission | transmission.xtrm-lab.org | + +**Total: 12 protected routes** (including root redirects for Pi-holes) + +--- + +## Services WITHOUT Authentik Protection + +These services have their own authentication or are public: + +| Service | Domain | Reason | +|---------|--------|--------| +| Authentik | auth.xtrm-lab.org | Self (would cause redirect loop) | +| Plex | plex.xtrm-lab.org | Has own Plex authentication | +| Vaultwarden | vault.xtrm-lab.org | Has own authentication | +| Home Assistant | ha.xtrm-lab.org | Has own authentication | +| Karakeep | karakeep.xtrm-lab.org | Public/own auth | +| RustFS CDN | cdn.xtrm-lab.org | Public CDN (S3 auth) | +| Pangolin API | pangolin.xtrm-lab.org | API access | +| Nextcloud | nextcloud.xtrm-lab.org | Has own authentication | + +--- + +## Traefik Forward Auth Middleware + +Configured in `/mnt/user/appdata/traefik/dynamic.yml`: + +```yaml +authentik-forward-auth: + forwardAuth: + address: "http://authentik:9000/outpost.goauthentik.io/auth/traefik" + trustForwardHeader: true + authResponseHeaders: + - X-authentik-username + - X-authentik-groups + - X-authentik-email + - X-authentik-name + - X-authentik-uid +``` + +--- + +## Verification Checklist + +- [x] Authentik initial setup completed (admin password set) +- [x] Outpost running and connected (embedded outpost) +- [x] User groups created (authentik Admins, authentik Read-only) +- [x] Application/provider pairs configured (2 applications) +- [x] Traefik config updated with forward auth middleware +- [x] Services tested successfully (302 redirect to login) +- [x] All planned services protected (12 routes) +- [x] 2FA enabled for admin accounts (2 TOTP devices) + +--- + +## Architecture + +``` + Internet User + │ + ┌────────────▼────────────┐ + │ Traefik (Reverse Proxy)│ + │ *.xtrm-lab.org:443 │ + └────────────┬────────────┘ + │ + ┌────────────▼────────────┐ + │ Forward Auth Check │ + │ → Authentik Outpost │ + └────────────┬────────────┘ + │ + ┌──────────────────┴──────────────────┐ + │ │ + ┌─────────▼─────────┐ ┌─────────▼─────────┐ + │ Authenticated? │ │ Login Required │ + │ YES → Pass │ │ Redirect to │ + │ through to │ │ auth.xtrm-lab.org│ + │ backend service │ └───────────────────┘ + └───────────────────┘ +``` + +--- + +## Maintenance Notes + +### Database +- PostgreSQL database: `authentik_db` +- User: `authentik_user` +- Host: `postgresql17` container +- Data path: `/mnt/user/appdata/postgresql` + +### Backup Recommendation +Regularly backup: +- PostgreSQL database (contains all Authentik config) +- `/mnt/user/appdata/traefik/dynamic.yml` + +### Rollback Procedure + +**Remove all protection (emergency):** +1. Edit `/mnt/user/appdata/traefik/dynamic.yml` +2. Remove `authentik-forward-auth` from all router middlewares +3. Traefik will auto-reload + +--- + +## Related Documents + +- [00-CURRENT-STATE.md](./00-CURRENT-STATE.md) - Infrastructure overview +- [02-PHASE2-FOSSORIAL-STACK.md](./02-PHASE2-FOSSORIAL-STACK.md) - Pangolin integration diff --git a/docs/04-PHASE4-REMOTE-GAMING.md b/docs/04-PHASE4-REMOTE-GAMING.md new file mode 100644 index 0000000..69780f5 --- /dev/null +++ b/docs/04-PHASE4-REMOTE-GAMING.md @@ -0,0 +1,465 @@ +# Phase 4: Remote Gaming (Sunshine + Moonlight) + +## Goal +Enable low-latency 60FPS game streaming from Nobara Linux (AMD GPU) to MacBook and Android devices, using Tailscale for optimal network pathing. + +--- + +## Prerequisites + +- **Gaming PC:** Nobara Linux with AMD GPU (VA-API support) +- **Tailscale:** Installed on gaming PC (from Phase 1) +- **Clients:** MacBook and Android devices with Tailscale + +--- + +## Architecture Overview + +``` + ┌─────────────────────────────────┐ + │ Tailscale Mesh Network │ + │ (Encrypted, P2P when possible) │ + └─────────────────┬───────────────┘ + │ + ┌────────────────────────────┼────────────────────────────┐ + │ │ │ + ▼ ▼ ▼ +┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ +│ Nobara Gaming PC│ │ MacBook │ │ Android Device │ +│ Sunshine Host │ │ Moonlight Client│ │ Moonlight Client│ +│ 100.x.x.x (TS) │ │ 100.x.x.x (TS) │ │ 100.x.x.x (TS) │ +│ AMD VA-API │ │ │ │ │ +└─────────────────┘ └─────────────────┘ └─────────────────┘ +``` + +**Why Tailscale for Gaming?** +- Automatic P2P connection when on same LAN +- Encrypted tunnel when remote +- No manual port forwarding required +- MagicDNS for easy hostname resolution +- NAT traversal handled automatically + +--- + +## Implementation Steps + +### Step 4.1: Install Sunshine on Nobara + +**Method A: Flatpak (Recommended for Nobara)** +```bash +# Install via Flatpak +flatpak install flathub dev.lizardbyte.app.Sunshine + +# Enable autostart +flatpak run --command=sunshine dev.lizardbyte.app.Sunshine & +``` + +**Method B: Native Package** +```bash +# Add Sunshine repository +sudo dnf copr enable lizardbyte/stable +sudo dnf install sunshine + +# Enable service +sudo systemctl enable --now sunshine +``` + +**Post-Install:** +1. Access Sunshine web UI: `https://localhost:47990` +2. Set initial admin password +3. Complete setup wizard + +--- + +### Step 4.2: Configure AMD VA-API Hardware Encoding + +**Verify AMD GPU:** +```bash +# Check for AMD GPU +lspci | grep -i vga + +# Verify VA-API support +vainfo +``` + +**Expected output:** +``` +vainfo: VA-API version: 1.x +vainfo: Driver version: Mesa Gallium driver ... +vainfo: Supported profile and entrypoints: + VAProfileH264Main : VAEntrypointEncSlice + VAProfileHEVCMain : VAEntrypointEncSlice +``` + +**Configure Sunshine for VA-API:** + +1. Open Sunshine Web UI → Configuration → Video +2. Set: + - **Encoder:** `vaapi` + - **Adapter:** `/dev/dri/renderD128` (default AMD) + - **Codec:** `H.265/HEVC` (better compression, lower latency) + +**Sunshine config file** (`~/.config/sunshine/sunshine.conf`): +```ini +[video] +encoder = vaapi +adapter_name = /dev/dri/renderD128 +hevc_mode = 2 # Always use HEVC when client supports + +[stream] +fps = [30, 60, 120] +resolutions = [ + 1920x1080, + 2560x1440, + 3840x2160 +] +``` + +--- + +### Step 4.3: Add Applications to Sunshine + +**Access:** Sunshine Web UI → Applications + +**Recommended Applications:** + +| Name | Command | Working Directory | +|------|---------|-------------------| +| Desktop | - | - | +| Steam Big Picture | `steam -bigpicture` | - | +| Lutris | `lutris` | - | +| Heroic Games | `heroic` | - | + +**For specific games:** +``` +Name: Cyberpunk 2077 +Command: steam steam://rungameid/1091500 +Detached: ["steam"] +``` + +--- + +### Step 4.4: Install Tailscale on Nobara + +```bash +# Install Tailscale +curl -fsSL https://tailscale.com/install.sh | sh + +# Authenticate +sudo tailscale up + +# Get Tailscale IP +tailscale ip -4 +``` + +**Note the Tailscale IP** (e.g., `100.64.x.x`) - clients will connect to this. + +--- + +### Step 4.5: Configure Sunshine for Tailscale + +**Sunshine listens on all interfaces by default.** For security, restrict to Tailscale: + +**Option A: Bind to Tailscale interface only** +```ini +# ~/.config/sunshine/sunshine.conf +[network] +address_family = both +origin_address = 100.64.x.x # Your Tailscale IP +``` + +**Option B: Firewall rules (recommended)** +```bash +# Allow Sunshine ports only from Tailscale +sudo firewall-cmd --permanent --zone=trusted --add-source=100.64.0.0/10 +sudo firewall-cmd --permanent --zone=trusted --add-port=47984-48010/tcp +sudo firewall-cmd --permanent --zone=trusted --add-port=47998-48010/udp +sudo firewall-cmd --permanent --zone=trusted --add-port=47989/tcp # Web UI +sudo firewall-cmd --permanent --zone=trusted --add-port=47990/tcp # Web UI HTTPS +sudo firewall-cmd --reload +``` + +--- + +### Step 4.6: Install Moonlight Clients + +#### MacBook +```bash +# Homebrew +brew install --cask moonlight + +# Or download from: https://moonlight-stream.org/ +``` + +#### Android +- Install "Moonlight Game Streaming" from Google Play Store + +--- + +### Step 4.7: Pair Moonlight with Sunshine + +1. **On Moonlight client:** + - Add PC manually: Enter Tailscale IP (e.g., `100.64.x.x`) + - Or use MagicDNS hostname: `nobara-pc` (if enabled in Tailscale) + +2. **Enter PIN:** + - Moonlight displays a 4-digit PIN + - Enter in Sunshine Web UI → PIN Pairing + +3. **Verify connection:** + - Moonlight should show your configured applications + +--- + +### Step 4.8: MikroTik QoS for Gaming (Optional but Recommended) + +**Goal:** Prioritize Nobara PC traffic to prevent bufferbloat during gaming sessions. + +**SSH to MikroTik:** +```bash +ssh -i /root/.ssh/mikrotik_key -p 2222 unraid@192.168.31.1 +``` + +**Create Simple Queue for Gaming PC:** +```routeros +# First, find Nobara's IP (replace with actual) +# Assuming Nobara is at 192.168.31.50 + +# Create queue for gaming priority +/queue simple add \ + name="Gaming-Priority" \ + target=192.168.31.50 \ + max-limit=0/0 \ + priority=1/1 \ + queue=default-small/default-small \ + comment="Nobara Gaming PC Priority" + +# Alternative: Use queue tree for more control +/queue tree add \ + name="Gaming-Upload" \ + parent=global \ + packet-mark=gaming-upload \ + priority=1 \ + max-limit=50M + +/queue tree add \ + name="Gaming-Download" \ + parent=global \ + packet-mark=gaming-download \ + priority=1 \ + max-limit=100M + +# Mark gaming traffic +/ip firewall mangle add \ + chain=prerouting \ + src-address=192.168.31.50 \ + action=mark-packet \ + new-packet-mark=gaming-upload \ + passthrough=yes + +/ip firewall mangle add \ + chain=postrouting \ + dst-address=192.168.31.50 \ + action=mark-packet \ + new-packet-mark=gaming-download \ + passthrough=yes +``` + +--- + +### Step 4.9: Optimize Streaming Settings + +**Sunshine Settings (Server):** + +| Setting | LAN Value | Remote Value | +|---------|-----------|--------------| +| Bitrate | 50-80 Mbps | 20-40 Mbps | +| FPS | 60-120 | 60 | +| Resolution | Native | 1080p | +| Codec | HEVC | HEVC | + +**Moonlight Settings (Client):** + +| Setting | LAN Value | Remote Value | +|---------|-----------|--------------| +| Video Codec | HEVC (if supported) | HEVC | +| Frame Pacing | V-Sync | On | +| Bitrate | Auto or 50+ Mbps | 20 Mbps | +| Resolution | Match display | 1080p | + +--- + +## Network Path Analysis + +**Tailscale P2P (Same Network):** +``` +MacBook → Router → Nobara PC +Latency: <1ms additional overhead +``` + +**Tailscale Relayed (Different Network):** +``` +MacBook → Tailscale DERP → Nobara PC +Latency: ~20-50ms additional overhead +``` + +**With Tailscale Direct (NAT Traversal Success):** +``` +MacBook (Office) → Internet → Home Router → Nobara PC +Latency: RTT/2 + encoding latency (~30-80ms typical) +``` + +--- + +## Service Interruption Assessment + +| Action | Risk | Impact | Mitigation | +|--------|------|--------|------------| +| Install Sunshine | NONE | Nobara only | - | +| Install Tailscale | NONE | Nobara only | - | +| MikroTik QoS | LOW | May affect other traffic briefly | Test during low usage | +| Firewall rules | LOW | Nobara only | Can revert | + +--- + +## Verification Checklist + +- [ ] Sunshine installed and running on Nobara +- [ ] VA-API encoding verified: `vainfo` shows HEVC support +- [ ] Tailscale running on Nobara: `tailscale status` +- [ ] Sunshine Web UI accessible: `https://:47990` +- [ ] Moonlight paired successfully +- [ ] Desktop streaming works (low latency) +- [ ] Game streaming works at 60 FPS +- [ ] Remote streaming works (via Tailscale from external network) +- [ ] MikroTik QoS queue active (optional) + +--- + +## Troubleshooting + +### High Latency / Stuttering + +1. **Check Tailscale connection type:** + ```bash + tailscale status + # Look for "direct" vs "relay" + ``` + +2. **Force direct connection:** + ```bash + tailscale ping + ``` + +3. **Lower bitrate in Moonlight** + +### Encoding Errors + +1. **Verify VA-API:** + ```bash + sudo vainfo + # Should show HEVC/H264 encode support + ``` + +2. **Check Sunshine logs:** + ```bash + journalctl -u sunshine -f + # Or: ~/.config/sunshine/sunshine.log + ``` + +3. **Fall back to software encoding:** + ```ini + # sunshine.conf + encoder = software + ``` + +### No Audio + +1. **Check PulseAudio/PipeWire:** + ```bash + pactl list sinks + ``` + +2. **Set Sunshine audio sink:** + - Web UI → Audio → Select correct output + +--- + +## Security Considerations + +1. **Tailscale-only access:** Sunshine is only reachable via Tailscale network + +2. **PIN pairing:** Each client must be manually paired + +3. **Web UI protection:** Consider adding Authentik forward auth (optional) + +4. **Firewall:** Block Sunshine ports from non-Tailscale interfaces + +--- + +## Optional: Expose Sunshine Web UI via Traefik + +If you want to manage Sunshine remotely via browser: + +**Add to Traefik dynamic.yml:** +```yaml +http: + routers: + sunshine-secure: + rule: "Host(`sunshine.xtrm-lab.org`)" + entryPoints: + - https + middlewares: + - default-headers + - authentik-forward-auth # Protect with Authentik + tls: + certResolver: cloudflare + service: sunshine + + services: + sunshine: + loadBalancer: + servers: + - url: "https://192.168.31.50:47990" # Nobara IP +``` + +**MikroTik hairpin NAT (if needed):** +```routeros +# Only if accessing from LAN via external hostname +/ip/firewall/nat add chain=srcnat \ + action=masquerade \ + src-address=192.168.31.0/24 \ + dst-address=192.168.31.50 \ + dst-port=47990 \ + protocol=tcp +``` + +--- + +## Files Modified + +| File/System | Change | Backup Required | +|-------------|--------|-----------------| +| Nobara: ~/.config/sunshine/ | Sunshine config | No (new install) | +| Nobara: firewalld | Allow Tailscale ports | Can revert | +| MikroTik: Queue | Gaming priority | N/A | +| Traefik dynamic.yml (optional) | Sunshine route | YES | + +--- + +## Performance Expectations + +| Scenario | Expected Latency | FPS | +|----------|------------------|-----| +| LAN (same network) | 5-15ms | 60-120 | +| Remote (Tailscale direct) | 30-60ms | 60 | +| Remote (Tailscale relay) | 50-100ms | 60 | + +--- + +## Dependencies for Next Phase + +Phase 5 (RustDesk) provides: +- Alternative remote access when gaming not required +- Lower resource usage for general desktop access +- Additional fallback for remote management diff --git a/docs/05-PHASE5-RUSTDESK.md b/docs/05-PHASE5-RUSTDESK.md new file mode 100644 index 0000000..d79a335 --- /dev/null +++ b/docs/05-PHASE5-RUSTDESK.md @@ -0,0 +1,187 @@ +# Phase 5: Hardened RustDesk Self-Hosted Setup + +## Status: ✅ SERVER-SIDE COMPLETE + +**Last Verified:** 2026-01-18 + +--- + +## Goal +Deploy a high-security, self-hosted RustDesk infrastructure with custom ID server, relay server, and end-to-end encryption using your own keypair. + +--- + +## Current State + +### Server Components + +| Component | Container | Status | Ports | +|-----------|-----------|--------|-------| +| ID Server | rustdesk-hbbs | ✅ Running | TCP 21115-21116, UDP 21116, WS 21118-21119 | +| Relay Server | rustdesk-hbbr | ✅ Running | TCP 21117 | + +### Configuration + +| Parameter | Value | +|-----------|-------| +| Public Key | `+Xlxh96tqwh9tD58ctOmB05Qpfs0ByCoLQcF+yCw0J8=` | +| ID Server | rustdesk.xtrm-lab.org:21116 | +| Relay Server | rustdesk.xtrm-lab.org:21117 | +| DNS | rustdesk.xtrm-lab.org → 62.73.120.142 | +| Data Path | /mnt/user/appdata/rustdesk-server | + +### MikroTik NAT Rules + +| Rule | Protocol | WAN Port | Destination | +|------|----------|----------|-------------| +| RustDesk NAT Test | TCP | 21115 | 192.168.31.2:21115 | +| RustDesk ID Server | TCP | 21116 | 192.168.31.2:21116 | +| RustDesk ID Server | UDP | 21116 | 192.168.31.2:21116 | +| RustDesk Relay | TCP | 21117 | 192.168.31.2:21117 | + +### Port Connectivity (Verified) + +| Port | Protocol | Status | +|------|----------|--------| +| 21116 | TCP | ✅ Accessible | +| 21117 | TCP | ✅ Accessible | + +--- + +## Client Configuration + +To connect RustDesk clients to your self-hosted server: + +### Settings +``` +ID Server: rustdesk.xtrm-lab.org +Relay Server: rustdesk.xtrm-lab.org +Key: +Xlxh96tqwh9tD58ctOmB05Qpfs0ByCoLQcF+yCw0J8= +``` + +### Connection String (for quick setup) +``` +rustdesk.xtrm-lab.org,+Xlxh96tqwh9tD58ctOmB05Qpfs0ByCoLQcF+yCw0J8= +``` + +--- + +## Verification Checklist + +### Server-Side (Complete) +- [x] Keypair generated: `/mnt/user/appdata/rustdesk-server/id_ed25519*` +- [x] hbbs container running +- [x] hbbr container running +- [x] MikroTik NAT rules configured (4 rules) +- [x] DNS resolves: rustdesk.xtrm-lab.org → 62.73.120.142 +- [x] Port 21116 accessible from external +- [x] Port 21117 accessible from external + +### Client-Side (Pending User Testing) +- [ ] Client connects with public key +- [ ] Remote session works between two clients +- [ ] Relay works when direct P2P fails + +--- + +## Architecture + +``` + Internet + │ + ┌────────────▼────────────┐ + │ MikroTik (62.73.120.142)│ + │ NAT Rules: │ + │ TCP 21115-21117 │ + │ UDP 21116 │ + └────────────┬────────────┘ + │ + ┌──────────────────┼──────────────────┐ + │ │ │ + ▼ ▼ ▼ + ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ + │ hbbs (ID Server)│ │ hbbr (Relay) │ │ RustDesk Client │ + │ TCP 21115-21116 │ │ TCP 21117 │ │ Your devices │ + │ UDP 21116 │ │ │ │ │ + │ WS 21118-21119 │ │ │ │ │ + └─────────────────┘ └─────────────────┘ └─────────────────┘ +``` + +--- + +## Container Details + +### hbbs (ID/Rendezvous Server) + +``` +Image: rustdesk/rustdesk-server:latest +Command: hbbs -r rustdesk.xtrm-lab.org:21117 -k _ +Volume: /mnt/user/appdata/rustdesk-server:/root +Ports: 21115, 21116 (TCP+UDP), 21118, 21119 +``` + +### hbbr (Relay Server) + +``` +Image: rustdesk/rustdesk-server:latest +Command: hbbr -k _ +Volume: /mnt/user/appdata/rustdesk-server:/root +Ports: 21117 +``` + +**Note:** The `-k _` flag enforces encrypted connections using the keypair. + +--- + +## Security Features + +1. **End-to-End Encryption:** All connections encrypted with Ed25519 keypair +2. **Key Verification:** Clients must have correct public key to connect +3. **Self-Hosted:** No third-party servers involved +4. **Encrypted-Only Mode:** Unencrypted connections rejected + +--- + +## Maintenance + +### View Logs +```bash +docker logs rustdesk-hbbs --tail 50 +docker logs rustdesk-hbbr --tail 50 +``` + +### Restart Services +```bash +docker restart rustdesk-hbbs rustdesk-hbbr +``` + +### Key Rotation +```bash +# Generate new keypair +docker run --rm -v /mnt/user/appdata/rustdesk-server:/data rustdesk/rustdesk-server hbbs -g + +# Restart containers +docker restart rustdesk-hbbs rustdesk-hbbr + +# Update all clients with new public key +``` + +--- + +## Rollback Procedure + +```bash +# Stop and remove containers +docker stop rustdesk-hbbs rustdesk-hbbr +docker rm rustdesk-hbbs rustdesk-hbbr + +# Remove MikroTik NAT rules (via SSH) +/ip/firewall/nat remove [find comment~RustDesk] +``` + +--- + +## Related Documents + +- [00-CURRENT-STATE.md](./00-CURRENT-STATE.md) - Infrastructure overview +- [04-PHASE4-REMOTE-GAMING.md](./04-PHASE4-REMOTE-GAMING.md) - Sunshine/Moonlight setup diff --git a/docs/06-CHANGELOG.md b/docs/06-CHANGELOG.md new file mode 100644 index 0000000..5876669 --- /dev/null +++ b/docs/06-CHANGELOG.md @@ -0,0 +1,39 @@ +# Changelog + +## 2026-01-18 + +- [PHASE 1] DNS Portability - COMPLETED + - Added DoH route to Traefik dynamic.yml (doh.xtrm-lab.org) + - Verified DoH endpoint working with ad-blocking + - Updated verification checklist - all items complete + - Fixed hostname in docs: dns.xtrm-lab.org → doh.xtrm-lab.org + - Updated nebula-sync status: unhealthy → healthy +- [SERVICE] DoH-Server: Now routed via Traefik at doh.xtrm-lab.org +- [SERVICE] stunnel-dot: Confirmed running for DoT on port 853 +- [ISSUE] Certificate renewal failing - Cloudflare API token needs Zone:DNS:Edit permission (certs expire Feb 11, 2026) +- [PHASE 1] DNS Redundancy verified: + - 2x Pi-hole: MikroTik (172.17.0.2) + Unraid (192.168.31.4) + - 2x Unbound: MikroTik (172.17.0.3) + Unraid (192.168.31.5) + - nebula-sync: Healthy, syncing every 5 minutes + - NAT rules: Properly configured for failover + - Added DNS Redundancy Architecture section to Phase 1 doc +- [DOC] Rewrote 01-PHASE1-DNS-PORTABILITY.md - removed implementation guides, kept only current state + +## 2026-01-18 +- [INFRA] Updated static IP proposal: dockersocket→172.18.0.2, traefik→172.18.0.3, vaultwarden→172.18.0.15 +- [INFRA] Static IP assignment for critical services - COMPLETED + +## 2026-01-18 (Phase 7 Deployment) +- [PHASE 7] Gitea deployed - git.xtrm-lab.org - COMPLETED +- [PHASE 7] Woodpecker CI Server deployed - ci.xtrm-lab.org - COMPLETED +- [PHASE 7] Woodpecker CI Agent deployed and connected - COMPLETED +- [SERVICE] gitea: PostgreSQL database (gitea_db) created +- [SERVICE] woodpecker-server: Port 8008, OAuth via Gitea +- [SERVICE] woodpecker-agent: Connected to server, 2 parallel workflows +- [DNS] Added git.xtrm-lab.org and ci.xtrm-lab.org A records + +## 2026-01-18 (Woodpecker Update) +- [PHASE 7] Woodpecker Server updated to v3.13.0 +- [PHASE 7] Woodpecker Agent updated to v3.13.0 +- [SERVICE] Fixed SQLite database permissions for migration +- [CI] First pipeline test successful (infrastructure repo) diff --git a/docs/06-PHASE6-PORTAINER-MANAGEMENT.md b/docs/06-PHASE6-PORTAINER-MANAGEMENT.md new file mode 100644 index 0000000..2bfdcf9 --- /dev/null +++ b/docs/06-PHASE6-PORTAINER-MANAGEMENT.md @@ -0,0 +1,159 @@ +# Phase 6: Multi-Host Docker Management with Portainer + +## Overview + +**Goal:** Unified container management dashboard for Unraid Docker. + +| Component | Role | +|-----------|------| +| Portainer CE | Management hub (runs on Unraid) | +| Unraid Docker | Local host via Unix socket | + +> **Note:** MikroTik RouterOS containers cannot be managed via Portainer - see [Limitation](#mikrotik-limitation) section. + +--- + +## Phase 6.1: Unraid Server Setup ✅ COMPLETED + +**Goal:** Install and configure the Portainer controller. + +### Tasks + +- [x] Install Portainer CE container via Docker CLI +- [x] Configure container settings: + - Network Type: **Bridge** + - Port Mapping: Container **9000** → Host **9002** (changed due to Authentik conflict) + - Port Mapping: Container **9443** → Host **9444** + - Path Mappings: + - Host `/var/run/docker.sock` → Container `/var/run/docker.sock` + - Host `/mnt/user/appdata/portainer` → Container `/data` +- [x] Add Unraid labels (`net.unraid.docker.managed`, `net.unraid.docker.icon`) +- [x] Add Tailscale labels (`tailscale.expose`, `tailscale.host`, `tailscale.port`) +- [x] Start container +- [x] Initialize Portainer via web UI + +### Container Configuration + +```bash +docker run -d \ + --name=portainer \ + --restart=unless-stopped \ + -p 9002:9000 \ + -p 9444:9443 \ + -v /var/run/docker.sock:/var/run/docker.sock \ + -v /mnt/user/appdata/portainer:/data \ + --label 'net.unraid.docker.managed=dockerman' \ + --label 'net.unraid.docker.icon=https://raw.githubusercontent.com/lllllllillllllillll/Dashboard-Icons/main/png/portainer.png' \ + --label 'net.unraid.docker.webui=http://100.100.208.70:9002' \ + --label 'tailscale.expose=true' \ + --label 'tailscale.host=100.100.208.70' \ + --label 'tailscale.port=9002' \ + portainer/portainer-ce:latest +``` + +### Access URLs +- LAN: `http://192.168.31.2:9002` +- Tailscale: `http://100.100.208.70:9002` +- HTTPS LAN: `https://192.168.31.2:9444` +- HTTPS Tailscale: `https://100.100.208.70:9444` + +### Verification +- [x] Portainer container running +- [x] Portainer UI accessible +- [x] Local Unraid environment connected + +--- + +## Phase 6.2 & 6.3: MikroTik Integration ❌ NOT FEASIBLE + +### MikroTik Limitation + +**MikroTik RouterOS does not use Docker.** It has its own proprietary container runtime that: + +- Does NOT have a Docker daemon +- Does NOT expose `/var/run/docker.sock` +- Does NOT support Docker API +- Can ONLY be managed via RouterOS CLI/API + +### What Was Attempted + +1. Created veth interface (`veth-socat` at 172.17.0.5) +2. Added bridge port to `docker-bridge` +3. Created mount for `/var/run/docker.sock` +4. Deployed `alpine/socat` container +5. Added firewall and NAT rules + +### Why It Failed + +``` +socat[2] E connect(, AF=1 "/var/run/docker.sock", 22): No such file or directory +``` + +The socket doesn't exist because MikroTik's container system is not Docker-based. + +### Cleanup Performed + +All MikroTik changes were reverted: +- Removed socat container +- Removed veth-socat interface +- Removed docker_sock mount +- Removed firewall/NAT rules + +--- + +## MikroTik Container Management Alternatives + +Since Portainer cannot connect to MikroTik, use these methods instead: + +### 1. RouterOS CLI (SSH) + +```bash +# From Unraid +ssh -i /root/.ssh/mikrotik_key -p 2222 unraid@192.168.31.1 + +# List containers +/container/print + +# Start/stop containers +/container/start 0 +/container/stop 0 + +# View logs +/log/print where topics~"container" +``` + +### 2. WinBox/WebFig + +Access MikroTik web interface at `http://192.168.31.1` to manage containers via GUI. + +### 3. RouterOS REST API + +MikroTik RouterOS 7+ has a REST API that can be used for automation: +``` +GET https://192.168.31.1/rest/container +``` + +--- + +## Current Status Summary + +| Component | Status | Access | +|-----------|--------|--------| +| Portainer (Unraid) | ✅ Running | http://100.100.208.70:9002 | +| Unraid Docker | ✅ Connected | Via Portainer | +| MikroTik Containers | ⚠️ Separate | Via RouterOS CLI | + +--- + +## Rollback Plan + +If Portainer issues occur: +```bash +docker stop portainer && docker rm portainer +``` + +--- + +## Related Documents + +- [00-CURRENT-STATE.md](./00-CURRENT-STATE.md) - Infrastructure overview diff --git a/docs/07-CHANGELOG.md b/docs/07-CHANGELOG.md new file mode 100644 index 0000000..dc7fac9 --- /dev/null +++ b/docs/07-CHANGELOG.md @@ -0,0 +1,580 @@ +# Infrastructure Changelog +## 2026-01-18 +- [INFRA] Added pending task: Static IP assignment for critical services on dockerproxy and bridge networks +- [SERVICE] postgresql17: Recreated container (was stopped due to port conflict) +- [SERVICE] authentik + authentik-worker: Restarted after PostgreSQL fix +- [TEMPLATE] Added RustDesk container templates with icons +- [TEMPLATE] Updated Pi-hole template with proper Unraid CA metadata + + +Track all changes to services, configurations, and phase progress. + +--- + +## 2026-01-17 - Homarr + Portainer Integration + +### Portainer App Added to Homarr +- [SERVICE] homarr: Added Portainer app to dashboard +- Section: Monitoring +- URL: http://100.100.208.70:9002 (Tailscale) +- Ping URL: http://192.168.31.2:9002 (LAN) + +### Docker Integration Added +- [SERVICE] homarr: Added Docker integration via socket +- Integration name: Docker (Unraid) +- Socket: unix:///var/run/docker.sock +- Linked to Portainer app for container status display + +### Database Changes +- Added app record for Portainer +- Added item and item_layout for Monitoring section +- Added integration record for Docker +- Linked integration to Portainer item + +### Access +- Homarr: https://xtrm-lab.org +- Portainer visible in Monitoring section + +--- + +## 2026-01-17 - Phase 6.2/6.3 Cancelled: MikroTik Incompatible + +### Discovery +- MikroTik RouterOS containers are NOT Docker-based +- No `/var/run/docker.sock` exists on MikroTik +- Portainer cannot connect to MikroTik's container runtime + +### What Was Attempted +- Created veth-socat interface (172.17.0.5) +- Deployed alpine/socat container +- Added firewall and NAT rules for port 2375 +- Socat failed: `No such file or directory` for docker.sock + +### Cleanup Performed +- Removed socat container +- Removed veth-socat interface and bridge port +- Removed docker_sock mount +- Removed firewall/NAT rules for port 2375 + +### Conclusion +- Phase 6.2 and 6.3 are NOT FEASIBLE +- MikroTik containers must be managed via RouterOS CLI/WebFig +- Portainer remains useful for Unraid-only container management + +### Status Update +- [PHASE 6.1] COMPLETED - Portainer managing Unraid +- [PHASE 6.2] CANCELLED - MikroTik incompatible +- [PHASE 6.3] CANCELLED - MikroTik incompatible + +--- + +## 2026-01-17 - Unraid Container Labels Fixed + +### Containers Updated +- [SERVICE] unbound: Added Unraid labels (`net.unraid.docker.managed`, `net.unraid.docker.icon`) +- [SERVICE] portainer: Added Unraid labels + Tailscale labels + +### Portainer Labels +- `net.unraid.docker.managed=dockerman` +- `net.unraid.docker.icon` - Portainer icon +- `net.unraid.docker.webui=http://100.100.208.70:9002` +- `tailscale.expose=true` +- `tailscale.host=100.100.208.70` +- `tailscale.port=9002` + +### Unbound Labels +- `net.unraid.docker.managed=dockerman` +- `net.unraid.docker.icon` - Unbound icon + +### Note +Both containers recreated to apply labels. Services verified working after recreation. + +--- + +## 2026-01-17 - Phase 6.1 Completed: Portainer CE Deployed + +### Portainer CE Installation +- [PHASE 6.1] Portainer CE deployed on Unraid - COMPLETED +- Container: `portainer/portainer-ce:latest` +- HTTP Port: **9002** (changed from 9000 due to Authentik conflict) +- HTTPS Port: **9444** +- Data: `/mnt/user/appdata/portainer` +- LAN URL: `http://192.168.31.2:9002` +- Tailscale URL: `http://100.100.208.70:9002` + +### Port Conflict Resolution +- Original plan: port 9000 +- Conflict: Authentik already using port 9000 +- Resolution: Mapped to port 9002 (HTTP) and 9444 (HTTPS) + +### Next Steps +- Phase 6.2: Deploy Socat proxy on MikroTik (port 2375) +- Phase 6.3: Connect MikroTik environment to Portainer + +### Status +- [PHASE 6.1] COMPLETED - Portainer running, needs initial setup via web UI +- [PHASE 6.2] NOT STARTED +- [PHASE 6.3] NOT STARTED + +--- + +## 2026-01-17 - Phase 6 Added: Multi-Host Docker Management + +### New Documentation +- [PHASE 6] Created 06-PHASE6-PORTAINER-MANAGEMENT.md +- Portainer CE deployment plan for unified Docker management +- Covers Unraid local setup and MikroTik remote API via Socat + +### Phase 6 Components +- Phase 6.1: Portainer CE installation on Unraid (port 9002) +- Phase 6.2: MikroTik Socat proxy for Docker API exposure (port 2375) +- Phase 6.3: Unified dashboard connection + +### Security Considerations +- MikroTik firewall rules to restrict Docker API access to Unraid only +- Unauthenticated API requires network-level security + +### Status +- [PHASE 6] IN PROGRESS - Phase 6.1 completed + +--- + + +## 2026-01-17 - Status Audit + +### Verified Working +- [PHASE 1] Tailscale on Unraid - WORKING (100.100.208.70) +- [PHASE 1] nebula-sync Pi-hole sync - HEALTHY (was unhealthy, now fixed) +- [PHASE 1] stunnel-dot (DoT on 853) - WORKING +- [PHASE 2] Pangolin controller - RUNNING (13 days uptime) +- [PHASE 3] Authentik main container - HEALTHY + +### Issues Found +- [PHASE 1] DoH-Server - RUNNING but BROKEN (can't reach Pi-hole upstream from dockerproxy network) +- [PHASE 2] gerbil - CRASHED (Pangolin returns empty CIDR for WG config) +- [PHASE 3] authentik-worker - CRASHED (PostgreSQL DNS resolution failure) +- [PHASE 5] RustDesk - NOT DEPLOYED + +### MikroTik NAT Status +- Port 853 (DoT) -> Unraid stunnel - OK +- Port 5443 (DoH) -> MikroTik Pi-hole (wrong target, should be Unraid DoH-Server) +- Port 51820 (Fossorial WG) - NOT CONFIGURED +- Ports 21115-21119 (RustDesk) - NOT CONFIGURED + +--- + +## Template for Future Entries + +## YYYY-MM-DD - Description + +### Changes +- [PHASE X] description - STATUS +- [SERVICE] name: what changed + +### Issues +- description of any problems found + +### Notes +- any relevant context + +## 2026-01-17 - DNS Infrastructure Fixes + +### DoH-Server +- [PHASE 1] DoH-Server - WORKING at `doh.xtrm-lab.org` (not `dns.xtrm-lab.org` as documented) +- [ISSUE] Infrastructure docs reference `dns.xtrm-lab.org` but container uses `doh.xtrm-lab.org` +- [ACTION NEEDED] Either update docs OR add Traefik route for `dns.xtrm-lab.org` + +### Unraid Unbound - FIXED +- [PHASE 1] Replaced broken klutchell/unbound with mvance/unbound:latest +- [ROOT CAUSE 1] Original image missing root.hints/root.key files (distroless image issue) +- [ROOT CAUSE 2] MikroTik NAT rules were hijacking Unbound's outbound DNS (192.168.31.0/24 -> Pi-hole) +- [ROOT CAUSE 3] IPv6 not working on br0 macvlan, causing timeout loops + +### MikroTik NAT Changes +- Added rule 6: "Allow Unraid Unbound" - accept UDP from 192.168.31.5 port 53 +- Added rule 8: "Allow Unraid Unbound TCP" - accept TCP from 192.168.31.5 port 53 +- These rules placed BEFORE the "Force DNS to Pi-hole" rules + +### Unbound Configuration +- Location: /mnt/user/appdata/unbound-mvance/ +- Custom config: a-records.conf (disables IPv6, sets logging) +- Image: mvance/unbound:latest +- Network: br0 (macvlan) at 192.168.31.5 + +### Verified Working +- Unraid Unbound (192.168.31.5) - RESOLVED google.com, github.com, cloudflare.com +- Unraid Pi-hole upstreams: 172.17.0.3 (MikroTik Unbound) + 192.168.31.5 (Unraid Unbound) +- DoH endpoint working at doh.xtrm-lab.org +- stunnel-dot (DoT) - already working + +### Still Pending +- MikroTik Pi-hole upstream config needs verification (check if it uses both Unbounds) +- Docs need update: dns.xtrm-lab.org vs doh.xtrm-lab.org + +### MikroTik Pi-hole Upstreams - FIXED +- [PHASE 1] MikroTik Pi-hole was using Google DNS (8.8.8.8, 8.8.4.4) instead of local Unbounds +- Changed upstreams via Pi-hole v6 API to: + - 172.17.0.3#53 - MikroTik local Unbound + - 192.168.31.5#53 - Unraid Unbound +- DNS resolution tested and working + +### Full DNS Redundancy Now Achieved +- Unraid Pi-hole upstreams: 172.17.0.3, 192.168.31.5 +- MikroTik Pi-hole upstreams: 172.17.0.3, 192.168.31.5 +- Both Unbounds working as recursive resolvers +- nebula-sync keeps blocklists in sync between Pi-holes + +--- + +## 2026-01-17 - Gerbil Investigation: Feature Not Available + +### Issue +- Gerbil kept crashing with "invalid CIDR address" error +- Exit node was correctly configured in database +- API returned empty data despite valid configuration + +### Root Cause +- **Pangolin 1.14 Community Edition does not include Exit Nodes feature** +- Exit Nodes / Gerbil functionality requires paid Pangolin license +- The API endpoint exists but returns empty data for CE users + +### Resolution +- Removed gerbil container (feature not available) +- Existing MikroTik WireGuard VPN provides equivalent remote access functionality +- Phase 2 (Fossorial Stack) marked as blocked pending license upgrade + +### Status +- [PHASE 2] gerbil - REMOVED (requires paid Pangolin license) +- [PHASE 2] Pangolin controller - RUNNING (limited to CE features) + +--- + +## 2026-01-18 - Actual Budget OIDC Integration with Authentik + +### Problem +- Actual Budget OIDC login failing with multiple errors + +### Fixes Applied + +#### 1. DNS Resolution (EAI_AGAIN) +- **Issue:** Container couldn't resolve auth.xtrm-lab.org +- **Fix:** Added `--add-host=auth.xtrm-lab.org:` to container +- **Template:** /boot/config/plugins/dockerMan/templates-user/my-actual-budget.xml + +#### 2. JWT Signing Algorithm (HS256 vs RS256) +- **Issue:** Authentik signed tokens with HS256, Actual Budget expected RS256 +- **Root Cause:** OAuth2 provider had no signing key configured +- **Fix:** Set signing_key_id to 'authentik Internal JWT Certificate' in database +- **SQL:** `UPDATE authentik_providers_oauth2_oauth2provider SET signing_key_id = '48203833-f562-4ec6-b782-f566e6d960d5' WHERE client_id = 'actual-budget';` + +#### 3. Insufficient Scope +- **Issue:** Provider had no scope mappings assigned +- **Fix:** Added openid, email, profile scopes to provider +- **SQL:** `INSERT INTO authentik_core_provider_property_mappings (provider_id, propertymapping_id) VALUES (3, 'a24eea06-...'), (3, '4394c150-...'), (3, '7272ab52-...');` + +### Traefik Static IP +- **Issue:** Traefik IP was dynamic, would break actual-budget on restart +- **Fix:** Assigned static IP 172.18.0.10 to Traefik on dockerproxy network +- **Template:** Added `--ip=172.18.0.10` to ExtraParams in my-traefik.xml + +### Final Configuration + +| Component | Setting | +|-----------|---------| +| Traefik | 172.18.0.10 (static) on dockerproxy | +| Actual Budget | --add-host=auth.xtrm-lab.org:172.18.0.10 | +| Authentik Provider | actual-budget with RS256 signing + scopes | + +### Actual Budget OIDC Environment +``` +ACTUAL_OPENID_DISCOVERY_URL=https://auth.xtrm-lab.org/application/o/actual-budget/.well-known/openid-configuration +ACTUAL_OPENID_CLIENT_ID=actual-budget +ACTUAL_OPENID_CLIENT_SECRET= +ACTUAL_OPENID_SERVER_HOSTNAME=https://actual.xtrm-lab.org +``` + +### Status +- [PHASE 3] Actual Budget OIDC - WORKING +- [SERVICE] traefik: Static IP 172.18.0.10 configured +- [SERVICE] actual-budget: OIDC login via Authentik working + +## 2026-01-18 - Phase 5 Completed: RustDesk Self-Hosted Deployment + +### Keypair Generation +- [PHASE 5] Generated Ed25519 keypair for encrypted connections +- Public Key: `+Xlxh96tqwh9tD58ctOmB05Qpfs0ByCoLQcF+yCw0J8=` +- Data directory: /mnt/user/appdata/rustdesk-server/ + +### Containers Deployed +- [SERVICE] rustdesk-hbbs: ID/Rendezvous server on ports 21115-21116 (TCP), 21116 (UDP), 21118-21119 +- [SERVICE] rustdesk-hbbr: Relay server on port 21117 +- Both containers configured with `-k _` for mandatory encryption +- AutoKuma labels added for Uptime Kuma monitoring + +### MikroTik Configuration +- Added NAT rules 24-27 for RustDesk ports +- Added firewall forward rules (Allow RustDesk TCP/UDP) +- Ports forwarded: 21115 (NAT test), 21116 (TCP+UDP), 21117 (Relay) + +### DNS +- rustdesk.xtrm-lab.org already resolving to 62.73.120.142 (DNS only, no proxy) + +### Verification +- All TCP ports (21115, 21116, 21117) accessible externally +- Both containers running healthy +- Logs show successful startup with keypair loaded + +### Client Configuration +| Setting | Value | +|---------|-------| +| ID Server | rustdesk.xtrm-lab.org | +| Relay Server | rustdesk.xtrm-lab.org | +| Public Key | +Xlxh96tqwh9tD58ctOmB05Qpfs0ByCoLQcF+yCw0J8= | + +### Status +- [PHASE 5] RustDesk Self-Hosted - COMPLETED + +## 2026-01-18 - Vaultwarden 502 Fix + +### Issue +- Vaultwarden returning unexpected error when creating new logins +- Traefik logs showed 502 Bad Gateway errors + +### Root Cause +- Traefik config pointed to `http://192.168.31.2:4743` +- Vaultwarden container had no port 4743 mapping (port 80/tcp was not published) +- Both containers on `dockerproxy` network but config used host IP + +### Fix +- Updated `/mnt/user/appdata/traefik/dynamic.yml` +- Changed: `url: "http://192.168.31.2:4743"` → `url: "http://vaultwarden:80"` +- Uses Docker internal DNS which resolves to container IP on dockerproxy network + +### Status +- [SERVICE] vaultwarden: Working - can create/edit logins + +## 2026-01-18 - Progress Summary + +### Completed Phases +- [PHASE 1] DNS Portability - COMPLETE (DoH, DoT, Unbound redundancy) +- [PHASE 5] RustDesk Self-Hosted - COMPLETE (hbbs/hbbr deployed) +- [PHASE 6] Portainer Management - COMPLETE (6.2/6.3 cancelled - MikroTik incompatible) + +### In Progress +- [PHASE 3] Authentik Zero Trust - Actual Budget integrated, more services pending + +### Blocked +- [PHASE 2] Fossorial Stack - Gerbil requires paid Pangolin license + +### Not Started +- [PHASE 4] Remote Gaming (Sunshine/Moonlight) - Starting now + +### Known Issues +- HomeAssistant_inabox: Exited (1) 3 days ago +- pgAdmin4: Exited (137) 2 weeks ago + +## 2026-01-18 - Phase 4 Started: MacBook Prepared + +### MacBook Setup Complete +- [PHASE 4] Moonlight v6.1.0 already installed +- [PHASE 4] Tailscale connected (100.68.118.59) + +### Pending - Nobara Setup +- Install Sunshine on Nobara +- Configure VA-API encoding +- Pair with Moonlight + +### Instructions Saved +- MacBook: ~/Documents/NOBARA-SUNSHINE-SETUP.md + +### Status +- [PHASE 4] MacBook client ready, awaiting Nobara server setup + +## 2026-01-18 - NetAlertX & Uptime Kuma Fixes (Partial) + +### Uptime Kuma - FIXED +- [SERVICE] Added Traefik route for uptime.xtrm-lab.org +- Protected with Authentik forward auth +- Service URL: http://192.168.31.2:3001 + +### NetAlertX - IN PROGRESS +- [ISSUE] Container not scanning network - shows 0 devices +- [ROOT CAUSE] Multiple config files exist: + - /app/config/app.conf (mounted from host) - updated correctly + - /app/back/app.conf (container internal) - has old value '--localnet' +- [ATTEMPTED] Updated /mnt/user/appdata/netalertx/config/app.conf +- [ATTEMPTED] Updated database Settings table +- [ATTEMPTED] Deleted database to force reload +- [DISCOVERED] App reads from /app/back/app.conf which is generated at startup + +### NetAlertX Fix Required +1. The /app/back/app.conf needs to be updated to: + SCAN_SUBNETS=['192.168.31.0/24 --interface=br0'] +2. This file is regenerated on container start from /app/config/app.conf +3. May need to use Settings UI at https://netalert.xtrm-lab.org to change SCAN_SUBNETS + +### Manual ARP Scan Test - WORKS +Command: docker exec NetAlertX arp-scan --localnet --interface=br0 +Result: Found 20 devices on 192.168.31.0/24 + +### Pending Tasks +- Fix NetAlertX to use correct subnet config +- Add Tailscale network scanning (may not work - ARP doesn't work over tunnels) +- User requested: RustFS for personal CDN (assets hosting) + +### Status +- [SERVICE] uptime.xtrm-lab.org - WORKING +- [SERVICE] netalertx - PARTIALLY BROKEN (config issue) + +## 2026-01-18 - NetAlertX FIXED + +### Resolution +- [SERVICE] NetAlertX now scanning network correctly - found 21 devices +- [FIX] Updated config in multiple locations: + - /data/config/app.conf (runtime config inside container) + - /app/back/app.conf (plugin reads from here) + - /mnt/user/appdata/netalertx/config/app.conf (host mount for persistence) + +### Config Change +``` +SCAN_SUBNETS=['192.168.31.0/24 --interface=br0'] +``` + +### Root Cause Summary +- NetAlertX has complex config handling with multiple config file locations +- /app/config (mounted) -> copied to /data/config on startup +- /data/config/app.conf is read by the app +- /app/back/app.conf is read by plugins at runtime +- All three needed to be updated + +### Verified Working +- ARP scan found 21 devices on 192.168.31.0/24 +- Devices visible at https://netalert.xtrm-lab.org/devices.php + +### Note on Tailscale Scanning +- ARP scanning does NOT work over Tailscale (point-to-point tunnel, no broadcast) +- Tailscale devices need to be added manually or via different discovery method + +## 2026-01-18 - RustFS CDN Deployed + +### Service Details +- [SERVICE] RustFS - S3-compatible object storage for personal CDN +- Image: rustfs/rustfs:latest +- Ports: 9010 (S3 API), 9011 (Console) +- Data: /mnt/user/appdata/rustfs/data +- Logs: /mnt/user/appdata/rustfs/logs + +### Access URLs +- S3 API: https://cdn.xtrm-lab.org +- Console: http://192.168.31.2:9011/rustfs/console/ +- Credentials stored in: /mnt/user/appdata/rustfs/CREDENTIALS.txt + +### Traefik Route +- Host: cdn.xtrm-lab.org +- No Authentik protection (public CDN for assets) +- S3 authentication handles access control + +### Usage +Create bucket and upload assets via: +- RustFS Console at port 9011 +- S3-compatible CLI tools (aws-cli, rclone, etc.) +- Direct S3 API calls + +### Example S3 CLI Usage +```bash +# Configure aws-cli +aws configure set aws_access_key_id +aws configure set aws_secret_access_key + +# Create bucket +aws --endpoint-url https://cdn.xtrm-lab.org s3 mb s3://assets + +# Upload file +aws --endpoint-url https://cdn.xtrm-lab.org s3 cp image.png s3://assets/ + +# Public URL (after setting bucket policy) +https://cdn.xtrm-lab.org/assets/image.png +``` + +### Status +- [SERVICE] rustfs - RUNNING +- [PHASE N/A] Personal CDN - COMPLETED + +## 2026-01-18 - PostgreSQL Data Path Restored & Phase 3 Verified + +### Root Cause Analysis +- [INCIDENT] PostgreSQL container was recreated at 07:40 UTC with wrong data path +- [CAUSE] Container used default path `/mnt/user/appdata/postgresql17` instead of configured `/mnt/user/appdata/postgresql` +- [IMPACT] Authentik started with empty database, all configuration appeared lost +- [DATA] Original data was safe in `/mnt/user/appdata/postgresql/` the entire time + +### Resolution +- [FIX] Stopped postgresql17 container +- [FIX] Recreated container with correct volume mount: `/mnt/user/appdata/postgresql:/var/lib/postgresql/data` +- [FIX] Restarted Authentik containers +- [VERIFIED] All Authentik data restored (users, groups, applications, providers) + +### Other Fixes This Session +- [SERVICE] Uptime-Kuma-API: Added missing ADMIN_PASSWORD environment variable +- [TRAEFIK] Added Docker provider constraint to filter broken container labels +- [TRAEFIK] Added missing routes: authentik, transmission, nextcloud to dynamic.yml + +### Phase 3 Verification - COMPLETED +Verified Authentik Zero Trust configuration: +- Users: akadmin, admin, jazzymc (3 active users) +- Groups: authentik Admins, authentik Read-only +- Outpost: Embedded Outpost running (proxy type) +- Applications: XTRM-Lab Protected Services, Actual Budget +- Proxy Provider: forward_domain mode for auth.xtrm-lab.org +- 2FA: 2 TOTP devices configured +- Protected Services: 12 routes using authentik-forward-auth middleware + +### Services Status +- [SERVICE] auth.xtrm-lab.org - WORKING (302 redirect to login) +- [SERVICE] uptime.xtrm-lab.org - WORKING (forward auth active) +- [SERVICE] ph2.xtrm-lab.org - WORKING (forward auth active) +- [SERVICE] All forward-auth protected services - WORKING + +### Documentation Updated +- [DOC] 03-PHASE3-AUTHENTIK-ZEROTRUST.md - Marked as COMPLETED with verified state + +## 2026-01-18 - Phase 5 RustDesk Verified + +### Server-Side Verification Complete +- [x] Keypair exists: /mnt/user/appdata/rustdesk-server/id_ed25519* +- [x] Public Key: +Xlxh96tqwh9tD58ctOmB05Qpfs0ByCoLQcF+yCw0J8= +- [x] hbbs container: Up 10+ hours +- [x] hbbr container: Up 10+ hours +- [x] MikroTik NAT: 4 rules configured (21115-21117 TCP, 21116 UDP) +- [x] DNS: rustdesk.xtrm-lab.org → 62.73.120.142 +- [x] Port 21116 TCP: Externally accessible (verified via nc) +- [x] Port 21117 TCP: Externally accessible (verified via nc) + +### Client Configuration +ID Server: rustdesk.xtrm-lab.org +Relay Server: rustdesk.xtrm-lab.org +Key: +Xlxh96tqwh9tD58ctOmB05Qpfs0ByCoLQcF+yCw0J8= + +### Documentation Updated +- [DOC] 05-PHASE5-RUSTDESK.md - Rewritten with verified state, marked SERVER-SIDE COMPLETE +- [PENDING] Client-side testing (user to verify remote sessions work) + +## 2026-01-18 - Phase 7 GitOps Plan Created + +### New Phase: Gitea + Woodpecker CI +- [DOC] Created 08-PHASE7-GITEA-GITOPS.md +- [PLAN] Lightweight GitOps stack for infrastructure management +- [COMPONENTS] Gitea (~200MB) + Woodpecker Server/Agent (~200MB) +- [TOTAL RESOURCES] ~400MB RAM, ~700MB storage + +### Planned Features +- Git version control for all configs +- Automated YAML validation +- CI/CD pipeline for deployments +- Auto-rollback on health check failure +- Authentik SSO integration +- Safe AI agent integration + +### URLs (Planned) +- git.xtrm-lab.org - Gitea web UI +- ci.xtrm-lab.org - Woodpecker CI dashboard diff --git a/docs/08-PHASE7-GITEA-GITOPS.md b/docs/08-PHASE7-GITEA-GITOPS.md new file mode 100644 index 0000000..4b6929a --- /dev/null +++ b/docs/08-PHASE7-GITEA-GITOPS.md @@ -0,0 +1,168 @@ +# Phase 7: Gitea + Woodpecker CI (GitOps for Homelab) + +## Status: ✅ COMPLETED + +**Deployed:** 2026-01-18 + +--- + +## Deployed Components + +| Service | Container | Version | Port | URL | Status | +|---------|-----------|---------|------|-----|--------| +| Gitea | gitea | 1.25.3 | 3005→3000, 2222→22 | https://git.xtrm-lab.org | ✅ Running | +| Woodpecker Server | woodpecker-server | 3.13.0 | 8008→8000 | https://ci.xtrm-lab.org | ✅ Running | +| Woodpecker Agent | woodpecker-agent | 3.13.0 | - | - | ✅ Running | + +--- + +## Configuration + +### Gitea + +| Parameter | Value | +|-----------|-------| +| Admin User | jazzymc | +| Database | PostgreSQL (gitea_db @ 172.18.0.13) | +| DB User | gitea | +| SSH Port | 2222 | +| Data Path | /mnt/user/appdata/gitea/data | +| Network | dockerproxy | + +### Woodpecker CI + +| Parameter | Value | +|-----------|-------| +| Version | 3.13.0 | +| Admin User | jazzymc (via Gitea OAuth) | +| Server IP | 172.18.0.134 | +| gRPC Port | 9000 | +| HTTP Port | 8000 (mapped to 8008) | +| Max Workflows | 2 (parallel) | +| Data Path | /mnt/user/appdata/woodpecker/server | +| Agent Secret | 564a5716400532874a8e02313a491b4f3864ce9b77a5122ce0eb14777749e740 | + +### Gitea OAuth App (for Woodpecker) + +| Parameter | Value | +|-----------|-------| +| Client ID | 924b3300-b607-4a48-bc26-35b06dbf18c7 | +| Redirect URI | https://ci.xtrm-lab.org/authorize | + +--- + +## Network Configuration + +All services on `dockerproxy` network: + +| Service | Internal IP | DNS Name | +|---------|-------------|----------| +| Gitea | Dynamic | gitea | +| Woodpecker Server | 172.18.0.134 | woodpecker-server | +| PostgreSQL | 172.18.0.13 | postgresql17 | + +### Traefik Routes + +| Domain | Service | Port | +|--------|---------|------| +| git.xtrm-lab.org | gitea | 3000 | +| ci.xtrm-lab.org | woodpecker-server | 8000 | + +--- + +## Verification Checklist + +- [x] Gitea container running +- [x] Gitea accessible at https://git.xtrm-lab.org +- [x] Admin account created (jazzymc) +- [x] OAuth app created for Woodpecker +- [x] Woodpecker Server v3.13.0 running +- [x] Woodpecker Agent v3.13.0 running and connected +- [x] Woodpecker accessible at https://ci.xtrm-lab.org +- [x] Gitea OAuth login working +- [x] CI pipeline tested successfully + +--- + +## Usage + +### Git Operations + +```bash +# Clone via HTTPS +git clone https://git.xtrm-lab.org/jazzymc/infrastructure.git + +# Clone via SSH (port 2222) +git clone ssh://git@git.xtrm-lab.org:2222/jazzymc/infrastructure.git +``` + +### CI Pipeline (.woodpecker.yml) + +```yaml +steps: + - name: test + image: alpine + commands: + - echo 'Hello from Woodpecker CI!' + - date +``` + +### Example: Node.js Pipeline + +```yaml +steps: + - name: install + image: node:20 + commands: + - npm install + + - name: test + image: node:20 + commands: + - npm test + + - name: build + image: node:20 + commands: + - npm run build + when: + branch: main +``` + +### Example: Docker Build + +```yaml +steps: + - name: build + image: docker + commands: + - docker build -t myapp . + volumes: + - /var/run/docker.sock:/var/run/docker.sock +``` + +--- + +## Maintenance + +### Update Woodpecker + +```bash +docker pull woodpeckerci/woodpecker-server:v3 +docker pull woodpeckerci/woodpecker-agent:v3 +docker restart woodpecker-server woodpecker-agent +``` + +### Backup + +Important paths to backup: +- /mnt/user/appdata/gitea/data +- /mnt/user/appdata/woodpecker/server +- PostgreSQL database: gitea_db + +--- + +## Related Documents + +- [00-CURRENT-STATE.md](./00-CURRENT-STATE.md) - Infrastructure overview +- [03-PHASE3-AUTHENTIK-ZEROTRUST.md](./03-PHASE3-AUTHENTIK-ZEROTRUST.md) - SSO setup