jazzymc/infrastructure
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add infrastructure documentation
All checks were successful
ci/woodpecker/push/woodpecker Pipeline was successful
Details

Browse Source
This commit is contained in:
jazzymc
2026-01-18 16:57:25 +02:00
parent 69e1338a5b
commit 62a6267026
10 changed files with 2658 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

265
docs/00-CURRENT-STATE.md Normal file
View File

@@ -0,0 +1,265 @@
# Infrastructure Upgrade Proposal: xtrm-lab.org (v2)
## Current Infrastructure State
**Document Updated:** 2026-01-18
**Target Domain:** xtrm-lab.org
---
## Network Topology
### MikroTik hAP ax³ Router (192.168.31.1)
| Parameter | Value |
|-----------|-------|
| RouterOS Version | 7.20.6 (stable) |
| WAN IP (Static) | 62.73.120.142 |
| LAN Subnet | 192.168.31.0/24 |
| Docker Bridge | 172.17.0.0/24 |
| SSH Access | `ssh -i /root/.ssh/mikrotik_key -p 2222 unraid@192.168.31.1` |
**Interfaces:**
- `ether1` - WAN (62.73.120.142/23)
- `bridge` - LAN (192.168.31.1/24)
- `docker-bridge` - Container network (172.17.0.1/24)
- `back-to-home-vpn` - WireGuard VPN (192.168.216.1/24)
**Running Containers on MikroTik:**
| Container | IP | Purpose |
|-----------|-----|---------|
| pihole:latest | 172.17.0.2 | DNS sinkhole (Pi-hole v6) |
| unbound:latest | 172.17.0.3 | Recursive DNS resolver |
### Unraid Server (192.168.31.2)
**Tailscale IP:** 100.100.208.70
**Key Services:**
| Service | Container Name | Port(s) | Network | External URL |
|---------|---------------|---------|---------|--------------|
| Portainer | portainer | 9002→9000, 9444→9443 | bridge | http://100.100.208.70:9002 (Tailscale) |
| Pi-hole | binhex-official-pihole | 53, 80, 67 | br0 (192.168.31.4) | ph1.xtrm-lab.org |
| Unbound | unbound | 53 | br0 (192.168.31.5) | - |
| Traefik | traefik | 8001→80, 44301→443 | dockerproxy | traefik.xtrm-lab.org |
| Authentik | authentik | 9000, 9443 | dockerproxy | auth.xtrm-lab.org |
| Authentik Worker | authentik-worker | - | authentik | - |
| Vaultwarden | vaultwarden | 4743→80 | bridge | vault.xtrm-lab.org |
| Plex | plex | 32400 | host | plex.xtrm-lab.org |
| Home Assistant | HomeAssistant_inabox | 8123 | host (192.168.31.15) | ha.xtrm-lab.org |
| Transmission | transmission | 9091, 51413 | bridge | - |
| Nextcloud | Nextcloud | 8666→80 | bridge | - |
| PostgreSQL | postgresql17 | 5432 | bridge | - |
| Redis | Redis | 6379 | bridge | - |
| Uptime Kuma | UptimeKuma | 3001 | bridge | - |
| NetAlertX | NetAlertX | 20211 | host | netalert.xtrm-lab.org |
| UrBackup | UrBackup | 55414 | host | urbackup.xtrm-lab.org |
| Homarr | homarr | 10004→7575 | bridge | - |
| Nebula Sync | nebula-sync | - | - | Pi-hole sync |
| DoH Server | DoH-Server | 8053 | dockerproxy | doh.xtrm-lab.org |
| stunnel DoT | stunnel-dot | 853 | bridge | dns.xtrm-lab.org:853 |
| Pangolin | pangolin | 3003→3001, 3004→3002 | bridge | Fossorial controller |
| Gitea | gitea | 3005→3000, 2222→22 | dockerproxy | git.xtrm-lab.org |
| Woodpecker Server | woodpecker-server | 8008→8000 | dockerproxy | ci.xtrm-lab.org |
| Woodpecker Agent | woodpecker-agent | - | dockerproxy | - |
| RustDesk ID | rustdesk-hbbs | 21115-21116, 21118-21119 | bridge | rustdesk.xtrm-lab.org |
| RustDesk Relay | rustdesk-hbbr | 21117 | bridge | rustdesk.xtrm-lab.org |
---
## Current NAT/Port Forwarding (MikroTik)
| Rule | Protocol | WAN Port | Destination | Purpose |
|------|----------|----------|-------------|---------|
| Forward HTTP | TCP | 80 | 192.168.31.2:8001 | Traefik HTTP |
| Forward HTTPS | TCP | 443 | 192.168.31.2:44301 | Traefik HTTPS |
| Plex | TCP | 32400 | 192.168.31.2:32400 | Plex Media Server |
| Transmission | TCP/UDP | 51413 | 192.168.31.2:51413 | BitTorrent |
| DoT | TCP | 853 | 172.17.0.2:853 | DNS over TLS |
| DoH | TCP/UDP | 5443 | 172.17.0.2:443 | DNS over HTTPS |
| DNS Force | UDP/TCP | 53 | 172.17.0.2:53 | Force LAN DNS to Pi-hole |
| RustDesk NAT Test | TCP | 21115 | 192.168.31.2:21115 | RustDesk NAT Test |
| RustDesk ID TCP | TCP | 21116 | 192.168.31.2:21116 | RustDesk ID Server |
| RustDesk ID UDP | UDP | 21116 | 192.168.31.2:21116 | RustDesk ID Server |
| RustDesk Relay | TCP | 21117 | 192.168.31.2:21117 | RustDesk Relay |
---
## Current WireGuard Configuration
**Interface:** `back-to-home-vpn`
- Listen Port: 59188
- Address: 192.168.216.1/24
- Public Key: `3e+p++SJ6f5EURt6WCKApOLMQHWpURm/vn/0s9+EKzs=`
**Existing Peers:**
1. hAP ax³ (secondary device)
2. Kaloyan's S25 Ultra (mobile)
3. Additional peer (unnamed)
---
## Traefik Configuration
**Entry Points:**
- HTTP (:80) → Redirects to HTTPS
- HTTPS (:443)
**Certificate Resolver:** Cloudflare DNS Challenge
- Email: admin@xtrm-lab.org
- DNS Provider: Cloudflare
**Existing Middlewares:**
- `default-headers` - Security headers (HSTS, XSS protection, etc.)
- `authentik-forward-auth` - Forward auth to Authentik (configured but not applied)
- `pihole1-redirect` / `pihole2-redirect` - Redirect root to /admin/
---
## Authentik Configuration
| Parameter | Value |
|-----------|-------|
| Version | 2025.8.1 |
| URL | auth.xtrm-lab.org |
| PostgreSQL Host | postgresql17 |
| Database | authentik_db |
| Redis Host | redis |
| Network | dockerproxy |
**Status:** Deployed but not yet integrated with services
---
## Portainer Configuration (Phase 6)
| Parameter | Value |
|-----------|-------|
| Version | CE Latest |
| HTTP Port | 9002 |
| HTTPS Port | 9444 |
| Data Path | /mnt/user/appdata/portainer |
| Tailscale URL | http://100.100.208.70:9002 |
| Local URL | http://192.168.31.2:9002 |
**Status:** Deployed, awaiting initial setup and MikroTik connection (Phase 6.2/6.3)
---
## DNS Architecture
```
┌─────────────────────────────────────┐
│ Internet │
└───────────────┬─────────────────────┘
│
┌───────────────▼─────────────────────┐
│ MikroTik hAP ax³ (192.168.31.1) │
│ WAN: 62.73.120.142 │
└───────────────┬─────────────────────┘
│
┌────────────────────────┼────────────────────────┐
│ │ │
▼ ▼ ▼
┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐
│ Pi-hole (Router) │ │ Unraid Server │ │ LAN Devices │
│ 172.17.0.2 │ │ 192.168.31.2 │ │ 192.168.31.x │
│ Primary DNS │ │ │ │ │
└────────┬─────────┘ └────────┬─────────┘ └──────────────────┘
│ │
▼ ▼
┌──────────────────┐ ┌──────────────────┐
│ Unbound (Router) │ │ Unbound (Unraid) │
│ 172.17.0.3 │ │ 192.168.31.5 │
│ Recursive DNS │ │ Recursive DNS │
└──────────────────┘ └──────────────────┘
│
▼
┌──────────────────┐
│ Pi-hole (Unraid) │
│ 192.168.31.4 │
│ Secondary DNS │
└──────────────────┘
```
---
## Service Interruption Risk Assessment
| Phase | Component | Interruption Risk | Mitigation |
|-------|-----------|-------------------|------------|
| 1 | Tailscale Integration | LOW | Add-on service, no changes to existing |
| 1 | DoH Endpoint | LOW | New endpoint, existing DNS unaffected |
| 2 | Pangolin/Gerbil | MEDIUM | New containers, may conflict with WG port 51820 |
| 2 | Newt Connector | LOW | Outbound only |
| 3 | Authentik Forward Auth | HIGH | Will gate all services - test thoroughly |
| 4 | Sunshine/Moonlight | LOW | New service, Tailscale-only access |
| 5 | RustDesk | MEDIUM | New ports required on MikroTik |
| 6 | Portainer | LOW | Management tool only, no service impact |
---
## Ports Required for Full Implementation
### New MikroTik Port Forwards Needed:
| Service | Protocol | Port(s) | Destination | Phase |
|---------|----------|---------|-------------|-------|
| WireGuard (Fossorial) | UDP | 51820 | 192.168.31.2:51820 | 2 |
| RustDesk ID TCP | TCP | 21115-21117 | 192.168.31.2:21115-21117 | 5 |
| RustDesk Relay | TCP | 21118-21119 | 192.168.31.2:21118-21119 | 5 |
| RustDesk NAT | UDP | 21116 | 192.168.31.2:21116 | 5 |
---
## Next Steps
Proceed to individual phase documents:
1. [Phase 1: Global DNS Portability](./01-PHASE1-DNS-PORTABILITY.md)
2. [Phase 2: Fossorial Tunnel Stack](./02-PHASE2-FOSSORIAL-STACK.md)
3. [Phase 3: Identity & Zero Trust](./03-PHASE3-AUTHENTIK-ZEROTRUST.md)
4. [Phase 4: Remote Gaming](./04-PHASE4-REMOTE-GAMING.md)
5. [Phase 5: RustDesk Setup](./05-PHASE5-RUSTDESK.md)
6. [Phase 6: Portainer Management](./06-PHASE6-PORTAINER-MANAGEMENT.md)
---
## Completed Infrastructure Tasks
### Static IP Assignment for Critical Services
**Status:** COMPLETED (2026-01-18)
**Priority:** High
**Reason:** Critical services should have static IPs outside DHCP/dynamic lease range to prevent IP conflicts and ensure reliable inter-container communication.
#### dockerproxy Network (172.18.0.0/16)
Static IP range: 172.18.0.2 - 172.18.0.50
| Service | Static IP |
|---------|-----------|
| dockersocket | 172.18.0.2 |
| traefik | 172.18.0.3 |
| authentik | 172.18.0.11 |
| authentik-worker | 172.18.0.12 |
| postgresql17 | 172.18.0.13 |
| Redis | 172.18.0.14 |
| vaultwarden | 172.18.0.15 |
#### bridge Network (172.17.0.0/16)
Static IP range: 172.17.0.2 - 172.17.0.50
| Service | Static IP |
|---------|-----------|
| portainer | 172.17.0.2 |
| rustdesk-hbbs | 172.17.0.3 |
| rustdesk-hbbr | 172.17.0.4 |
#### Implementation Steps
1. [x] Update Docker network IPAM config to reserve static range
2. [x] Recreate critical containers with --ip flag or docker-compose static IP
3. [x] Update any hardcoded references to old IPs
4. [x] Test inter-container connectivity
5. [x] Document final IP assignments
**Note:** IPs assigned via `docker network connect --ip`. To persist across container recreation, update Unraid Docker templates or use docker-compose.

208
docs/01-PHASE1-DNS-PORTABILITY.md Normal file
View File

@@ -0,0 +1,208 @@
# Phase 1: Global DNS Portability
## Status: ✅ COMPLETED (2026-01-18)
Pi-hole ad-blocking works on all devices via Tailscale MagicDNS and DoH/DoT endpoints.
---
## Tailscale Configuration
| Parameter | Value |
|-----------|-------|
| Unraid Tailscale IP | 100.100.208.70 |
| Hostname | xtrm-unraid |
| Subnet Route | 192.168.31.0/24 (advertised & approved) |
| Global DNS | Pi-hole via MagicDNS |
| Override Local DNS | Enabled |
### Connected Devices
| Device | Tailscale IP | Status |
|--------|--------------|--------|
| xtrm-unraid | 100.100.208.70 | Online |
| kaloyans-macbook-air | 100.68.118.59 | Active |
| mikrotik-tailscale-1 | 100.75.93.123 | Online |
| samsung-sm-s938b | 100.111.64.56 | Offline |
---
## DNS Services
### Pi-hole Instances
| Instance | Location | IP | Web UI | Status |
|----------|----------|-----|--------|--------|
| Primary | MikroTik Container | 172.17.0.2 | ph2.xtrm-lab.org | ✅ Running |
| Secondary | Unraid (macvlan br0) | 192.168.31.4 | ph1.xtrm-lab.org | ✅ Running |
**Sync:** nebula-sync (healthy) syncs Unraid → MikroTik every 5 minutes
### Unbound Instances (Recursive DNS)
| Instance | Location | IP | Status |
|----------|----------|-----|--------|
| Primary | MikroTik Container | 172.17.0.3 | ✅ Running |
| Secondary | Unraid (macvlan br0) | 192.168.31.5 | ✅ Running |
### Pi-hole Upstream Configuration
**Unraid Pi-hole (192.168.31.4):**
```
upstreams = ["172.17.0.3#53", "192.168.31.5#53"]
```
---
## DoH Endpoint (DNS over HTTPS)
| Parameter | Value |
|-----------|-------|
| URL | `https://doh.xtrm-lab.org/dns-query` |
| Container | DoH-Server (ghcr.io/ich777/doh-server) |
| Listen Port | 8053 |
| Upstream DNS | udp:192.168.31.1:53 |
| Network | dockerproxy |
**Traefik Route (dynamic.yml):**
```yaml
doh-secure:
rule: "Host(`doh.xtrm-lab.org`)"
entryPoints: [https]
tls:
certResolver: cloudflare
service: doh # → http://DoH-Server:8053
```
**Test:**
```bash
curl -H 'accept: application/dns-json' 'https://doh.xtrm-lab.org/dns-query?name=google.com&type=A'
```
---
## DoT Endpoint (DNS over TLS)
| Parameter | Value |
|-----------|-------|
| Hostname | doh.xtrm-lab.org:853 |
| Container | stunnel-dot (dweomer/stunnel) |
| Accept Port | 853 |
| Forward To | 192.168.31.4:53 (Unraid Pi-hole) |
**MikroTik NAT:** WAN:853 → 192.168.31.2:853
**Android Private DNS:** Settings → Private DNS → `doh.xtrm-lab.org`
---
## DNS Architecture
```
┌─────────────────────────────────────┐
│ External Clients │
└──────────────┬──────────────────────┘
│
┌─────────────────────────┼─────────────────────────┐
│ │ │
▼ ▼ ▼
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Tailscale │ │ DoH │ │ DoT │
│ MagicDNS │ │ doh.xtrm-lab.org│ │ :853 │
│ 100.100.100.100 │ │ (Traefik→DoH) │ │ (stunnel) │
└────────┬────────┘ └────────┬────────┘ └────────┬────────┘
│ │ │
└────────────────────────┼────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────┐
│ LAN (192.168.31.0/24) │
│ │
│ ┌─────────────────────────┐ ┌─────────────────────────┐ │
│ │ MikroTik Pi-hole │ │ Unraid Pi-hole │ │
│ │ 172.17.0.2 │ │ 192.168.31.4 │ │
│ │ (NAT forced for LAN) │ │ (Direct access allowed) │ │
│ └───────────┬─────────────┘ └───────────┬─────────────┘ │
│ │ │ │
│ ▼ ▼ │
│ ┌─────────────────────────┐ ┌─────────────────────────┐ │
│ │ MikroTik Unbound │ │ Unraid Unbound │ │
│ │ 172.17.0.3 (recursive) │ │ 192.168.31.5 (recursive)│ │
│ └─────────────────────────┘ └─────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
```
---
## DHCP DNS Servers (MikroTik)
| Priority | Server | Notes |
|----------|--------|-------|
| Primary | 192.168.31.1 | MikroTik (NAT forces to Pi-hole 172.17.0.2) |
| Secondary | 192.168.31.4 | Unraid Pi-hole (direct, for failover) |
---
## MikroTik DNS NAT Rules
| Rule | Action | Description |
|------|--------|-------------|
| 3 | ACCEPT | Traffic TO 192.168.31.4:53 (allows Unraid Pi-hole) |
| 7 | DST-NAT | Force LAN DNS to 172.17.0.2 (MikroTik Pi-hole) |
| 21 | DST-NAT | DoT WAN:853 → 192.168.31.2:853 |
---
## Failover Behavior
| Scenario | Behavior |
|----------|----------|
| MikroTik Pi-hole down | Clients use secondary DNS (192.168.31.4) |
| MikroTik Unbound down | Pi-holes use Unraid Unbound (192.168.31.5) |
| Unraid down | MikroTik services continue independently |
---
## Client Configuration
### macOS (DoH)
- Firefox/Chrome: Settings → Security → Custom DNS → `https://doh.xtrm-lab.org/dns-query`
- System-wide: Install DNS profile with DoH URL
### Android (DoT)
- Settings → Network → Private DNS → `doh.xtrm-lab.org`
### Tailscale Clients
- Automatic via MagicDNS (no configuration needed)
---
## Verification Commands
```bash
# Test DoH endpoint
curl -H 'accept: application/dns-json' 'https://doh.xtrm-lab.org/dns-query?name=google.com&type=A'
# Test ad-blocking via DoH
curl -H 'accept: application/dns-json' 'https://doh.xtrm-lab.org/dns-query?name=ads.google.com&type=A'
# Expected: 0.0.0.0
# Test Pi-holes directly
dig +short google.com @172.17.0.2 # MikroTik Pi-hole
dig +short google.com @192.168.31.4 # Unraid Pi-hole
# Test Unbound directly
dig +short google.com @172.17.0.3 # MikroTik Unbound
dig +short google.com @192.168.31.5 # Unraid Unbound
# Check Tailscale ad-blocking
dig +short ads.google.com @100.100.100.100
# Expected: 0.0.0.0
```
---
## Known Issues
| Issue | Status | Notes |
|-------|--------|-------|
| Certificate renewal failing | ⚠️ Open | Cloudflare API token needs Zone:DNS:Edit permission. Certs expire Feb 11, 2026. |

391
docs/02-PHASE2-FOSSORIAL-STACK.md Normal file
View File

@@ -0,0 +1,391 @@
# Phase 2: The "Fossorial" Tunnel Stack (Pangolin, Gerbil, Newt)
## Goal
Deploy the Fossorial tunnel stack using your MikroTik's static IP (62.73.120.142) to host tunnels without requiring a VPS, with MikroTik container fallback for resilience.
---
## What is Fossorial?
Fossorial is a self-hosted tunnel solution consisting of:
| Component | Purpose | Role |
|-----------|---------|------|
| **Pangolin** | Central controller/dashboard | Manages tunnels, provides web UI |
| **Gerbil** | WireGuard manager | Handles WireGuard peer configuration |
| **Newt** | Tunnel connector | Lightweight agent that "dials out" to establish tunnels |
**Why Fossorial over plain WireGuard?**
- Automatic peer management
- Web-based tunnel configuration
- Self-healing connections via Newt
- Easier certificate/identity management
---
## Current WireGuard State (MikroTik)
```
Interface: back-to-home-vpn
├── Listen Port: 59188 (non-standard - good!)
├── Address: 192.168.216.1/24
├── Public Key: 3e+p++SJ6f5EURt6WCKApOLMQHWpURm/vn/0s9+EKzs=
└── Peers: 3 configured
```
**Port 51820 Status:** NOT in use - available for Fossorial
---
## Architecture Overview
```
Internet
│
┌────────────▼────────────┐
│ MikroTik (62.73.120.142)│
│ Port Forward: │
│ UDP 51820 → Unraid │
│ TCP 443 → Traefik │
└────────────┬────────────┘
│
┌──────────────────┼──────────────────┐
│ │ │
▼ ▼ ▼
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Pangolin │ │ Gerbil │ │ Newt │
│ (Controller) │ │ (WG Manager) │ │ (Connector) │
│ :3000 web UI │ │ :51820 WG │ │ Outbound only │
│ │ │ :8080 API │ │ │
└────────┬────────┘ └────────┬────────┘ └────────┬────────┘
│ │ │
└───────────────────┴───────────────────┘
Internal Network
192.168.31.0/24
```
---
## Implementation Steps
### Step 2.1: Create Docker Network for Fossorial
```bash
docker network create --driver bridge fossorial
```
---
### Step 2.2: Deploy Pangolin (Controller)
**Unraid Docker Template:**
```xml
<?xml version="1.0"?>
<Container version="2">
<Name>pangolin</Name>
<Repository>fossoriumtech/pangolin:latest</Repository>
<Registry>https://hub.docker.com/r/fossoriumtech/pangolin</Registry>
<Network>fossorial</Network>
<Shell>sh</Shell>
<Privileged>false</Privileged>
<Overview>Pangolin - Fossorial tunnel controller and dashboard</Overview>
<Category>Network:VPN</Category>
<WebUI>https://pangolin.xtrm-lab.org</WebUI>
<ExtraParams>--restart unless-stopped</ExtraParams>
<!-- Ports -->
<Config Name="Web UI" Target="3000" Default="3000" Mode="tcp" Type="Port" Display="always" Required="true">3000</Config>
<!-- Volumes -->
<Config Name="Data" Target="/app/data" Default="/mnt/user/appdata/pangolin/data" Mode="rw" Type="Path" Display="always" Required="true">/mnt/user/appdata/pangolin/data</Config>
<Config Name="Config" Target="/app/config" Default="/mnt/user/appdata/pangolin/config" Mode="rw" Type="Path" Display="always" Required="true">/mnt/user/appdata/pangolin/config</Config>
<!-- Environment -->
<Config Name="BASE_URL" Target="PANGOLIN_BASE_URL" Default="https://pangolin.xtrm-lab.org" Mode="" Type="Variable" Display="always" Required="true">https://pangolin.xtrm-lab.org</Config>
<Config Name="SECRET_KEY" Target="PANGOLIN_SECRET_KEY" Default="" Mode="" Type="Variable" Display="always" Required="true" Mask="true">GENERATE_A_SECURE_32_CHAR_KEY</Config>
<!-- Traefik Labels -->
<Config Name="traefik.enable" Target="traefik.enable" Type="Label" Display="always">true</Config>
<Config Name="traefik.http.routers.pangolin.rule" Target="traefik.http.routers.pangolin.rule" Type="Label" Display="always">Host(`pangolin.xtrm-lab.org`)</Config>
<Config Name="traefik.http.routers.pangolin.entrypoints" Target="traefik.http.routers.pangolin.entrypoints" Type="Label" Display="always">https</Config>
<Config Name="traefik.http.routers.pangolin.tls.certresolver" Target="traefik.http.routers.pangolin.tls.certresolver" Type="Label" Display="always">cloudflare</Config>
<Config Name="traefik.http.routers.pangolin.middlewares" Target="traefik.http.routers.pangolin.middlewares" Type="Label" Display="always">default-headers@file</Config>
<Config Name="traefik.http.services.pangolin.loadbalancer.server.port" Target="traefik.http.services.pangolin.loadbalancer.server.port" Type="Label" Display="always">3000</Config>
<Config Name="traefik.docker.network" Target="traefik.docker.network" Type="Label" Display="always">dockerproxy</Config>
<!-- AutoKuma -->
<Config Name="kuma" Target="kuma" Type="Label" Display="advanced">https://pangolin.xtrm-lab.org</Config>
<Config Name="kuma.name" Target="kuma.name" Type="Label" Display="advanced">Pangolin Dashboard</Config>
<Config Name="kuma.type" Target="kuma.type" Type="Label" Display="advanced">http</Config>
<!-- Tailscale (optional) -->
<Config Name="TailScale Fallback State Directory" Target="CA_TS_FALLBACK_DIR" Type="Variable" Display="advanced">/app/data</Config>
</Container>
```
**Generate Secret Key:**
```bash
openssl rand -hex 32
```
---
### Step 2.3: Deploy Gerbil (WireGuard Manager)
**Unraid Docker Template:**
```xml
<?xml version="1.0"?>
<Container version="2">
<Name>gerbil</Name>
<Repository>fossoriumtech/gerbil:latest</Repository>
<Registry>https://hub.docker.com/r/fossoriumtech/gerbil</Registry>
<Network>fossorial</Network>
<Shell>sh</Shell>
<Privileged>true</Privileged>
<Overview>Gerbil - Fossorial WireGuard manager</Overview>
<Category>Network:VPN</Category>
<ExtraParams>--cap-add=NET_ADMIN --cap-add=SYS_MODULE --sysctl net.ipv4.ip_forward=1 --sysctl net.ipv4.conf.all.src_valid_mark=1</ExtraParams>
<!-- Ports -->
<Config Name="WireGuard UDP" Target="51820" Default="51820" Mode="udp" Type="Port" Display="always" Required="true">51820</Config>
<Config Name="API" Target="8080" Default="8080" Mode="tcp" Type="Port" Display="always" Required="true">8080</Config>
<!-- Volumes -->
<Config Name="WireGuard Config" Target="/etc/wireguard" Default="/mnt/user/appdata/gerbil/wireguard" Mode="rw" Type="Path" Display="always" Required="true">/mnt/user/appdata/gerbil/wireguard</Config>
<Config Name="Data" Target="/app/data" Default="/mnt/user/appdata/gerbil/data" Mode="rw" Type="Path" Display="always" Required="true">/mnt/user/appdata/gerbil/data</Config>
<!-- Environment -->
<Config Name="PANGOLIN_URL" Target="GERBIL_PANGOLIN_URL" Default="http://pangolin:3000" Mode="" Type="Variable" Display="always" Required="true">http://pangolin:3000</Config>
<Config Name="PUBLIC_IP" Target="GERBIL_PUBLIC_IP" Default="" Mode="" Type="Variable" Display="always" Required="true">62.73.120.142</Config>
<Config Name="PUBLIC_PORT" Target="GERBIL_PUBLIC_PORT" Default="51820" Mode="" Type="Variable" Display="always" Required="true">51820</Config>
<Config Name="WG_INTERFACE" Target="GERBIL_WG_INTERFACE" Default="wg0" Mode="" Type="Variable" Display="always" Required="true">wg0</Config>
<Config Name="API_KEY" Target="GERBIL_API_KEY" Default="" Mode="" Type="Variable" Display="always" Required="true" Mask="true">SAME_AS_PANGOLIN_SECRET</Config>
<!-- AutoKuma -->
<Config Name="kuma" Target="kuma" Type="Label" Display="advanced">http://192.168.31.2:8080/health</Config>
<Config Name="kuma.name" Target="kuma.name" Type="Label" Display="advanced">Gerbil WireGuard</Config>
<Config Name="kuma.type" Target="kuma.type" Type="Label" Display="advanced">http</Config>
</Container>
```
---
### Step 2.4: Deploy Newt (Connector)
**Unraid Docker Template:**
```xml
<?xml version="1.0"?>
<Container version="2">
<Name>newt</Name>
<Repository>fossoriumtech/newt:latest</Repository>
<Registry>https://hub.docker.com/r/fossoriumtech/newt</Registry>
<Network>fossorial</Network>
<Shell>sh</Shell>
<Privileged>false</Privileged>
<Overview>Newt - Fossorial tunnel connector (dials out to establish tunnels)</Overview>
<Category>Network:VPN</Category>
<ExtraParams>--restart unless-stopped</ExtraParams>
<!-- Volumes -->
<Config Name="Data" Target="/app/data" Default="/mnt/user/appdata/newt/data" Mode="rw" Type="Path" Display="always" Required="true">/mnt/user/appdata/newt/data</Config>
<!-- Environment -->
<Config Name="PANGOLIN_URL" Target="NEWT_PANGOLIN_URL" Default="" Mode="" Type="Variable" Display="always" Required="true">https://pangolin.xtrm-lab.org</Config>
<Config Name="ENDPOINT" Target="NEWT_ENDPOINT" Default="" Mode="" Type="Variable" Display="always" Required="true">62.73.120.142:51820</Config>
<Config Name="API_KEY" Target="NEWT_API_KEY" Default="" Mode="" Type="Variable" Display="always" Required="true" Mask="true">GENERATE_VIA_PANGOLIN_UI</Config>
<Config Name="TUNNEL_NAME" Target="NEWT_TUNNEL_NAME" Default="unraid-local" Mode="" Type="Variable" Display="always" Required="true">unraid-local</Config>
<!-- AutoKuma -->
<Config Name="kuma" Target="kuma" Type="Label" Display="advanced">docker</Config>
<Config Name="kuma.name" Target="kuma.name" Type="Label" Display="advanced">Newt Connector</Config>
<Config Name="kuma.type" Target="kuma.type" Type="Label" Display="advanced">docker</Config>
</Container>
```
---
### Step 2.5: MikroTik Port Forward for WireGuard
**Add NAT rule for Fossorial WireGuard:**
```routeros
# Connect via SSH
ssh -i /root/.ssh/mikrotik_key -p 2222 unraid@192.168.31.1
# Add port forward
/ip/firewall/nat add chain=dstnat \
action=dst-nat \
to-addresses=192.168.31.2 \
to-ports=51820 \
protocol=udp \
dst-address=62.73.120.142 \
dst-port=51820 \
comment="Fossorial WireGuard"
# Add firewall rule to allow
/ip/firewall/filter add chain=forward \
action=accept \
protocol=udp \
dst-address=192.168.31.2 \
dst-port=51820 \
comment="Allow Fossorial WireGuard" \
place-before=14
```
**Verification:**
```routeros
/ip/firewall/nat print where comment~"Fossorial"
```
---
### Step 2.6: Connect Networks (fossorial ↔ dockerproxy)
Pangolin needs to be accessible via Traefik. Either:
**Option A: Connect Pangolin to both networks**
```bash
docker network connect dockerproxy pangolin
```
**Option B: Use Traefik external routing in dynamic.yml**
```yaml
# Add to /mnt/user/appdata/traefik/dynamic.yml
http:
routers:
pangolin-secure:
rule: "Host(`pangolin.xtrm-lab.org`)"
entryPoints:
- https
middlewares:
- default-headers
tls:
certResolver: cloudflare
service: pangolin
services:
pangolin:
loadBalancer:
servers:
- url: "http://192.168.31.2:3000"
```
---
### Step 2.7: MikroTik Container Fallback (Optional)
Deploy a lightweight Gerbil instance on MikroTik for resilience:
**Prerequisites:**
- USB storage connected to MikroTik (already present: `usb1`)
- Container mode enabled
**MikroTik Commands:**
```routeros
# Create container for Gerbil fallback
/container/config set registry-url=https://registry-1.docker.io tmpdir=usb1/tmp
# Pull gerbil image
/container add \
remote-image=fossoriumtech/gerbil:latest \
interface=docker-bridge \
root-dir=usb1/gerbil \
start-on-boot=yes \
comment="Fossorial Gerbil Fallback"
# Configure environment
/container/envs add name=gerbil-env key=GERBIL_PUBLIC_IP value="62.73.120.142"
/container/envs add name=gerbil-env key=GERBIL_PUBLIC_PORT value="51821"
/container/envs add name=gerbil-env key=GERBIL_MODE value="standalone"
```
**Note:** MikroTik containers have limited resources. This is a fallback for critical services only (Pi-hole access, Authentik).
---
## Service Interruption Assessment
| Action | Risk | Impact | Mitigation |
|--------|------|--------|------------|
| Deploy Pangolin/Gerbil/Newt | NONE | New containers | - |
| Port forward 51820 | LOW | New port, existing WG on 59188 unaffected | - |
| Connect fossorial network | LOW | Container networking | Test connectivity |
| MikroTik container | MEDIUM | Router resources | Monitor CPU/memory |
**Existing WireGuard (back-to-home-vpn) Impact:** NONE
- Uses port 59188, not 51820
- Completely separate interface
---
## Verification Checklist
- [ ] All three containers running: `docker ps | grep -E "pangolin|gerbil|newt"`
- [ ] Pangolin web UI accessible: https://pangolin.xtrm-lab.org
- [ ] Gerbil API responding: `curl http://192.168.31.2:8080/health`
- [ ] MikroTik NAT rule in place: `ssh ... "/ip/firewall/nat print"`
- [ ] External WireGuard test: Connect from external network to 62.73.120.142:51820
- [ ] Newt connected in Pangolin dashboard
---
## Initial Pangolin Setup
1. Navigate to https://pangolin.xtrm-lab.org
2. Create admin account
3. Add Gerbil node:
- Name: `unraid-gerbil`
- API URL: `http://gerbil:8080`
- API Key: (same as GERBIL_API_KEY)
4. Create a tunnel:
- Name: `home-services`
- Assign to Gerbil node
5. Generate Newt API key in Pangolin UI
6. Update Newt container with the API key
---
## Rollback Procedure
1. **Stop containers:**
```bash
docker stop newt gerbil pangolin
docker rm newt gerbil pangolin
```
2. **Remove MikroTik NAT:**
```routeros
/ip/firewall/nat remove [find comment="Fossorial WireGuard"]
```
3. **Remove network:**
```bash
docker network rm fossorial
```
4. **Clean up data (if desired):**
```bash
rm -rf /mnt/user/appdata/pangolin /mnt/user/appdata/gerbil /mnt/user/appdata/newt
```
---
## Files Modified
| File/System | Change | Backup Required |
|-------------|--------|-----------------|
| MikroTik NAT | Add UDP 51820 forward | N/A (can remove) |
| /mnt/user/appdata/traefik/dynamic.yml | Add pangolin route | YES |
| New directories created | /mnt/user/appdata/pangolin,gerbil,newt | N/A |
---
## Dependencies for Next Phase
Phase 3 (Authentik) can now use Fossorial tunnels to:
- Expose Authentik externally without Cloudflare dependency
- Create secure tunnels for mobile OIDC authentication

196
docs/03-PHASE3-AUTHENTIK-ZEROTRUST.md Normal file
View File

@@ -0,0 +1,196 @@
# Phase 3: Identity & Zero Trust (Authentik)
## Status: ✅ COMPLETED
**Last Verified:** 2026-01-18
---
## Goal
Gate every `*.xtrm-lab.org` service behind OIDC authentication using Authentik, implementing Zero Trust access control.
---
## Current Authentik State
| Parameter | Value |
|-----------|-------|
| Container | authentik (+ authentik-worker) |
| Version | 2025.8.1 |
| Network | dockerproxy |
| Ports | 9000 (HTTP), 9443 (HTTPS) |
| URL | https://auth.xtrm-lab.org |
| PostgreSQL | postgresql17 (authentik_db) |
| Redis | redis |
| Status | ✅ Running (healthy) |
---
## Verified Configuration
### Users
| Username | Name | Status |
|----------|------|--------|
| akadmin | authentik Default Admin | Active |
| admin | Admin User | Active |
| jazzymc | Kaloyan Danchev | Active |
### Groups
| Group Name | Purpose |
|------------|---------|
| authentik Admins | Administrative access |
| authentik Read-only | Read-only access |
### Outpost
| Name | Type | Status |
|------|------|--------|
| authentik Embedded Outpost | proxy | ✅ Running |
### Applications
| Application | Slug |
|-------------|------|
| XTRM-Lab Protected Services | xtrm-lab-protected |
| Actual Budget | actual-budget |
### Proxy Provider
| External Host | Mode |
|---------------|------|
| https://auth.xtrm-lab.org | forward_domain |
### 2FA Status
| Type | Count |
|------|-------|
| TOTP Devices | 2 |
| WebAuthn Devices | 0 |
---
## Services Protected by Authentik Forward Auth
The following services require Authentik authentication:
| Service | Domain |
|---------|--------|
| n8n | n8n.xtrm-lab.org |
| Traefik Dashboard | traefik.xtrm-lab.org |
| NetAlertX | netalert.xtrm-lab.org |
| UrBackup | urbackup.xtrm-lab.org |
| Pi-hole 1 | ph1.xtrm-lab.org |
| Pi-hole 2 | ph2.xtrm-lab.org |
| Unimus | unimus.xtrm-lab.org |
| Homarr | xtrm-lab.org |
| Uptime Kuma | uptime.xtrm-lab.org |
| Transmission | transmission.xtrm-lab.org |
**Total: 12 protected routes** (including root redirects for Pi-holes)
---
## Services WITHOUT Authentik Protection
These services have their own authentication or are public:
| Service | Domain | Reason |
|---------|--------|--------|
| Authentik | auth.xtrm-lab.org | Self (would cause redirect loop) |
| Plex | plex.xtrm-lab.org | Has own Plex authentication |
| Vaultwarden | vault.xtrm-lab.org | Has own authentication |
| Home Assistant | ha.xtrm-lab.org | Has own authentication |
| Karakeep | karakeep.xtrm-lab.org | Public/own auth |
| RustFS CDN | cdn.xtrm-lab.org | Public CDN (S3 auth) |
| Pangolin API | pangolin.xtrm-lab.org | API access |
| Nextcloud | nextcloud.xtrm-lab.org | Has own authentication |
---
## Traefik Forward Auth Middleware
Configured in `/mnt/user/appdata/traefik/dynamic.yml`:
```yaml
authentik-forward-auth:
forwardAuth:
address: "http://authentik:9000/outpost.goauthentik.io/auth/traefik"
trustForwardHeader: true
authResponseHeaders:
- X-authentik-username
- X-authentik-groups
- X-authentik-email
- X-authentik-name
- X-authentik-uid
```
---
## Verification Checklist
- [x] Authentik initial setup completed (admin password set)
- [x] Outpost running and connected (embedded outpost)
- [x] User groups created (authentik Admins, authentik Read-only)
- [x] Application/provider pairs configured (2 applications)
- [x] Traefik config updated with forward auth middleware
- [x] Services tested successfully (302 redirect to login)
- [x] All planned services protected (12 routes)
- [x] 2FA enabled for admin accounts (2 TOTP devices)
---
## Architecture
```
Internet User
│
┌────────────▼────────────┐
│ Traefik (Reverse Proxy)│
│ *.xtrm-lab.org:443 │
└────────────┬────────────┘
│
┌────────────▼────────────┐
│ Forward Auth Check │
│ → Authentik Outpost │
└────────────┬────────────┘
│
┌──────────────────┴──────────────────┐
│ │
┌─────────▼─────────┐ ┌─────────▼─────────┐
│ Authenticated? │ │ Login Required │
│ YES → Pass │ │ Redirect to │
│ through to │ │ auth.xtrm-lab.org│
│ backend service │ └───────────────────┘
└───────────────────┘
```
---
## Maintenance Notes
### Database
- PostgreSQL database: `authentik_db`
- User: `authentik_user`
- Host: `postgresql17` container
- Data path: `/mnt/user/appdata/postgresql`
### Backup Recommendation
Regularly backup:
- PostgreSQL database (contains all Authentik config)
- `/mnt/user/appdata/traefik/dynamic.yml`
### Rollback Procedure
**Remove all protection (emergency):**
1. Edit `/mnt/user/appdata/traefik/dynamic.yml`
2. Remove `authentik-forward-auth` from all router middlewares
3. Traefik will auto-reload
---
## Related Documents
- [00-CURRENT-STATE.md](./00-CURRENT-STATE.md) - Infrastructure overview
- [02-PHASE2-FOSSORIAL-STACK.md](./02-PHASE2-FOSSORIAL-STACK.md) - Pangolin integration

465
docs/04-PHASE4-REMOTE-GAMING.md Normal file
View File

@@ -0,0 +1,465 @@
# Phase 4: Remote Gaming (Sunshine + Moonlight)
## Goal
Enable low-latency 60FPS game streaming from Nobara Linux (AMD GPU) to MacBook and Android devices, using Tailscale for optimal network pathing.
---
## Prerequisites
- **Gaming PC:** Nobara Linux with AMD GPU (VA-API support)
- **Tailscale:** Installed on gaming PC (from Phase 1)
- **Clients:** MacBook and Android devices with Tailscale
---
## Architecture Overview
```
┌─────────────────────────────────┐
│ Tailscale Mesh Network │
│ (Encrypted, P2P when possible) │
└─────────────────┬───────────────┘
│
┌────────────────────────────┼────────────────────────────┐
│ │ │
▼ ▼ ▼
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Nobara Gaming PC│ │ MacBook │ │ Android Device │
│ Sunshine Host │ │ Moonlight Client│ │ Moonlight Client│
│ 100.x.x.x (TS) │ │ 100.x.x.x (TS) │ │ 100.x.x.x (TS) │
│ AMD VA-API │ │ │ │ │
└─────────────────┘ └─────────────────┘ └─────────────────┘
```
**Why Tailscale for Gaming?**
- Automatic P2P connection when on same LAN
- Encrypted tunnel when remote
- No manual port forwarding required
- MagicDNS for easy hostname resolution
- NAT traversal handled automatically
---
## Implementation Steps
### Step 4.1: Install Sunshine on Nobara
**Method A: Flatpak (Recommended for Nobara)**
```bash
# Install via Flatpak
flatpak install flathub dev.lizardbyte.app.Sunshine
# Enable autostart
flatpak run --command=sunshine dev.lizardbyte.app.Sunshine &
```
**Method B: Native Package**
```bash
# Add Sunshine repository
sudo dnf copr enable lizardbyte/stable
sudo dnf install sunshine
# Enable service
sudo systemctl enable --now sunshine
```
**Post-Install:**
1. Access Sunshine web UI: `https://localhost:47990`
2. Set initial admin password
3. Complete setup wizard
---
### Step 4.2: Configure AMD VA-API Hardware Encoding
**Verify AMD GPU:**
```bash
# Check for AMD GPU
lspci | grep -i vga
# Verify VA-API support
vainfo
```
**Expected output:**
```
vainfo: VA-API version: 1.x
vainfo: Driver version: Mesa Gallium driver ...
vainfo: Supported profile and entrypoints:
VAProfileH264Main : VAEntrypointEncSlice
VAProfileHEVCMain : VAEntrypointEncSlice
```
**Configure Sunshine for VA-API:**
1. Open Sunshine Web UI → Configuration → Video
2. Set:
- **Encoder:** `vaapi`
- **Adapter:** `/dev/dri/renderD128` (default AMD)
- **Codec:** `H.265/HEVC` (better compression, lower latency)
**Sunshine config file** (`~/.config/sunshine/sunshine.conf`):
```ini
[video]
encoder = vaapi
adapter_name = /dev/dri/renderD128
hevc_mode = 2 # Always use HEVC when client supports
[stream]
fps = [30, 60, 120]
resolutions = [
1920x1080,
2560x1440,
3840x2160
]
```
---
### Step 4.3: Add Applications to Sunshine
**Access:** Sunshine Web UI → Applications
**Recommended Applications:**
| Name | Command | Working Directory |
|------|---------|-------------------|
| Desktop | - | - |
| Steam Big Picture | `steam -bigpicture` | - |
| Lutris | `lutris` | - |
| Heroic Games | `heroic` | - |
**For specific games:**
```
Name: Cyberpunk 2077
Command: steam steam://rungameid/1091500
Detached: ["steam"]
```
---
### Step 4.4: Install Tailscale on Nobara
```bash
# Install Tailscale
curl -fsSL https://tailscale.com/install.sh | sh
# Authenticate
sudo tailscale up
# Get Tailscale IP
tailscale ip -4
```
**Note the Tailscale IP** (e.g., `100.64.x.x`) - clients will connect to this.
---
### Step 4.5: Configure Sunshine for Tailscale
**Sunshine listens on all interfaces by default.** For security, restrict to Tailscale:
**Option A: Bind to Tailscale interface only**
```ini
# ~/.config/sunshine/sunshine.conf
[network]
address_family = both
origin_address = 100.64.x.x # Your Tailscale IP
```
**Option B: Firewall rules (recommended)**
```bash
# Allow Sunshine ports only from Tailscale
sudo firewall-cmd --permanent --zone=trusted --add-source=100.64.0.0/10
sudo firewall-cmd --permanent --zone=trusted --add-port=47984-48010/tcp
sudo firewall-cmd --permanent --zone=trusted --add-port=47998-48010/udp
sudo firewall-cmd --permanent --zone=trusted --add-port=47989/tcp # Web UI
sudo firewall-cmd --permanent --zone=trusted --add-port=47990/tcp # Web UI HTTPS
sudo firewall-cmd --reload
```
---
### Step 4.6: Install Moonlight Clients
#### MacBook
```bash
# Homebrew
brew install --cask moonlight
# Or download from: https://moonlight-stream.org/
```
#### Android
- Install "Moonlight Game Streaming" from Google Play Store
---
### Step 4.7: Pair Moonlight with Sunshine
1. **On Moonlight client:**
- Add PC manually: Enter Tailscale IP (e.g., `100.64.x.x`)
- Or use MagicDNS hostname: `nobara-pc` (if enabled in Tailscale)
2. **Enter PIN:**
- Moonlight displays a 4-digit PIN
- Enter in Sunshine Web UI → PIN Pairing
3. **Verify connection:**
- Moonlight should show your configured applications
---
### Step 4.8: MikroTik QoS for Gaming (Optional but Recommended)
**Goal:** Prioritize Nobara PC traffic to prevent bufferbloat during gaming sessions.
**SSH to MikroTik:**
```bash
ssh -i /root/.ssh/mikrotik_key -p 2222 unraid@192.168.31.1
```
**Create Simple Queue for Gaming PC:**
```routeros
# First, find Nobara's IP (replace with actual)
# Assuming Nobara is at 192.168.31.50
# Create queue for gaming priority
/queue simple add \
name="Gaming-Priority" \
target=192.168.31.50 \
max-limit=0/0 \
priority=1/1 \
queue=default-small/default-small \
comment="Nobara Gaming PC Priority"
# Alternative: Use queue tree for more control
/queue tree add \
name="Gaming-Upload" \
parent=global \
packet-mark=gaming-upload \
priority=1 \
max-limit=50M
/queue tree add \
name="Gaming-Download" \
parent=global \
packet-mark=gaming-download \
priority=1 \
max-limit=100M
# Mark gaming traffic
/ip firewall mangle add \
chain=prerouting \
src-address=192.168.31.50 \
action=mark-packet \
new-packet-mark=gaming-upload \
passthrough=yes
/ip firewall mangle add \
chain=postrouting \
dst-address=192.168.31.50 \
action=mark-packet \
new-packet-mark=gaming-download \
passthrough=yes
```
---
### Step 4.9: Optimize Streaming Settings
**Sunshine Settings (Server):**
| Setting | LAN Value | Remote Value |
|---------|-----------|--------------|
| Bitrate | 50-80 Mbps | 20-40 Mbps |
| FPS | 60-120 | 60 |
| Resolution | Native | 1080p |
| Codec | HEVC | HEVC |
**Moonlight Settings (Client):**
| Setting | LAN Value | Remote Value |
|---------|-----------|--------------|
| Video Codec | HEVC (if supported) | HEVC |
| Frame Pacing | V-Sync | On |
| Bitrate | Auto or 50+ Mbps | 20 Mbps |
| Resolution | Match display | 1080p |
---
## Network Path Analysis
**Tailscale P2P (Same Network):**
```
MacBook → Router → Nobara PC
Latency: <1ms additional overhead
```
**Tailscale Relayed (Different Network):**
```
MacBook → Tailscale DERP → Nobara PC
Latency: ~20-50ms additional overhead
```
**With Tailscale Direct (NAT Traversal Success):**
```
MacBook (Office) → Internet → Home Router → Nobara PC
Latency: RTT/2 + encoding latency (~30-80ms typical)
```
---
## Service Interruption Assessment
| Action | Risk | Impact | Mitigation |
|--------|------|--------|------------|
| Install Sunshine | NONE | Nobara only | - |
| Install Tailscale | NONE | Nobara only | - |
| MikroTik QoS | LOW | May affect other traffic briefly | Test during low usage |
| Firewall rules | LOW | Nobara only | Can revert |
---
## Verification Checklist
- [ ] Sunshine installed and running on Nobara
- [ ] VA-API encoding verified: `vainfo` shows HEVC support
- [ ] Tailscale running on Nobara: `tailscale status`
- [ ] Sunshine Web UI accessible: `https://<tailscale-ip>:47990`
- [ ] Moonlight paired successfully
- [ ] Desktop streaming works (low latency)
- [ ] Game streaming works at 60 FPS
- [ ] Remote streaming works (via Tailscale from external network)
- [ ] MikroTik QoS queue active (optional)
---
## Troubleshooting
### High Latency / Stuttering
1. **Check Tailscale connection type:**
```bash
tailscale status
# Look for "direct" vs "relay"
```
2. **Force direct connection:**
```bash
tailscale ping <client-hostname>
```
3. **Lower bitrate in Moonlight**
### Encoding Errors
1. **Verify VA-API:**
```bash
sudo vainfo
# Should show HEVC/H264 encode support
```
2. **Check Sunshine logs:**
```bash
journalctl -u sunshine -f
# Or: ~/.config/sunshine/sunshine.log
```
3. **Fall back to software encoding:**
```ini
# sunshine.conf
encoder = software
```
### No Audio
1. **Check PulseAudio/PipeWire:**
```bash
pactl list sinks
```
2. **Set Sunshine audio sink:**
- Web UI → Audio → Select correct output
---
## Security Considerations
1. **Tailscale-only access:** Sunshine is only reachable via Tailscale network
2. **PIN pairing:** Each client must be manually paired
3. **Web UI protection:** Consider adding Authentik forward auth (optional)
4. **Firewall:** Block Sunshine ports from non-Tailscale interfaces
---
## Optional: Expose Sunshine Web UI via Traefik
If you want to manage Sunshine remotely via browser:
**Add to Traefik dynamic.yml:**
```yaml
http:
routers:
sunshine-secure:
rule: "Host(`sunshine.xtrm-lab.org`)"
entryPoints:
- https
middlewares:
- default-headers
- authentik-forward-auth # Protect with Authentik
tls:
certResolver: cloudflare
service: sunshine
services:
sunshine:
loadBalancer:
servers:
- url: "https://192.168.31.50:47990" # Nobara IP
```
**MikroTik hairpin NAT (if needed):**
```routeros
# Only if accessing from LAN via external hostname
/ip/firewall/nat add chain=srcnat \
action=masquerade \
src-address=192.168.31.0/24 \
dst-address=192.168.31.50 \
dst-port=47990 \
protocol=tcp
```
---
## Files Modified
| File/System | Change | Backup Required |
|-------------|--------|-----------------|
| Nobara: ~/.config/sunshine/ | Sunshine config | No (new install) |
| Nobara: firewalld | Allow Tailscale ports | Can revert |
| MikroTik: Queue | Gaming priority | N/A |
| Traefik dynamic.yml (optional) | Sunshine route | YES |
---
## Performance Expectations
| Scenario | Expected Latency | FPS |
|----------|------------------|-----|
| LAN (same network) | 5-15ms | 60-120 |
| Remote (Tailscale direct) | 30-60ms | 60 |
| Remote (Tailscale relay) | 50-100ms | 60 |
---
## Dependencies for Next Phase
Phase 5 (RustDesk) provides:
- Alternative remote access when gaming not required
- Lower resource usage for general desktop access
- Additional fallback for remote management

187
docs/05-PHASE5-RUSTDESK.md Normal file
View File

@@ -0,0 +1,187 @@
# Phase 5: Hardened RustDesk Self-Hosted Setup
## Status: ✅ SERVER-SIDE COMPLETE
**Last Verified:** 2026-01-18
---
## Goal
Deploy a high-security, self-hosted RustDesk infrastructure with custom ID server, relay server, and end-to-end encryption using your own keypair.
---
## Current State
### Server Components
| Component | Container | Status | Ports |
|-----------|-----------|--------|-------|
| ID Server | rustdesk-hbbs | ✅ Running | TCP 21115-21116, UDP 21116, WS 21118-21119 |
| Relay Server | rustdesk-hbbr | ✅ Running | TCP 21117 |
### Configuration
| Parameter | Value |
|-----------|-------|
| Public Key | `+Xlxh96tqwh9tD58ctOmB05Qpfs0ByCoLQcF+yCw0J8=` |
| ID Server | rustdesk.xtrm-lab.org:21116 |
| Relay Server | rustdesk.xtrm-lab.org:21117 |
| DNS | rustdesk.xtrm-lab.org → 62.73.120.142 |
| Data Path | /mnt/user/appdata/rustdesk-server |
### MikroTik NAT Rules
| Rule | Protocol | WAN Port | Destination |
|------|----------|----------|-------------|
| RustDesk NAT Test | TCP | 21115 | 192.168.31.2:21115 |
| RustDesk ID Server | TCP | 21116 | 192.168.31.2:21116 |
| RustDesk ID Server | UDP | 21116 | 192.168.31.2:21116 |
| RustDesk Relay | TCP | 21117 | 192.168.31.2:21117 |
### Port Connectivity (Verified)
| Port | Protocol | Status |
|------|----------|--------|
| 21116 | TCP | ✅ Accessible |
| 21117 | TCP | ✅ Accessible |
---
## Client Configuration
To connect RustDesk clients to your self-hosted server:
### Settings
```
ID Server: rustdesk.xtrm-lab.org
Relay Server: rustdesk.xtrm-lab.org
Key: +Xlxh96tqwh9tD58ctOmB05Qpfs0ByCoLQcF+yCw0J8=
```
### Connection String (for quick setup)
```
rustdesk.xtrm-lab.org,+Xlxh96tqwh9tD58ctOmB05Qpfs0ByCoLQcF+yCw0J8=
```
---
## Verification Checklist
### Server-Side (Complete)
- [x] Keypair generated: `/mnt/user/appdata/rustdesk-server/id_ed25519*`
- [x] hbbs container running
- [x] hbbr container running
- [x] MikroTik NAT rules configured (4 rules)
- [x] DNS resolves: rustdesk.xtrm-lab.org → 62.73.120.142
- [x] Port 21116 accessible from external
- [x] Port 21117 accessible from external
### Client-Side (Pending User Testing)
- [ ] Client connects with public key
- [ ] Remote session works between two clients
- [ ] Relay works when direct P2P fails
---
## Architecture
```
Internet
│
┌────────────▼────────────┐
│ MikroTik (62.73.120.142)│
│ NAT Rules: │
│ TCP 21115-21117 │
│ UDP 21116 │
└────────────┬────────────┘
│
┌──────────────────┼──────────────────┐
│ │ │
▼ ▼ ▼
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ hbbs (ID Server)│ │ hbbr (Relay) │ │ RustDesk Client │
│ TCP 21115-21116 │ │ TCP 21117 │ │ Your devices │
│ UDP 21116 │ │ │ │ │
│ WS 21118-21119 │ │ │ │ │
└─────────────────┘ └─────────────────┘ └─────────────────┘
```
---
## Container Details
### hbbs (ID/Rendezvous Server)
```
Image: rustdesk/rustdesk-server:latest
Command: hbbs -r rustdesk.xtrm-lab.org:21117 -k _
Volume: /mnt/user/appdata/rustdesk-server:/root
Ports: 21115, 21116 (TCP+UDP), 21118, 21119
```
### hbbr (Relay Server)
```
Image: rustdesk/rustdesk-server:latest
Command: hbbr -k _
Volume: /mnt/user/appdata/rustdesk-server:/root
Ports: 21117
```
**Note:** The `-k _` flag enforces encrypted connections using the keypair.
---
## Security Features
1. **End-to-End Encryption:** All connections encrypted with Ed25519 keypair
2. **Key Verification:** Clients must have correct public key to connect
3. **Self-Hosted:** No third-party servers involved
4. **Encrypted-Only Mode:** Unencrypted connections rejected
---
## Maintenance
### View Logs
```bash
docker logs rustdesk-hbbs --tail 50
docker logs rustdesk-hbbr --tail 50
```
### Restart Services
```bash
docker restart rustdesk-hbbs rustdesk-hbbr
```
### Key Rotation
```bash
# Generate new keypair
docker run --rm -v /mnt/user/appdata/rustdesk-server:/data rustdesk/rustdesk-server hbbs -g
# Restart containers
docker restart rustdesk-hbbs rustdesk-hbbr
# Update all clients with new public key
```
---
## Rollback Procedure
```bash
# Stop and remove containers
docker stop rustdesk-hbbs rustdesk-hbbr
docker rm rustdesk-hbbs rustdesk-hbbr
# Remove MikroTik NAT rules (via SSH)
/ip/firewall/nat remove [find comment~RustDesk]
```
---
## Related Documents
- [00-CURRENT-STATE.md](./00-CURRENT-STATE.md) - Infrastructure overview
- [04-PHASE4-REMOTE-GAMING.md](./04-PHASE4-REMOTE-GAMING.md) - Sunshine/Moonlight setup

39
docs/06-CHANGELOG.md Normal file
View File

@@ -0,0 +1,39 @@
# Changelog
## 2026-01-18
- [PHASE 1] DNS Portability - COMPLETED
- Added DoH route to Traefik dynamic.yml (doh.xtrm-lab.org)
- Verified DoH endpoint working with ad-blocking
- Updated verification checklist - all items complete
- Fixed hostname in docs: dns.xtrm-lab.org → doh.xtrm-lab.org
- Updated nebula-sync status: unhealthy → healthy
- [SERVICE] DoH-Server: Now routed via Traefik at doh.xtrm-lab.org
- [SERVICE] stunnel-dot: Confirmed running for DoT on port 853
- [ISSUE] Certificate renewal failing - Cloudflare API token needs Zone:DNS:Edit permission (certs expire Feb 11, 2026)
- [PHASE 1] DNS Redundancy verified:
- 2x Pi-hole: MikroTik (172.17.0.2) + Unraid (192.168.31.4)
- 2x Unbound: MikroTik (172.17.0.3) + Unraid (192.168.31.5)
- nebula-sync: Healthy, syncing every 5 minutes
- NAT rules: Properly configured for failover
- Added DNS Redundancy Architecture section to Phase 1 doc
- [DOC] Rewrote 01-PHASE1-DNS-PORTABILITY.md - removed implementation guides, kept only current state
## 2026-01-18
- [INFRA] Updated static IP proposal: dockersocket→172.18.0.2, traefik→172.18.0.3, vaultwarden→172.18.0.15
- [INFRA] Static IP assignment for critical services - COMPLETED
## 2026-01-18 (Phase 7 Deployment)
- [PHASE 7] Gitea deployed - git.xtrm-lab.org - COMPLETED
- [PHASE 7] Woodpecker CI Server deployed - ci.xtrm-lab.org - COMPLETED
- [PHASE 7] Woodpecker CI Agent deployed and connected - COMPLETED
- [SERVICE] gitea: PostgreSQL database (gitea_db) created
- [SERVICE] woodpecker-server: Port 8008, OAuth via Gitea
- [SERVICE] woodpecker-agent: Connected to server, 2 parallel workflows
- [DNS] Added git.xtrm-lab.org and ci.xtrm-lab.org A records
## 2026-01-18 (Woodpecker Update)
- [PHASE 7] Woodpecker Server updated to v3.13.0
- [PHASE 7] Woodpecker Agent updated to v3.13.0
- [SERVICE] Fixed SQLite database permissions for migration
- [CI] First pipeline test successful (infrastructure repo)

159
docs/06-PHASE6-PORTAINER-MANAGEMENT.md Normal file
View File

@@ -0,0 +1,159 @@
# Phase 6: Multi-Host Docker Management with Portainer
## Overview
**Goal:** Unified container management dashboard for Unraid Docker.
| Component | Role |
|-----------|------|
| Portainer CE | Management hub (runs on Unraid) |
| Unraid Docker | Local host via Unix socket |
> **Note:** MikroTik RouterOS containers cannot be managed via Portainer - see [Limitation](#mikrotik-limitation) section.
---
## Phase 6.1: Unraid Server Setup ✅ COMPLETED
**Goal:** Install and configure the Portainer controller.
### Tasks
- [x] Install Portainer CE container via Docker CLI
- [x] Configure container settings:
- Network Type: **Bridge**
- Port Mapping: Container **9000** → Host **9002** (changed due to Authentik conflict)
- Port Mapping: Container **9443** → Host **9444**
- Path Mappings:
- Host `/var/run/docker.sock` → Container `/var/run/docker.sock`
- Host `/mnt/user/appdata/portainer` → Container `/data`
- [x] Add Unraid labels (`net.unraid.docker.managed`, `net.unraid.docker.icon`)
- [x] Add Tailscale labels (`tailscale.expose`, `tailscale.host`, `tailscale.port`)
- [x] Start container
- [x] Initialize Portainer via web UI
### Container Configuration
```bash
docker run -d \
--name=portainer \
--restart=unless-stopped \
-p 9002:9000 \
-p 9444:9443 \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /mnt/user/appdata/portainer:/data \
--label 'net.unraid.docker.managed=dockerman' \
--label 'net.unraid.docker.icon=https://raw.githubusercontent.com/lllllllillllllillll/Dashboard-Icons/main/png/portainer.png' \
--label 'net.unraid.docker.webui=http://100.100.208.70:9002' \
--label 'tailscale.expose=true' \
--label 'tailscale.host=100.100.208.70' \
--label 'tailscale.port=9002' \
portainer/portainer-ce:latest
```
### Access URLs
- LAN: `http://192.168.31.2:9002`
- Tailscale: `http://100.100.208.70:9002`
- HTTPS LAN: `https://192.168.31.2:9444`
- HTTPS Tailscale: `https://100.100.208.70:9444`
### Verification
- [x] Portainer container running
- [x] Portainer UI accessible
- [x] Local Unraid environment connected
---
## Phase 6.2 & 6.3: MikroTik Integration ❌ NOT FEASIBLE
### MikroTik Limitation
**MikroTik RouterOS does not use Docker.** It has its own proprietary container runtime that:
- Does NOT have a Docker daemon
- Does NOT expose `/var/run/docker.sock`
- Does NOT support Docker API
- Can ONLY be managed via RouterOS CLI/API
### What Was Attempted
1. Created veth interface (`veth-socat` at 172.17.0.5)
2. Added bridge port to `docker-bridge`
3. Created mount for `/var/run/docker.sock`
4. Deployed `alpine/socat` container
5. Added firewall and NAT rules
### Why It Failed
```
socat[2] E connect(, AF=1 "/var/run/docker.sock", 22): No such file or directory
```
The socket doesn't exist because MikroTik's container system is not Docker-based.
### Cleanup Performed
All MikroTik changes were reverted:
- Removed socat container
- Removed veth-socat interface
- Removed docker_sock mount
- Removed firewall/NAT rules
---
## MikroTik Container Management Alternatives
Since Portainer cannot connect to MikroTik, use these methods instead:
### 1. RouterOS CLI (SSH)
```bash
# From Unraid
ssh -i /root/.ssh/mikrotik_key -p 2222 unraid@192.168.31.1
# List containers
/container/print
# Start/stop containers
/container/start 0
/container/stop 0
# View logs
/log/print where topics~"container"
```
### 2. WinBox/WebFig
Access MikroTik web interface at `http://192.168.31.1` to manage containers via GUI.
### 3. RouterOS REST API
MikroTik RouterOS 7+ has a REST API that can be used for automation:
```
GET https://192.168.31.1/rest/container
```
---
## Current Status Summary
| Component | Status | Access |
|-----------|--------|--------|
| Portainer (Unraid) | ✅ Running | http://100.100.208.70:9002 |
| Unraid Docker | ✅ Connected | Via Portainer |
| MikroTik Containers | ⚠️ Separate | Via RouterOS CLI |
---
## Rollback Plan
If Portainer issues occur:
```bash
docker stop portainer && docker rm portainer
```
---
## Related Documents
- [00-CURRENT-STATE.md](./00-CURRENT-STATE.md) - Infrastructure overview

580
docs/07-CHANGELOG.md Normal file
View File

@@ -0,0 +1,580 @@
# Infrastructure Changelog
## 2026-01-18
- [INFRA] Added pending task: Static IP assignment for critical services on dockerproxy and bridge networks
- [SERVICE] postgresql17: Recreated container (was stopped due to port conflict)
- [SERVICE] authentik + authentik-worker: Restarted after PostgreSQL fix
- [TEMPLATE] Added RustDesk container templates with icons
- [TEMPLATE] Updated Pi-hole template with proper Unraid CA metadata
Track all changes to services, configurations, and phase progress.
---
## 2026-01-17 - Homarr + Portainer Integration
### Portainer App Added to Homarr
- [SERVICE] homarr: Added Portainer app to dashboard
- Section: Monitoring
- URL: http://100.100.208.70:9002 (Tailscale)
- Ping URL: http://192.168.31.2:9002 (LAN)
### Docker Integration Added
- [SERVICE] homarr: Added Docker integration via socket
- Integration name: Docker (Unraid)
- Socket: unix:///var/run/docker.sock
- Linked to Portainer app for container status display
### Database Changes
- Added app record for Portainer
- Added item and item_layout for Monitoring section
- Added integration record for Docker
- Linked integration to Portainer item
### Access
- Homarr: https://xtrm-lab.org
- Portainer visible in Monitoring section
---
## 2026-01-17 - Phase 6.2/6.3 Cancelled: MikroTik Incompatible
### Discovery
- MikroTik RouterOS containers are NOT Docker-based
- No `/var/run/docker.sock` exists on MikroTik
- Portainer cannot connect to MikroTik's container runtime
### What Was Attempted
- Created veth-socat interface (172.17.0.5)
- Deployed alpine/socat container
- Added firewall and NAT rules for port 2375
- Socat failed: `No such file or directory` for docker.sock
### Cleanup Performed
- Removed socat container
- Removed veth-socat interface and bridge port
- Removed docker_sock mount
- Removed firewall/NAT rules for port 2375
### Conclusion
- Phase 6.2 and 6.3 are NOT FEASIBLE
- MikroTik containers must be managed via RouterOS CLI/WebFig
- Portainer remains useful for Unraid-only container management
### Status Update
- [PHASE 6.1] COMPLETED - Portainer managing Unraid
- [PHASE 6.2] CANCELLED - MikroTik incompatible
- [PHASE 6.3] CANCELLED - MikroTik incompatible
---
## 2026-01-17 - Unraid Container Labels Fixed
### Containers Updated
- [SERVICE] unbound: Added Unraid labels (`net.unraid.docker.managed`, `net.unraid.docker.icon`)
- [SERVICE] portainer: Added Unraid labels + Tailscale labels
### Portainer Labels
- `net.unraid.docker.managed=dockerman`
- `net.unraid.docker.icon` - Portainer icon
- `net.unraid.docker.webui=http://100.100.208.70:9002`
- `tailscale.expose=true`
- `tailscale.host=100.100.208.70`
- `tailscale.port=9002`
### Unbound Labels
- `net.unraid.docker.managed=dockerman`
- `net.unraid.docker.icon` - Unbound icon
### Note
Both containers recreated to apply labels. Services verified working after recreation.
---
## 2026-01-17 - Phase 6.1 Completed: Portainer CE Deployed
### Portainer CE Installation
- [PHASE 6.1] Portainer CE deployed on Unraid - COMPLETED
- Container: `portainer/portainer-ce:latest`
- HTTP Port: **9002** (changed from 9000 due to Authentik conflict)
- HTTPS Port: **9444**
- Data: `/mnt/user/appdata/portainer`
- LAN URL: `http://192.168.31.2:9002`
- Tailscale URL: `http://100.100.208.70:9002`
### Port Conflict Resolution
- Original plan: port 9000
- Conflict: Authentik already using port 9000
- Resolution: Mapped to port 9002 (HTTP) and 9444 (HTTPS)
### Next Steps
- Phase 6.2: Deploy Socat proxy on MikroTik (port 2375)
- Phase 6.3: Connect MikroTik environment to Portainer
### Status
- [PHASE 6.1] COMPLETED - Portainer running, needs initial setup via web UI
- [PHASE 6.2] NOT STARTED
- [PHASE 6.3] NOT STARTED
---
## 2026-01-17 - Phase 6 Added: Multi-Host Docker Management
### New Documentation
- [PHASE 6] Created 06-PHASE6-PORTAINER-MANAGEMENT.md
- Portainer CE deployment plan for unified Docker management
- Covers Unraid local setup and MikroTik remote API via Socat
### Phase 6 Components
- Phase 6.1: Portainer CE installation on Unraid (port 9002)
- Phase 6.2: MikroTik Socat proxy for Docker API exposure (port 2375)
- Phase 6.3: Unified dashboard connection
### Security Considerations
- MikroTik firewall rules to restrict Docker API access to Unraid only
- Unauthenticated API requires network-level security
### Status
- [PHASE 6] IN PROGRESS - Phase 6.1 completed
---
## 2026-01-17 - Status Audit
### Verified Working
- [PHASE 1] Tailscale on Unraid - WORKING (100.100.208.70)
- [PHASE 1] nebula-sync Pi-hole sync - HEALTHY (was unhealthy, now fixed)
- [PHASE 1] stunnel-dot (DoT on 853) - WORKING
- [PHASE 2] Pangolin controller - RUNNING (13 days uptime)
- [PHASE 3] Authentik main container - HEALTHY
### Issues Found
- [PHASE 1] DoH-Server - RUNNING but BROKEN (can't reach Pi-hole upstream from dockerproxy network)
- [PHASE 2] gerbil - CRASHED (Pangolin returns empty CIDR for WG config)
- [PHASE 3] authentik-worker - CRASHED (PostgreSQL DNS resolution failure)
- [PHASE 5] RustDesk - NOT DEPLOYED
### MikroTik NAT Status
- Port 853 (DoT) -> Unraid stunnel - OK
- Port 5443 (DoH) -> MikroTik Pi-hole (wrong target, should be Unraid DoH-Server)
- Port 51820 (Fossorial WG) - NOT CONFIGURED
- Ports 21115-21119 (RustDesk) - NOT CONFIGURED
---
## Template for Future Entries
## YYYY-MM-DD - Description
### Changes
- [PHASE X] description - STATUS
- [SERVICE] name: what changed
### Issues
- description of any problems found
### Notes
- any relevant context
## 2026-01-17 - DNS Infrastructure Fixes
### DoH-Server
- [PHASE 1] DoH-Server - WORKING at `doh.xtrm-lab.org` (not `dns.xtrm-lab.org` as documented)
- [ISSUE] Infrastructure docs reference `dns.xtrm-lab.org` but container uses `doh.xtrm-lab.org`
- [ACTION NEEDED] Either update docs OR add Traefik route for `dns.xtrm-lab.org`
### Unraid Unbound - FIXED
- [PHASE 1] Replaced broken klutchell/unbound with mvance/unbound:latest
- [ROOT CAUSE 1] Original image missing root.hints/root.key files (distroless image issue)
- [ROOT CAUSE 2] MikroTik NAT rules were hijacking Unbound's outbound DNS (192.168.31.0/24 -> Pi-hole)
- [ROOT CAUSE 3] IPv6 not working on br0 macvlan, causing timeout loops
### MikroTik NAT Changes
- Added rule 6: "Allow Unraid Unbound" - accept UDP from 192.168.31.5 port 53
- Added rule 8: "Allow Unraid Unbound TCP" - accept TCP from 192.168.31.5 port 53
- These rules placed BEFORE the "Force DNS to Pi-hole" rules
### Unbound Configuration
- Location: /mnt/user/appdata/unbound-mvance/
- Custom config: a-records.conf (disables IPv6, sets logging)
- Image: mvance/unbound:latest
- Network: br0 (macvlan) at 192.168.31.5
### Verified Working
- Unraid Unbound (192.168.31.5) - RESOLVED google.com, github.com, cloudflare.com
- Unraid Pi-hole upstreams: 172.17.0.3 (MikroTik Unbound) + 192.168.31.5 (Unraid Unbound)
- DoH endpoint working at doh.xtrm-lab.org
- stunnel-dot (DoT) - already working
### Still Pending
- MikroTik Pi-hole upstream config needs verification (check if it uses both Unbounds)
- Docs need update: dns.xtrm-lab.org vs doh.xtrm-lab.org
### MikroTik Pi-hole Upstreams - FIXED
- [PHASE 1] MikroTik Pi-hole was using Google DNS (8.8.8.8, 8.8.4.4) instead of local Unbounds
- Changed upstreams via Pi-hole v6 API to:
- 172.17.0.3#53 - MikroTik local Unbound
- 192.168.31.5#53 - Unraid Unbound
- DNS resolution tested and working
### Full DNS Redundancy Now Achieved
- Unraid Pi-hole upstreams: 172.17.0.3, 192.168.31.5
- MikroTik Pi-hole upstreams: 172.17.0.3, 192.168.31.5
- Both Unbounds working as recursive resolvers
- nebula-sync keeps blocklists in sync between Pi-holes
---
## 2026-01-17 - Gerbil Investigation: Feature Not Available
### Issue
- Gerbil kept crashing with "invalid CIDR address" error
- Exit node was correctly configured in database
- API returned empty data despite valid configuration
### Root Cause
- **Pangolin 1.14 Community Edition does not include Exit Nodes feature**
- Exit Nodes / Gerbil functionality requires paid Pangolin license
- The API endpoint exists but returns empty data for CE users
### Resolution
- Removed gerbil container (feature not available)
- Existing MikroTik WireGuard VPN provides equivalent remote access functionality
- Phase 2 (Fossorial Stack) marked as blocked pending license upgrade
### Status
- [PHASE 2] gerbil - REMOVED (requires paid Pangolin license)
- [PHASE 2] Pangolin controller - RUNNING (limited to CE features)
---
## 2026-01-18 - Actual Budget OIDC Integration with Authentik
### Problem
- Actual Budget OIDC login failing with multiple errors
### Fixes Applied
#### 1. DNS Resolution (EAI_AGAIN)
- **Issue:** Container couldn't resolve auth.xtrm-lab.org
- **Fix:** Added `--add-host=auth.xtrm-lab.org:<traefik-ip>` to container
- **Template:** /boot/config/plugins/dockerMan/templates-user/my-actual-budget.xml
#### 2. JWT Signing Algorithm (HS256 vs RS256)
- **Issue:** Authentik signed tokens with HS256, Actual Budget expected RS256
- **Root Cause:** OAuth2 provider had no signing key configured
- **Fix:** Set signing_key_id to 'authentik Internal JWT Certificate' in database
- **SQL:** `UPDATE authentik_providers_oauth2_oauth2provider SET signing_key_id = '48203833-f562-4ec6-b782-f566e6d960d5' WHERE client_id = 'actual-budget';`
#### 3. Insufficient Scope
- **Issue:** Provider had no scope mappings assigned
- **Fix:** Added openid, email, profile scopes to provider
- **SQL:** `INSERT INTO authentik_core_provider_property_mappings (provider_id, propertymapping_id) VALUES (3, 'a24eea06-...'), (3, '4394c150-...'), (3, '7272ab52-...');`
### Traefik Static IP
- **Issue:** Traefik IP was dynamic, would break actual-budget on restart
- **Fix:** Assigned static IP 172.18.0.10 to Traefik on dockerproxy network
- **Template:** Added `--ip=172.18.0.10` to ExtraParams in my-traefik.xml
### Final Configuration
| Component | Setting |
|-----------|---------|
| Traefik | 172.18.0.10 (static) on dockerproxy |
| Actual Budget | --add-host=auth.xtrm-lab.org:172.18.0.10 |
| Authentik Provider | actual-budget with RS256 signing + scopes |
### Actual Budget OIDC Environment
```
ACTUAL_OPENID_DISCOVERY_URL=https://auth.xtrm-lab.org/application/o/actual-budget/.well-known/openid-configuration
ACTUAL_OPENID_CLIENT_ID=actual-budget
ACTUAL_OPENID_CLIENT_SECRET=<secret>
ACTUAL_OPENID_SERVER_HOSTNAME=https://actual.xtrm-lab.org
```
### Status
- [PHASE 3] Actual Budget OIDC - WORKING
- [SERVICE] traefik: Static IP 172.18.0.10 configured
- [SERVICE] actual-budget: OIDC login via Authentik working
## 2026-01-18 - Phase 5 Completed: RustDesk Self-Hosted Deployment
### Keypair Generation
- [PHASE 5] Generated Ed25519 keypair for encrypted connections
- Public Key: `+Xlxh96tqwh9tD58ctOmB05Qpfs0ByCoLQcF+yCw0J8=`
- Data directory: /mnt/user/appdata/rustdesk-server/
### Containers Deployed
- [SERVICE] rustdesk-hbbs: ID/Rendezvous server on ports 21115-21116 (TCP), 21116 (UDP), 21118-21119
- [SERVICE] rustdesk-hbbr: Relay server on port 21117
- Both containers configured with `-k _` for mandatory encryption
- AutoKuma labels added for Uptime Kuma monitoring
### MikroTik Configuration
- Added NAT rules 24-27 for RustDesk ports
- Added firewall forward rules (Allow RustDesk TCP/UDP)
- Ports forwarded: 21115 (NAT test), 21116 (TCP+UDP), 21117 (Relay)
### DNS
- rustdesk.xtrm-lab.org already resolving to 62.73.120.142 (DNS only, no proxy)
### Verification
- All TCP ports (21115, 21116, 21117) accessible externally
- Both containers running healthy
- Logs show successful startup with keypair loaded
### Client Configuration
| Setting | Value |
|---------|-------|
| ID Server | rustdesk.xtrm-lab.org |
| Relay Server | rustdesk.xtrm-lab.org |
| Public Key | +Xlxh96tqwh9tD58ctOmB05Qpfs0ByCoLQcF+yCw0J8= |
### Status
- [PHASE 5] RustDesk Self-Hosted - COMPLETED
## 2026-01-18 - Vaultwarden 502 Fix
### Issue
- Vaultwarden returning unexpected error when creating new logins
- Traefik logs showed 502 Bad Gateway errors
### Root Cause
- Traefik config pointed to `http://192.168.31.2:4743`
- Vaultwarden container had no port 4743 mapping (port 80/tcp was not published)
- Both containers on `dockerproxy` network but config used host IP
### Fix
- Updated `/mnt/user/appdata/traefik/dynamic.yml`
- Changed: `url: "http://192.168.31.2:4743"` → `url: "http://vaultwarden:80"`
- Uses Docker internal DNS which resolves to container IP on dockerproxy network
### Status
- [SERVICE] vaultwarden: Working - can create/edit logins
## 2026-01-18 - Progress Summary
### Completed Phases
- [PHASE 1] DNS Portability - COMPLETE (DoH, DoT, Unbound redundancy)
- [PHASE 5] RustDesk Self-Hosted - COMPLETE (hbbs/hbbr deployed)
- [PHASE 6] Portainer Management - COMPLETE (6.2/6.3 cancelled - MikroTik incompatible)
### In Progress
- [PHASE 3] Authentik Zero Trust - Actual Budget integrated, more services pending
### Blocked
- [PHASE 2] Fossorial Stack - Gerbil requires paid Pangolin license
### Not Started
- [PHASE 4] Remote Gaming (Sunshine/Moonlight) - Starting now
### Known Issues
- HomeAssistant_inabox: Exited (1) 3 days ago
- pgAdmin4: Exited (137) 2 weeks ago
## 2026-01-18 - Phase 4 Started: MacBook Prepared
### MacBook Setup Complete
- [PHASE 4] Moonlight v6.1.0 already installed
- [PHASE 4] Tailscale connected (100.68.118.59)
### Pending - Nobara Setup
- Install Sunshine on Nobara
- Configure VA-API encoding
- Pair with Moonlight
### Instructions Saved
- MacBook: ~/Documents/NOBARA-SUNSHINE-SETUP.md
### Status
- [PHASE 4] MacBook client ready, awaiting Nobara server setup
## 2026-01-18 - NetAlertX & Uptime Kuma Fixes (Partial)
### Uptime Kuma - FIXED
- [SERVICE] Added Traefik route for uptime.xtrm-lab.org
- Protected with Authentik forward auth
- Service URL: http://192.168.31.2:3001
### NetAlertX - IN PROGRESS
- [ISSUE] Container not scanning network - shows 0 devices
- [ROOT CAUSE] Multiple config files exist:
- /app/config/app.conf (mounted from host) - updated correctly
- /app/back/app.conf (container internal) - has old value '--localnet'
- [ATTEMPTED] Updated /mnt/user/appdata/netalertx/config/app.conf
- [ATTEMPTED] Updated database Settings table
- [ATTEMPTED] Deleted database to force reload
- [DISCOVERED] App reads from /app/back/app.conf which is generated at startup
### NetAlertX Fix Required
1. The /app/back/app.conf needs to be updated to:
SCAN_SUBNETS=['192.168.31.0/24 --interface=br0']
2. This file is regenerated on container start from /app/config/app.conf
3. May need to use Settings UI at https://netalert.xtrm-lab.org to change SCAN_SUBNETS
### Manual ARP Scan Test - WORKS
Command: docker exec NetAlertX arp-scan --localnet --interface=br0
Result: Found 20 devices on 192.168.31.0/24
### Pending Tasks
- Fix NetAlertX to use correct subnet config
- Add Tailscale network scanning (may not work - ARP doesn't work over tunnels)
- User requested: RustFS for personal CDN (assets hosting)
### Status
- [SERVICE] uptime.xtrm-lab.org - WORKING
- [SERVICE] netalertx - PARTIALLY BROKEN (config issue)
## 2026-01-18 - NetAlertX FIXED
### Resolution
- [SERVICE] NetAlertX now scanning network correctly - found 21 devices
- [FIX] Updated config in multiple locations:
- /data/config/app.conf (runtime config inside container)
- /app/back/app.conf (plugin reads from here)
- /mnt/user/appdata/netalertx/config/app.conf (host mount for persistence)
### Config Change
```
SCAN_SUBNETS=['192.168.31.0/24 --interface=br0']
```
### Root Cause Summary
- NetAlertX has complex config handling with multiple config file locations
- /app/config (mounted) -> copied to /data/config on startup
- /data/config/app.conf is read by the app
- /app/back/app.conf is read by plugins at runtime
- All three needed to be updated
### Verified Working
- ARP scan found 21 devices on 192.168.31.0/24
- Devices visible at https://netalert.xtrm-lab.org/devices.php
### Note on Tailscale Scanning
- ARP scanning does NOT work over Tailscale (point-to-point tunnel, no broadcast)
- Tailscale devices need to be added manually or via different discovery method
## 2026-01-18 - RustFS CDN Deployed
### Service Details
- [SERVICE] RustFS - S3-compatible object storage for personal CDN
- Image: rustfs/rustfs:latest
- Ports: 9010 (S3 API), 9011 (Console)
- Data: /mnt/user/appdata/rustfs/data
- Logs: /mnt/user/appdata/rustfs/logs
### Access URLs
- S3 API: https://cdn.xtrm-lab.org
- Console: http://192.168.31.2:9011/rustfs/console/
- Credentials stored in: /mnt/user/appdata/rustfs/CREDENTIALS.txt
### Traefik Route
- Host: cdn.xtrm-lab.org
- No Authentik protection (public CDN for assets)
- S3 authentication handles access control
### Usage
Create bucket and upload assets via:
- RustFS Console at port 9011
- S3-compatible CLI tools (aws-cli, rclone, etc.)
- Direct S3 API calls
### Example S3 CLI Usage
```bash
# Configure aws-cli
aws configure set aws_access_key_id <access_key>
aws configure set aws_secret_access_key <secret_key>
# Create bucket
aws --endpoint-url https://cdn.xtrm-lab.org s3 mb s3://assets
# Upload file
aws --endpoint-url https://cdn.xtrm-lab.org s3 cp image.png s3://assets/
# Public URL (after setting bucket policy)
https://cdn.xtrm-lab.org/assets/image.png
```
### Status
- [SERVICE] rustfs - RUNNING
- [PHASE N/A] Personal CDN - COMPLETED
## 2026-01-18 - PostgreSQL Data Path Restored & Phase 3 Verified
### Root Cause Analysis
- [INCIDENT] PostgreSQL container was recreated at 07:40 UTC with wrong data path
- [CAUSE] Container used default path `/mnt/user/appdata/postgresql17` instead of configured `/mnt/user/appdata/postgresql`
- [IMPACT] Authentik started with empty database, all configuration appeared lost
- [DATA] Original data was safe in `/mnt/user/appdata/postgresql/` the entire time
### Resolution
- [FIX] Stopped postgresql17 container
- [FIX] Recreated container with correct volume mount: `/mnt/user/appdata/postgresql:/var/lib/postgresql/data`
- [FIX] Restarted Authentik containers
- [VERIFIED] All Authentik data restored (users, groups, applications, providers)
### Other Fixes This Session
- [SERVICE] Uptime-Kuma-API: Added missing ADMIN_PASSWORD environment variable
- [TRAEFIK] Added Docker provider constraint to filter broken container labels
- [TRAEFIK] Added missing routes: authentik, transmission, nextcloud to dynamic.yml
### Phase 3 Verification - COMPLETED
Verified Authentik Zero Trust configuration:
- Users: akadmin, admin, jazzymc (3 active users)
- Groups: authentik Admins, authentik Read-only
- Outpost: Embedded Outpost running (proxy type)
- Applications: XTRM-Lab Protected Services, Actual Budget
- Proxy Provider: forward_domain mode for auth.xtrm-lab.org
- 2FA: 2 TOTP devices configured
- Protected Services: 12 routes using authentik-forward-auth middleware
### Services Status
- [SERVICE] auth.xtrm-lab.org - WORKING (302 redirect to login)
- [SERVICE] uptime.xtrm-lab.org - WORKING (forward auth active)
- [SERVICE] ph2.xtrm-lab.org - WORKING (forward auth active)
- [SERVICE] All forward-auth protected services - WORKING
### Documentation Updated
- [DOC] 03-PHASE3-AUTHENTIK-ZEROTRUST.md - Marked as COMPLETED with verified state
## 2026-01-18 - Phase 5 RustDesk Verified
### Server-Side Verification Complete
- [x] Keypair exists: /mnt/user/appdata/rustdesk-server/id_ed25519*
- [x] Public Key: +Xlxh96tqwh9tD58ctOmB05Qpfs0ByCoLQcF+yCw0J8=
- [x] hbbs container: Up 10+ hours
- [x] hbbr container: Up 10+ hours
- [x] MikroTik NAT: 4 rules configured (21115-21117 TCP, 21116 UDP)
- [x] DNS: rustdesk.xtrm-lab.org → 62.73.120.142
- [x] Port 21116 TCP: Externally accessible (verified via nc)
- [x] Port 21117 TCP: Externally accessible (verified via nc)
### Client Configuration
ID Server: rustdesk.xtrm-lab.org
Relay Server: rustdesk.xtrm-lab.org
Key: +Xlxh96tqwh9tD58ctOmB05Qpfs0ByCoLQcF+yCw0J8=
### Documentation Updated
- [DOC] 05-PHASE5-RUSTDESK.md - Rewritten with verified state, marked SERVER-SIDE COMPLETE
- [PENDING] Client-side testing (user to verify remote sessions work)
## 2026-01-18 - Phase 7 GitOps Plan Created
### New Phase: Gitea + Woodpecker CI
- [DOC] Created 08-PHASE7-GITEA-GITOPS.md
- [PLAN] Lightweight GitOps stack for infrastructure management
- [COMPONENTS] Gitea (~200MB) + Woodpecker Server/Agent (~200MB)
- [TOTAL RESOURCES] ~400MB RAM, ~700MB storage
### Planned Features
- Git version control for all configs
- Automated YAML validation
- CI/CD pipeline for deployments
- Auto-rollback on health check failure
- Authentik SSO integration
- Safe AI agent integration
### URLs (Planned)
- git.xtrm-lab.org - Gitea web UI
- ci.xtrm-lab.org - Woodpecker CI dashboard

168
docs/08-PHASE7-GITEA-GITOPS.md Normal file
View File

@@ -0,0 +1,168 @@
# Phase 7: Gitea + Woodpecker CI (GitOps for Homelab)
## Status: ✅ COMPLETED
**Deployed:** 2026-01-18
---
## Deployed Components
| Service | Container | Version | Port | URL | Status |
|---------|-----------|---------|------|-----|--------|
| Gitea | gitea | 1.25.3 | 3005→3000, 2222→22 | https://git.xtrm-lab.org | ✅ Running |
| Woodpecker Server | woodpecker-server | 3.13.0 | 8008→8000 | https://ci.xtrm-lab.org | ✅ Running |
| Woodpecker Agent | woodpecker-agent | 3.13.0 | - | - | ✅ Running |
---
## Configuration
### Gitea
| Parameter | Value |
|-----------|-------|
| Admin User | jazzymc |
| Database | PostgreSQL (gitea_db @ 172.18.0.13) |
| DB User | gitea |
| SSH Port | 2222 |
| Data Path | /mnt/user/appdata/gitea/data |
| Network | dockerproxy |
### Woodpecker CI
| Parameter | Value |
|-----------|-------|
| Version | 3.13.0 |
| Admin User | jazzymc (via Gitea OAuth) |
| Server IP | 172.18.0.134 |
| gRPC Port | 9000 |
| HTTP Port | 8000 (mapped to 8008) |
| Max Workflows | 2 (parallel) |
| Data Path | /mnt/user/appdata/woodpecker/server |
| Agent Secret | 564a5716400532874a8e02313a491b4f3864ce9b77a5122ce0eb14777749e740 |
### Gitea OAuth App (for Woodpecker)
| Parameter | Value |
|-----------|-------|
| Client ID | 924b3300-b607-4a48-bc26-35b06dbf18c7 |
| Redirect URI | https://ci.xtrm-lab.org/authorize |
---
## Network Configuration
All services on `dockerproxy` network:
| Service | Internal IP | DNS Name |
|---------|-------------|----------|
| Gitea | Dynamic | gitea |
| Woodpecker Server | 172.18.0.134 | woodpecker-server |
| PostgreSQL | 172.18.0.13 | postgresql17 |
### Traefik Routes
| Domain | Service | Port |
|--------|---------|------|
| git.xtrm-lab.org | gitea | 3000 |
| ci.xtrm-lab.org | woodpecker-server | 8000 |
---
## Verification Checklist
- [x] Gitea container running
- [x] Gitea accessible at https://git.xtrm-lab.org
- [x] Admin account created (jazzymc)
- [x] OAuth app created for Woodpecker
- [x] Woodpecker Server v3.13.0 running
- [x] Woodpecker Agent v3.13.0 running and connected
- [x] Woodpecker accessible at https://ci.xtrm-lab.org
- [x] Gitea OAuth login working
- [x] CI pipeline tested successfully
---
## Usage
### Git Operations
```bash
# Clone via HTTPS
git clone https://git.xtrm-lab.org/jazzymc/infrastructure.git
# Clone via SSH (port 2222)
git clone ssh://git@git.xtrm-lab.org:2222/jazzymc/infrastructure.git
```
### CI Pipeline (.woodpecker.yml)
```yaml
steps:
- name: test
image: alpine
commands:
- echo 'Hello from Woodpecker CI!'
- date
```
### Example: Node.js Pipeline
```yaml
steps:
- name: install
image: node:20
commands:
- npm install
- name: test
image: node:20
commands:
- npm test
- name: build
image: node:20
commands:
- npm run build
when:
branch: main
```
### Example: Docker Build
```yaml
steps:
- name: build
image: docker
commands:
- docker build -t myapp .
volumes:
- /var/run/docker.sock:/var/run/docker.sock
```
---
## Maintenance
### Update Woodpecker
```bash
docker pull woodpeckerci/woodpecker-server:v3
docker pull woodpeckerci/woodpecker-agent:v3
docker restart woodpecker-server woodpecker-agent
```
### Backup
Important paths to backup:
- /mnt/user/appdata/gitea/data
- /mnt/user/appdata/woodpecker/server
- PostgreSQL database: gitea_db
---
## Related Documents
- [00-CURRENT-STATE.md](./00-CURRENT-STATE.md) - Infrastructure overview
- [03-PHASE3-AUTHENTIK-ZEROTRUST.md](./03-PHASE3-AUTHENTIK-ZEROTRUST.md) - SSO setup