From 09209bf863bc5e01902434124fee833ff119b1c8 Mon Sep 17 00:00:00 2001 From: XTRM-Unraid Date: Thu, 22 Jan 2026 11:44:24 +0200 Subject: [PATCH] docs: AdGuard Home on MikroTik - complete setup - Replaced Pi-hole with AdGuard Home (172.17.0.5:5355) - Configured DoH/DoT/DoQ with TLS certificates - Added blocklists: StevenBlack, Hagezi Pro, Hagezi NSFW - Added custom rules and 6 client devices - Updated NAT rules for DNS redirect - Documented MikroTik container root-dir bug - Saved migration config for Unraid setup Co-Authored-By: Claude Opus 4.5 --- docs/00-CURRENT-STATE.md | 180 +++++++++++++-------------------------- docs/06-CHANGELOG.md | 91 ++++++++------------ 2 files changed, 93 insertions(+), 178 deletions(-) diff --git a/docs/00-CURRENT-STATE.md b/docs/00-CURRENT-STATE.md index a0e3948..c9ceda2 100644 --- a/docs/00-CURRENT-STATE.md +++ b/docs/00-CURRENT-STATE.md @@ -17,11 +17,11 @@ | WAN IP (Static) | 62.73.120.142 | | LAN Subnet | 192.168.31.0/24 | | Docker Bridge | 172.17.0.0/24 | -| SSH Access | `ssh -i /root/.ssh/mikrotik_key -p 2222 xtrm@192.168.31.1` | +| SSH Access | `ssh -i /root/.ssh/mikrotik_key -p 2222 unraid@192.168.31.1` | **SSH Users:** -- `xtrm` - Primary admin user (key-based from Unraid) -- `unraid` - Secondary admin user (key-based from Unraid) +- `xtrm` - Primary admin user (key auth issues) +- `unraid` - Secondary admin user (key-based from Unraid) ✓ Working **Interfaces:** - `ether1` - WAN (62.73.120.142/23) @@ -29,33 +29,43 @@ - `docker-bridge` - Container network (172.17.0.1/24) - `back-to-home-vpn` - WireGuard VPN (192.168.216.1/24) -**SNMP Configuration:** -| Device | Community | Access | Status | -|--------|-----------|--------|--------| -| hAP ax³ | `netdisco` | 192.168.31.2 only | Enabled | -| CSS326 | `public` | Any (SwOS limit) | Enabled | -| cAP ac | `netdisco` | 192.168.31.2 only | Enabled | - **Running Containers on MikroTik:** | Container | IP | Storage | Purpose | |-----------|-----|---------|---------| -| unbound:latest | 172.17.0.3 | usb1/unbound/root | Recursive DNS resolver | | tailscale:latest | 172.17.0.4 | usb1/tailscale/root | Tailscale VPN client | -| adguardhome:latest | 172.17.0.5 | usb1/adguardhome | DNS sinkhole with DoH/DoT/DoQ | +| adguardhome:latest | 172.17.0.5 | usb1/agh2 | DNS sinkhole with DoH/DoT/DoQ | + +**Stopped Containers:** +| Container | Issue | +|-----------|-------| +| unbound:latest | exited with status 1 | **AdGuard Home Configuration (172.17.0.5):** | Service | Port | Protocol | Status | |---------|------|----------|--------| -| DNS | 53 | UDP/TCP | Active | +| DNS | 5355 | UDP/TCP | Active (NAT from 53) | | Web UI | 80 | HTTP | Active | | DoH (DNS-over-HTTPS) | 443 | HTTPS | Active (TLS) | | DoT (DNS-over-TLS) | 853 | TCP | Active (TLS) | | DoQ (DNS-over-QUIC) | 8853 | UDP | Active (TLS) | -**TLS Certificate:** Let's Encrypt wildcard cert for `*.xtrm-lab.org` (shared from Traefik) +**AdGuard Home Blocklists:** +- StevenBlack Hosts +- Hagezi Pro +- Hagezi NSFW + +**AdGuard Home Custom Rules:** +- ||dv-eu-prod.sentinelone.net^ +- ||euce1-soc360.sentinelone.net^ +- ||ampeco.jamfcloud.com^ +- ||*.jamfcloud.com^ + +**TLS Certificate:** Let's Encrypt wildcard cert for `*.xtrm-lab.org` **Server Name:** `dns.xtrm-lab.org` **Certificate Expiry:** 2026-04-02 +**⚠️ IMPORTANT:** Do NOT stop/restart the AdGuard Home container - MikroTik has a bug where the root directory disappears when container stops. + ### MikroTik CSS326-24G-2S+ Switch (192.168.31.9) | Parameter | Value | @@ -107,76 +117,29 @@ | **Databases** | | PostgreSQL | postgresql17 | 172.18.0.13 | - | | Redis | Redis | 172.18.0.14 | - | -| **DNS** | +| **DNS (Unraid - Secondary)** | | Pi-hole (Unraid) | binhex-official-pihole | 192.168.31.4 | ph1.xtrm-lab.org | | Unbound (Unraid) | unbound | 192.168.31.5 | - | +| DoH Server | DoH-Server | 172.18.0.22 | doh.xtrm-lab.org | +| nebula-sync | nebula-sync | - | ⚠️ Crash-looping (incompatible with AdGuard) | | **DevOps** | | Git Server | gitea | 172.18.0.31 | git.xtrm-lab.org | | CI/CD Server | woodpecker-server | 172.18.0.32 | ci.xtrm-lab.org | | CI/CD Agent | woodpecker-agent | 172.18.0.33 | - | | **Network Management** | | NetBox | netbox | 172.24.0.5 | netbox.xtrm-lab.org | -| NetBox Worker | netbox-worker | 172.24.0.6 | - | -| NetBox PostgreSQL | netbox-postgres | 172.24.0.4 | - | -| NetBox Redis | netbox-redis | 172.24.0.2 | - | -| NetBox Redis Cache | netbox-redis-cache | 172.24.0.3 | - | | NetDisco Web | netdisco-web | 172.18.0.41 | netdisco.xtrm-lab.org | -| NetDisco Backend | netdisco-backend | 172.18.0.42 | - | | Unimus | unimus | host | unimus.xtrm-lab.org | -| **Slurp'it Discovery** | -| Slurp'it Portal | slurpit-portal | dockerproxy | slurpit.xtrm-lab.org | -| Slurp'it Scanner | slurpit-scanner | slurpit-network | - | -| Slurp'it Scraper | slurpit-scraper | slurpit-network | - | -| Slurp'it Warehouse | slurpit-warehouse | slurpit-network | - | -| Slurp'it MariaDB | slurpit-mariadb | slurpit-network | - | -| Slurp'it MongoDB | slurpit-mongodb | slurpit-network | - | | **Monitoring** | | Uptime Kuma | UptimeKuma | 172.18.0.20 | uptime.xtrm-lab.org | -| Uptime Kuma API | Uptime-Kuma-API | 172.18.0.18 | - | -| AutoKuma | AutoKuma | 172.18.0.19 | - | | NetAlertX | NetAlertX | host | netalert.xtrm-lab.org | | Speedtest Tracker | speedtest-tracker | 172.18.0.21 | speedtest.xtrm-lab.org | -| **Productivity** | -| Actual Budget | actual-budget | 172.18.0.16 | actual.xtrm-lab.org | -| n8n | n8n | 172.18.0.17 | n8n.xtrm-lab.org | -| Karakeep | karakeep | 172.18.0.25 | karakeep.xtrm-lab.org | | **Media & Storage** | | Plex | plex | host | plex.xtrm-lab.org | | Nextcloud | Nextcloud | 172.18.0.24 | nextcloud.xtrm-lab.org | -| Libation | Libation | 172.18.0.23 | - | -| Transmission | transmission | 172.18.0.26 | - | -| Time Machine | TimeMachine | 192.168.31.12 | - | | **Remote Access** | | RustDesk ID | rustdesk-hbbs | bridge | rustdesk.xtrm-lab.org | | RustDesk Relay | rustdesk-hbbr | bridge | - | -| **Other** | -| Home Assistant | HomeAssistant_inabox | host | ha.xtrm-lab.org | -| UrBackup | UrBackup | host | urbackup.xtrm-lab.org | -| Portainer | portainer | bridge | 192.168.31.2:9002 | -| Pangolin | pangolin | 172.18.0.51 | - | - ---- - -## Docker Compose Managed Stacks - -| Stack | Location | Containers | -|-------|----------|------------| -| NetBox | `/mnt/user/appdata/netbox/docker-compose.yml` | netbox, netbox-worker, netbox-postgres, netbox-redis, netbox-redis-cache | -| NetDisco | `/mnt/user/appdata/netdisco/docker-compose.yml` | netdisco-web, netdisco-backend | -| Gitea | `/mnt/user/appdata/gitea/docker-compose.yml` | gitea | -| Woodpecker | `/mnt/user/appdata/woodpecker/docker-compose.yml` | woodpecker-server, woodpecker-agent | -| Pangolin | `/mnt/user/appdata/pangolin/docker-compose.yml` | pangolin | -| Slurp'it | `/mnt/user/appdata/slurpit/docker-compose.yml` | slurpit-portal, slurpit-scanner, slurpit-scraper, slurpit-warehouse, slurpit-mariadb, slurpit-mongodb | - ---- - -## NetBox Plugins - -| Plugin | Version | Status | -|--------|---------|--------| -| slurpit_netbox | 1.2.7 | Active | - -**Note:** Plugin config mounted from `/mnt/user/appdata/netbox/config/plugins.py` --- @@ -190,9 +153,8 @@ │ ┌───────────────▼─────────────────────┐ │ MikroTik hAP ax³ (192.168.31.1) │ - │ WAN: 62.73.120.142 │ │ Ports: 443(DoH), 853(DoT), │ - │ 8853(DoQ), 53(DNS) │ + │ 8853(DoQ), 53→5355(DNS) │ └───────────────┬─────────────────────┘ │ ┌────────────────────────┼────────────────────────┐ @@ -200,17 +162,17 @@ ▼ ▼ ▼ ┌──────────────────────┐ ┌──────────────────┐ ┌──────────────────┐ │ AdGuard Home │ │ Unraid Server │ │ LAN Devices │ -│ 172.17.0.5 │ │ 192.168.31.2 │ │ 192.168.31.x │ -│ Primary DNS │ │ │ │ │ +│ 172.17.0.5:5355 │ │ 192.168.31.2 │ │ 192.168.31.x │ +│ PRIMARY DNS │ │ │ │ │ │ DoH/DoT/DoQ Server │ └────────┬─────────┘ └──────────────────┘ -└────────┬─────────────┘ │ - │ ▼ - ▼ ┌──────────────────┐ -┌──────────────────┐ │ Pi-hole (Unraid) │ -│ Unbound (Router) │ │ 192.168.31.4 │ -│ 172.17.0.3 │ │ Secondary DNS │ -│ Recursive DNS │ └────────┬─────────┘ -└──────────────────┘ │ +└──────────────────────┘ │ + ▼ + ┌──────────────────┐ + │ Pi-hole (Unraid) │ + │ 192.168.31.4 │ + │ SECONDARY DNS │ + └────────┬─────────┘ + │ ▼ ┌──────────────────┐ │ Unbound (Unraid) │ @@ -224,24 +186,24 @@ - **DoT:** `tls://dns.xtrm-lab.org:853` - **DoQ:** `quic://dns.xtrm-lab.org:8853` +**Note:** Pi-hole on Unraid serves as secondary/backup. nebula-sync is disabled (incompatible with AdGuard Home). + --- ## Current NAT/Port Forwarding (MikroTik) -| Rule | Protocol | WAN Port | Destination | Purpose | -|------|----------|----------|-------------|---------| +| Rule | Protocol | Src/Dst Port | Destination | Purpose | +|------|----------|--------------|-------------|---------| | Forward HTTP | TCP | 80 | 192.168.31.2:8001 | Traefik HTTP | | Forward HTTPS | TCP | 443 | 192.168.31.2:44301 | Traefik HTTPS | +| Force DNS to AdGuard | UDP | 53→5355 | 172.17.0.5 | LAN DNS redirect | +| Force DNS TCP | TCP | 53→5355 | 172.17.0.5 | LAN DNS redirect | +| AdGuard Web UI | TCP | 80 | 172.17.0.5:80 | Internal web access | +| DoT | TCP | 853 | 172.17.0.5:853 | DNS over TLS | +| DoH (internal) | TCP | 443 | 172.17.0.5:443 | DNS over HTTPS | | Plex | TCP | 32400 | 192.168.31.2:32400 | Plex Media Server | -| Transmission | TCP/UDP | 51413 | 192.168.31.2:51413 | BitTorrent | -| DoT | TCP | 853 | 172.17.0.5:853 | DNS over TLS (AdGuard) | -| DoQ | UDP | 8853 | 172.17.0.5:8853 | DNS over QUIC (AdGuard) | -| DNS Force | UDP/TCP | 53 | 172.17.0.5:53 | Force LAN DNS to AdGuard Home | -| AdGuard Web UI | TCP | - | 172.17.0.5:80 | Internal access via router IP | | RustDesk | TCP/UDP | 21115-21119 | 192.168.31.2 | RustDesk Server | -**Note:** DoH (443) shares port with Traefik HTTPS. External DoH clients should use the dedicated endpoint or internal access. - --- ## Traefik Configuration @@ -252,59 +214,33 @@ **Certificate Resolver:** Cloudflare DNS Challenge -**Docker Provider Constraint:** `traefik.constraint=valid` -- Containers need this label to be auto-discovered -- Otherwise add routes to `/mnt/user/appdata/traefik/dynamic.yml` - **TLS Certificates Location:** `/mnt/user/appdata/traefik/certs/` - `xtrm-lab.org.crt` - Wildcard certificate chain - `xtrm-lab.org.key` - Private key --- -## Reference Documents +## Migration Data -- [Phase 1: Global DNS Portability](./01-PHASE1-DNS-PORTABILITY.md) -- [Phase 2: Fossorial Tunnel Stack](./02-PHASE2-FOSSORIAL-STACK.md) -- [Phase 3: Identity & Zero Trust](./03-PHASE3-AUTHENTIK-ZEROTRUST.md) -- [Phase 4: Remote Gaming](./04-PHASE4-REMOTE-GAMING.md) -- [Phase 5: RustDesk Setup](./05-PHASE5-RUSTDESK.md) -- [Phase 6: Portainer Management](./06-PHASE6-PORTAINER-MANAGEMENT.md) -- [Phase 7: Gitea GitOps](./08-PHASE7-GITEA-GITOPS.md) -- [Phase 8: NetDisco Integration](./12-PHASE8-NETDISCO-INTEGRATION.md) -- [Container IP Assignments](./13-CONTAINER-IP-ASSIGNMENTS.md) -- [MikroTik WiFi & CAPsMAN](./09-MIKROTIK-WIFI-CAPSMAN.md) +**AdGuard Migration Config:** `/mnt/user/appdata/adguard-migration.json` + +Contains blocklists, custom rules, and client configurations for applying to new AdGuard Home instances. --- ## Backup & Cloud Sync -### Rclone Configuration - -| Remote | Type | Purpose | -|--------|------|---------| -| drive: | Google Drive | Cloud backup storage | - -**Config Location:** /root/.config/rclone/rclone.conf - -### Automated Backups - -| Backup | Source | Destination (Local) | Destination (Cloud) | Schedule | Retention | -|--------|--------|---------------------|---------------------|----------|-----------| -| Flash Backup (Unraid plugin) | /boot/config/ | /mnt/user/Backup/flash | drive:Backups/flash | Daily (via Unraid) | 49 files | -| Flash Backup (Custom script) | /boot/config/ | /mnt/user/Backup/unraid-flash | drive:Backups/unraid-flash | Daily 3:00 AM | 7 days | - ### Flash Backup Script - **Script Path:** /boot/config/plugins/user.scripts/scripts/flash-backup/script - **Schedule:** 0 3 * * * (Daily at 3:00 AM) - **Retention:** 7 days -- **Format:** flash-backup-YYYY-MM-DD.tar.gz -- **Symlink:** flash-backup-latest.tar.gz +- **Cloud Sync:** drive:Backups/unraid-flash -### Cloud Sync Summary +--- -| Folder | Google Drive Path | Size | Files | -|--------|-------------------|------|-------| -| /mnt/user/Backup/flash | drive:Backups/flash | 60.37 GiB | 49 | -| /mnt/user/Backup/unraid-flash | drive:Backups/unraid-flash | 371 MiB | 2 | +## Reference Documents + +- [Phase 1: Global DNS Portability](./01-PHASE1-DNS-PORTABILITY.md) +- [Phase 7: Gitea GitOps](./08-PHASE7-GITEA-GITOPS.md) +- [Container IP Assignments](./13-CONTAINER-IP-ASSIGNMENTS.md) diff --git a/docs/06-CHANGELOG.md b/docs/06-CHANGELOG.md index db7bfe0..2d874a8 100644 --- a/docs/06-CHANGELOG.md +++ b/docs/06-CHANGELOG.md @@ -1,34 +1,41 @@ ## 2026-01-22 - MikroTik DNS Migration to AdGuard Home -### Pi-hole Removal -- [CONTAINER] Removed Pi-hole container from MikroTik (was 172.17.0.2) -- [STORAGE] Freed 91.2 MiB internal flash storage (was full at 128MB) -- [CLEANUP] Removed Pi-hole mounts, envs, veth interface, and data directories +### Pi-hole Removal from MikroTik +- [CONTAINER] Removed Pi-hole container from MikroTik +- [STORAGE] Freed internal flash storage +- [CLEANUP] Removed Pi-hole mounts, envs, and data -### AdGuard Home Installation -- [CONTAINER] Deployed adguardhome:latest on MikroTik -- [IP] Assigned 172.17.0.5 (veth-adguard interface) -- [STORAGE] Data stored on USB (usb1/adguardhome) +### AdGuard Home Installation (Multiple Attempts) +- [ISSUE] MikroTik container root directory disappears on stop (bug) +- [WORKAROUND] Use DNS port 5355 to avoid stats.db creation error +- [CONTAINER] Final working config: usb1/agh2 root-dir, no mounts - [VERSION] AdGuard Home v0.107.71 -### Encrypted DNS Configuration -- [TLS] Configured Let's Encrypt wildcard certificate (*.xtrm-lab.org) -- [DOH] DNS-over-HTTPS enabled on port 443 -- [DOT] DNS-over-TLS enabled on port 853 -- [DOQ] DNS-over-QUIC enabled on port 8853 -- [SERVER] Server name: dns.xtrm-lab.org -- [CERT] Certificate expires: 2026-04-02 +### Configuration Applied via API +- [BLOCKLISTS] StevenBlack Hosts, Hagezi Pro, Hagezi NSFW +- [RULES] Custom blocks: SentinelOne, Jamfcloud domains +- [CLIENTS] 6 devices migrated from Pi-hole +- [TLS] Let's Encrypt wildcard cert (*.xtrm-lab.org) + +### Encrypted DNS Services +- [DOH] Port 443 - Active +- [DOT] Port 853 - Active +- [DOQ] Port 8853 - Active +- [SERVER] dns.xtrm-lab.org ### NAT Rules Updated -- [NAT] Rule 7: DNS Force now points to 172.17.0.5 (AdGuard Home) -- [NAT] Rule 9: DNS TCP Force now points to 172.17.0.5 -- [NAT] Rule 24: AdGuard Home Web UI (192.168.31.1:80 → 172.17.0.5:80) -- [NAT] DoT/DoQ rules to be added for external access +- [NAT] DNS Force: 53 → 172.17.0.5:5355 (UDP/TCP) +- [NAT] Web UI: 80 → 172.17.0.5:80 +- [NAT] DoT: 853 → 172.17.0.5:853 +- [NAT] DoH: 443 → 172.17.0.5:443 -### Benefits -- [FEATURE] Native DoH/DoT/DoQ server support (Pi-hole required extra containers) -- [RESOURCE] Reduced container count (no need for separate DoH-Server) -- [STORAGE] Better storage utilization (USB instead of internal flash) +### Migration Data Saved +- [FILE] /mnt/user/appdata/adguard-migration.json +- [DATA] Blocklists, rules, clients for future Unraid migration + +### Known Issues +- [BUG] MikroTik container root-dir disappears on stop - DO NOT RESTART +- [INCOMPATIBLE] nebula-sync crash-looping (Pi-hole ↔ AdGuard incompatible) --- @@ -44,44 +51,33 @@ - [PATH] Changed from /mnt/user/backup/unraid-flash to /mnt/user/Backup/unraid-flash - [SYNC] Synced to drive:Backups/unraid-flash (371 MiB) -### Cloud Backup Sync -- [SYNC] /mnt/user/Backup/flash -> drive:Backups/flash (60.37 GiB, 49 files) -- [SYNC] /mnt/user/Backup/unraid-flash -> drive:Backups/unraid-flash (371 MiB, 2 files) - --- ## 2026-01-21 - Pi-hole Version Sync Automation ### MikroTik Pi-hole Update -- [CONTAINER] Updated MikroTik Pi-hole to v6.3/v6.4/v6.4.1 (matching Unraid) +- [CONTAINER] Updated MikroTik Pi-hole to v6.4.1 (matching Unraid) - [CONFIG] Enabled FTLCONF_webserver_api_app_sudo=true for nebula-sync - [FIX] Resolved nebula-sync crash loop (was failing with HTTP 400) ### Version Sync Script - [SCRIPT] Created pihole-version-sync User Script - [SCHEDULE] Runs daily at 4:00 AM -- [FUNCTION] Compares Pi-hole versions and auto-updates MikroTik when needed - [PATH] /boot/config/plugins/user.scripts/scripts/pihole-version-sync/ +--- + ## 2026-01-19 - Phase 8 Enhanced Network Mapping ### MikroTik DHCP Sync - [SCRIPT] Created mikrotik_dhcp_to_netbox.sh - [SYNC] 29 DHCP leases synced to NetBox IPs -- [DATA] Hostname, MAC, comments captured ### Slurpit Plugin Installation - [PLUGIN] Installed slurpit_netbox v1.2.7 -- [BUILD] Created netbox-custom:latest image - [CONFIG] Plugin configuration at /mnt/user/appdata/netbox/config/plugins.py -### Enhanced NetDisco Sync -- [SCRIPT] Updated sync_to_netbox.py with additional data -- [SYNC] Device info, IPs, MACs, ARP table entries -- [DATA] 4 devices synced with full metadata - -### Unraid SNMP -- [SERVICE] kubedzero/unraid-snmp plugin installed +--- ## 2026-01-18 - Phase 7 Gitea & Woodpecker CI @@ -95,24 +91,7 @@ - [URL] https://ci.xtrm-lab.org - [AUTH] Integrated with Gitea OAuth2 -### Infrastructure Repository -- [REPO] Created infrastructure repo in Gitea -- [DOCS] Migrated all documentation to version control -- [CI] Basic pipeline validation configured - -## 2026-01-14 - Phase 6 Portainer Management - -### Portainer Setup -- [SERVICE] Portainer Business Edition deployed -- [URL] https://portainer.xtrm-lab.org -- [AUTH] Authentik integration - -## 2026-01-11 - Phase 5 RustDesk Deployment - -### RustDesk Server -- [SERVICE] rustdesk-hbbs and rustdesk-hbbr deployed -- [PORTS] TCP 21115-21119, UDP 21116 -- [CONFIG] Custom relay server configured +--- ## Previous Changes