infrastructure/docs/archive/06-CHANGELOG.md

2026-01-23 - NetBox Discovery (Diode) Setup & Slurp'it Removal

Slurp'it Removal

• [SLURPIT] Removed entire Slurp'it stack (portal, scanner, scraper, warehouse, mariadb, mongodb)
• [SLURPIT] Decision: Use NetBox Discovery + NetDisco instead for better NetBox integration

NetBox Discovery (Diode) Installation

• [DIODE] Installed Diode server stack via quickstart.sh
• [DIODE] Containers: ingress-nginx, diode-auth, diode-ingester, diode-reconciler, hydra, postgres, redis
• [NETBOX] Installed netboxlabs-diode-netbox-plugin via custom Dockerfile
• [NETBOX] Ran plugin migrations for netbox_diode_plugin
• [DIODE] Configured Nginx with public /auth/introspect endpoint for NetBox plugin
• [DIODE] Connected NetBox to diode_default network for inter-container communication

Discovery Agent Setup

• [DIODE] Deployed orb-agent container for network discovery
• [DIODE] Configured network_discovery policy: 192.168.31.0/24, ports 22/80/161/443
• [DIODE] Schedule: Every 30 minutes (*/30 * * * *)
• [DIODE] Successfully discovered 26 hosts on first scan

NetDisco to NetBox Sync

• [SYNC] Created Python sync script at /mnt/user/appdata/netdisco-netbox-sync/
• [SYNC] Uses Diode SDK to ingest devices and IP addresses from NetDisco
• [SYNC] Syncs: 4 devices (with vendor, model, OS) and 42 ARP entries (with MAC)
• [SYNC] Containerized with Docker for easy deployment

Documentation

• [DOCS] Updated 00-CURRENT-STATE.md with new discovery architecture
• [DOCS] Removed Slurp'it references, added Diode and sync script documentation

---

2026-01-22 - NetBox Migration to Shared PostgreSQL

Database Consolidation

• [NETBOX] Migrated NetBox database to shared postgresql17
• [NETBOX] Created netbox_user and netbox database on postgresql17
• [NETBOX] Backed up and imported all NetBox data
• [NETBOX] Removed dedicated netbox-postgres container

Network Consolidation

• [NETBOX] Moved all NetBox containers to dockerproxy network
• [NETBOX] Assigned static IPs: netbox (172.18.0.61), worker (172.18.0.62), redis (172.18.0.63/64)
• [NETBOX] Removed unused netbox network (172.24.0.0/16)

Resource Savings

• Removed netbox-postgres container (~200-400MB RAM saved)
• Consolidated network infrastructure

---

2026-01-22 - Slurp'it Network Discovery Setup

SNMP Configuration

• [MIKROTIK] Enabled SNMP on router (192.168.31.1)
• [SNMP] Communities configured: public, netdisco
• [DISCOVERY] MikroTik router discovered via SNMP

Agent Service Account

• [UNRAID] Created agent user with SSH key access (port 422)
• [MIKROTIK-ROUTER] Created agent user with SSH key (port 2222)
• [MIKROTIK-AP] Created agent user with password auth (port 2222)
• [SWITCH] CSS326 uses SwOS - no SSH support

Slurp'it Configuration

• [SNMP] Added SNMP v2c credentials to vault (public, netdisco)
• [NETBOX] Enabled NetBox integration plugin
• [SCAN] Configured scan target: 192.168.31.0/24

Documentation

• [DOCS] Added Network Discovery section to 00-CURRENT-STATE.md
• [DOCS] Created AGENT-CREDENTIALS.md (gitignored)
• [DIAGRAM] Added INFRASTRUCTURE-DIAGRAM.md with complete topology

---

2026-01-22 - MikroTik DNS Migration to AdGuard Home

Pi-hole Removal from MikroTik

• [CONTAINER] Removed Pi-hole container from MikroTik
• [STORAGE] Freed internal flash storage

2026-01-22 - AdGuard Home Migration Complete

MikroTik AdGuard Home - Persistence Fix

• [CONTAINER] Fixed container persistence issue (root-dir on disk1, data on usb1)
• [CONFIG] Container now survives stop/start cycles
• [MOUNT] agh-work mount: usb1/adguard-home/work → /opt/adguardhome/work

Unraid AdGuard Home - Replaces Pi-hole

• [CONTAINER] Deployed AdGuard Home on br0 macvlan network
• [IP] 192.168.31.4 (same IP as Pi-hole was using)
• [STOPPED] binhex-official-pihole container stopped (not removed)
• [CONFIG] Same credentials and rules as MikroTik instance

Configuration Sync (Both Instances)

• [DNS] Upstream: Quad9 DoH (dns10.quad9.net)
• [TLS] Let's Encrypt wildcard cert for *.xtrm-lab.org
• [CLIENTS] 6 clients configured with MAC addresses
• [RULES] Custom filtering rules for SentinelOne, Jamf

Documentation

• [DOCS] Updated 00-CURRENT-STATE.md with Mermaid diagrams
• [DIAGRAM] Added network topology and DNS architecture diagrams

---

• [CLEANUP] Removed Pi-hole mounts, envs, and data

AdGuard Home Installation (Multiple Attempts)

• [ISSUE] MikroTik container root directory disappears on stop (bug)
• [WORKAROUND] Use DNS port 5355 to avoid stats.db creation error
• [CONTAINER] Final working config: usb1/agh2 root-dir, no mounts
• [VERSION] AdGuard Home v0.107.71

Configuration Applied via API

• [BLOCKLISTS] StevenBlack Hosts, Hagezi Pro, Hagezi NSFW
• [RULES] Custom blocks: SentinelOne, Jamfcloud domains
• [CLIENTS] 6 devices migrated from Pi-hole
• [TLS] Let's Encrypt wildcard cert (*.xtrm-lab.org)

Encrypted DNS Services

• [DOH] Port 443 - Active
• [DOT] Port 853 - Active
• [DOQ] Port 8853 - Active
• [SERVER] dns.xtrm-lab.org

NAT Rules Updated

• [NAT] DNS Force: 53 → 172.17.0.5:5355 (UDP/TCP)
• [NAT] Web UI: 80 → 172.17.0.5:80
• [NAT] DoT: 853 → 172.17.0.5:853
• [NAT] DoH: 443 → 172.17.0.5:443

Migration Data Saved

• [FILE] /mnt/user/appdata/adguard-migration.json
• [DATA] Blocklists, rules, clients for future Unraid migration

Known Issues

• [BUG] MikroTik container root-dir disappears on stop - DO NOT RESTART
• [INCOMPATIBLE] nebula-sync crash-looping (Pi-hole ↔ AdGuard incompatible)

---

2026-01-21 - Rclone & Cloud Backup Setup

Rclone Installation & Configuration

• [SERVICE] Installed rclone on Unraid
• [CONFIG] Configured Google Drive remote (drive:)
• [SYNC] Initial sync completed for backup folders

Flash Backup Updates

• [SCRIPT] Updated flash-backup script output path
• [PATH] Changed from /mnt/user/backup/unraid-flash to /mnt/user/Backup/unraid-flash
• [SYNC] Synced to drive:Backups/unraid-flash (371 MiB)

---

2026-01-21 - Pi-hole Version Sync Automation

MikroTik Pi-hole Update

• [CONTAINER] Updated MikroTik Pi-hole to v6.4.1 (matching Unraid)
• [CONFIG] Enabled FTLCONF_webserver_api_app_sudo=true for nebula-sync
• [FIX] Resolved nebula-sync crash loop (was failing with HTTP 400)

Version Sync Script

• [SCRIPT] Created pihole-version-sync User Script
• [SCHEDULE] Runs daily at 4:00 AM
• [PATH] /boot/config/plugins/user.scripts/scripts/pihole-version-sync/

---

2026-01-19 - Phase 8 Enhanced Network Mapping

MikroTik DHCP Sync

• [SCRIPT] Created mikrotik_dhcp_to_netbox.sh
• [SYNC] 29 DHCP leases synced to NetBox IPs

Slurpit Plugin Installation

• [PLUGIN] Installed slurpit_netbox v1.2.7
• [CONFIG] Plugin configuration at /mnt/user/appdata/netbox/config/plugins.py

---

2026-01-18 - Phase 7 Gitea & Woodpecker CI

Gitea Setup

• [SERVICE] gitea container deployed
• [URL] https://git.xtrm-lab.org
• [AUTH] Integrated with Authentik OAuth2

Woodpecker CI

• [SERVICE] woodpecker-server and woodpecker-agent deployed
• [URL] https://ci.xtrm-lab.org
• [AUTH] Integrated with Gitea OAuth2

---

Previous Changes

See git history for earlier changes.

2026-01-22

• [CLEANUP] Removed Pi-hole container (binhex-official-pihole) from Unraid - using AdGuard Home on MikroTik as primary DNS
• [CLEANUP] Removed nebula-sync container - not in use
• [CLEANUP] Removed Traefik routes for ph1.xtrm-lab.org
• [SERVICE] adguardhome-sync: Added for syncing rules between MikroTik AdGuard Home and Unraid AdGuard
• [MONITORING] Added 27 monitors to Uptime Kuma covering all web services and infrastructure
• [ORGANIZATION] Updated Unraid container categories for better organization
• [NETBOX] Migrated NetBox to shared postgresql17 database and dockerproxy network

2026-01-23 - Diode Stack Consolidation

Service Consolidation

• [POSTGRES] Removed dedicated diode-postgres container
• [REDIS] Removed dedicated diode-redis container
• [NETWORK] Migrated all Diode containers to dockerproxy network
• [SHARED] Using postgresql17 (172.18.0.13) for diode and hydra databases
• [SHARED] Using Redis (172.18.0.14) for queue management

Static IP Assignments (dockerproxy)

• diode-ingress: 172.18.0.70
• diode-ingester: 172.18.0.71
• diode-reconciler: 172.18.0.72
• diode-hydra: 172.18.0.73
• diode-auth: 172.18.0.74

Unraid Docker UI

• [LABELS] Added net.unraid.docker.managed=dockerman
• [ICONS] NetBox/Diode icon for all containers
• [FOLDERVIEW] Containers visible in Docker tab

Configuration Updates

• [ENV] Updated .env to use shared service hostnames
• [NGINX] Updated nginx.conf with new container names
• [AGENT] Updated discovery agent config with diode-ingress IP

---

2026-01-23 - NetBox Redis Consolidation

Service Consolidation

• [REDIS] Removed netbox-redis container (task queue)
• [SHARED] Using Redis (172.18.0.14) for NetBox task queue
• [CACHE] Kept netbox-redis-cache (172.18.0.64) for caching

Configuration Changes

• [ENV] REDIS_HOST changed from 172.18.0.63 to 172.18.0.14
• [LABELS] Added Unraid labels and icons to NetBox containers

Containers Removed

• netbox-redis (was 172.18.0.63)

---

2026-01-23 - Service Cleanup & Documentation Update

Services Removed

• [REMOVED] Unbound - redundant (AdGuard has upstream DoH)
• [REMOVED] DoH-Server - redundant (AdGuard has built-in DoH)
• [REMOVED] stunnel-dot - redundant (AdGuard has built-in DoT)
• [REMOVED] Pangolin - not in use

DNS Configuration

• [CONFIG] Unraid AdGuard: dns2.xtrm-lab.org (was dns.xtrm-lab.org)
• [CONFIG] MikroTik AdGuard: dns.xtrm-lab.org (primary)

Container Management

• [LABELS] Added net.unraid.docker.managed to all containers
• [LABELS] Added WebUI URLs to containers with web interfaces
• [LABELS] Updated icons to PNG format (from SVG)

FolderView2

• [CATEGORY] Added "Network Inventory" for NetBox/Diode/NetDisco

Documentation

• [DOCS] Updated 00-CURRENT-STATE.md with current architecture
• [DOCS] Added Mermaid diagrams for network topology
• [DOCS] Added RAM usage statistics
• [DOCS] Documented removed services

---