jazzymc/infrastructure
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
8aef54992a073d444b458dfcf8600afa1be095c8
infrastructure/docs/archive/06-CHANGELOG.md
XTRM-Unraid b250493d5a
All checks were successful
ci/woodpecker/push/woodpecker Pipeline was successful
Details
Major documentation restructure - consolidated docs 
New Structure:
- 01-NETWORK-MAP.md - Network topology, IPs, Docker networks, services
- 02-SERVICES-CRITICAL.md - DNS, Auth, Routing (P0/P1 services)
- 03-SERVICES-OTHER.md - All non-critical services
- 04-HARDWARE-INVENTORY.md - Physical devices and specs
- 05-CHANGELOG.md - Major events only

New Folders:
- docs/archive/ - Legacy docs (read-only reference)
- docs/wip/ - Planned changes and ideas
  - UPGRADE-2026-HARDWARE.md - N5 Air + N100 migration plan
  - GITOPS-CONTAINERS.md - Phase 2 container GitOps

Changes:
- Moved all 22 legacy docs to archive/
- Consolidated container IPs, physical map, and services into single network map
- Extracted critical vs non-critical service classification
- Simplified changelog to major events only

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-25 11:17:33 +02:00

282 lines
9.8 KiB
Markdown
Raw Blame History

## 2026-01-23 - NetBox Discovery (Diode) Setup & Slurp'it Removal
### Slurp'it Removal
- [SLURPIT] Removed entire Slurp'it stack (portal, scanner, scraper, warehouse, mariadb, mongodb)
- [SLURPIT] Decision: Use NetBox Discovery + NetDisco instead for better NetBox integration
### NetBox Discovery (Diode) Installation
- [DIODE] Installed Diode server stack via quickstart.sh
- [DIODE] Containers: ingress-nginx, diode-auth, diode-ingester, diode-reconciler, hydra, postgres, redis
- [NETBOX] Installed netboxlabs-diode-netbox-plugin via custom Dockerfile
- [NETBOX] Ran plugin migrations for netbox_diode_plugin
- [DIODE] Configured Nginx with public /auth/introspect endpoint for NetBox plugin
- [DIODE] Connected NetBox to diode_default network for inter-container communication
### Discovery Agent Setup
- [DIODE] Deployed orb-agent container for network discovery
- [DIODE] Configured network_discovery policy: 192.168.31.0/24, ports 22/80/161/443
- [DIODE] Schedule: Every 30 minutes (*/30 * * * *)
- [DIODE] Successfully discovered 26 hosts on first scan
### NetDisco to NetBox Sync
- [SYNC] Created Python sync script at /mnt/user/appdata/netdisco-netbox-sync/
- [SYNC] Uses Diode SDK to ingest devices and IP addresses from NetDisco
- [SYNC] Syncs: 4 devices (with vendor, model, OS) and 42 ARP entries (with MAC)
- [SYNC] Containerized with Docker for easy deployment
### Documentation
- [DOCS] Updated 00-CURRENT-STATE.md with new discovery architecture
- [DOCS] Removed Slurp'it references, added Diode and sync script documentation
---
## 2026-01-22 - NetBox Migration to Shared PostgreSQL
### Database Consolidation
- [NETBOX] Migrated NetBox database to shared postgresql17
- [NETBOX] Created netbox_user and netbox database on postgresql17
- [NETBOX] Backed up and imported all NetBox data
- [NETBOX] Removed dedicated netbox-postgres container
### Network Consolidation
- [NETBOX] Moved all NetBox containers to dockerproxy network
- [NETBOX] Assigned static IPs: netbox (172.18.0.61), worker (172.18.0.62), redis (172.18.0.63/64)
- [NETBOX] Removed unused netbox network (172.24.0.0/16)
### Resource Savings
- Removed netbox-postgres container (~200-400MB RAM saved)
- Consolidated network infrastructure
---
## 2026-01-22 - Slurp'it Network Discovery Setup
### SNMP Configuration
- [MIKROTIK] Enabled SNMP on router (192.168.31.1)
- [SNMP] Communities configured: public, netdisco
- [DISCOVERY] MikroTik router discovered via SNMP
### Agent Service Account
- [UNRAID] Created agent user with SSH key access (port 422)
- [MIKROTIK-ROUTER] Created agent user with SSH key (port 2222)
- [MIKROTIK-AP] Created agent user with password auth (port 2222)
- [SWITCH] CSS326 uses SwOS - no SSH support
### Slurp'it Configuration
- [SNMP] Added SNMP v2c credentials to vault (public, netdisco)
- [NETBOX] Enabled NetBox integration plugin
- [SCAN] Configured scan target: 192.168.31.0/24
### Documentation
- [DOCS] Added Network Discovery section to 00-CURRENT-STATE.md
- [DOCS] Created AGENT-CREDENTIALS.md (gitignored)
- [DIAGRAM] Added INFRASTRUCTURE-DIAGRAM.md with complete topology
---
## 2026-01-22 - MikroTik DNS Migration to AdGuard Home
### Pi-hole Removal from MikroTik
- [CONTAINER] Removed Pi-hole container from MikroTik
- [STORAGE] Freed internal flash storage
## 2026-01-22 - AdGuard Home Migration Complete
### MikroTik AdGuard Home - Persistence Fix
- [CONTAINER] Fixed container persistence issue (root-dir on disk1, data on usb1)
- [CONFIG] Container now survives stop/start cycles
- [MOUNT] agh-work mount: usb1/adguard-home/work → /opt/adguardhome/work
### Unraid AdGuard Home - Replaces Pi-hole
- [CONTAINER] Deployed AdGuard Home on br0 macvlan network
- [IP] 192.168.31.4 (same IP as Pi-hole was using)
- [STOPPED] binhex-official-pihole container stopped (not removed)
- [CONFIG] Same credentials and rules as MikroTik instance
### Configuration Sync (Both Instances)
- [DNS] Upstream: Quad9 DoH (dns10.quad9.net)
- [TLS] Let's Encrypt wildcard cert for *.xtrm-lab.org
- [CLIENTS] 6 clients configured with MAC addresses
- [RULES] Custom filtering rules for SentinelOne, Jamf
### Documentation
- [DOCS] Updated 00-CURRENT-STATE.md with Mermaid diagrams
- [DIAGRAM] Added network topology and DNS architecture diagrams
---
- [CLEANUP] Removed Pi-hole mounts, envs, and data
### AdGuard Home Installation (Multiple Attempts)
- [ISSUE] MikroTik container root directory disappears on stop (bug)
- [WORKAROUND] Use DNS port 5355 to avoid stats.db creation error
- [CONTAINER] Final working config: usb1/agh2 root-dir, no mounts
- [VERSION] AdGuard Home v0.107.71
### Configuration Applied via API
- [BLOCKLISTS] StevenBlack Hosts, Hagezi Pro, Hagezi NSFW
- [RULES] Custom blocks: SentinelOne, Jamfcloud domains
- [CLIENTS] 6 devices migrated from Pi-hole
- [TLS] Let's Encrypt wildcard cert (*.xtrm-lab.org)
### Encrypted DNS Services
- [DOH] Port 443 - Active
- [DOT] Port 853 - Active
- [DOQ] Port 8853 - Active
- [SERVER] dns.xtrm-lab.org
### NAT Rules Updated
- [NAT] DNS Force: 53 → 172.17.0.5:5355 (UDP/TCP)
- [NAT] Web UI: 80 → 172.17.0.5:80
- [NAT] DoT: 853 → 172.17.0.5:853
- [NAT] DoH: 443 → 172.17.0.5:443
### Migration Data Saved
- [FILE] /mnt/user/appdata/adguard-migration.json
- [DATA] Blocklists, rules, clients for future Unraid migration
### Known Issues
- [BUG] MikroTik container root-dir disappears on stop - DO NOT RESTART
- [INCOMPATIBLE] nebula-sync crash-looping (Pi-hole ↔ AdGuard incompatible)
---
## 2026-01-21 - Rclone & Cloud Backup Setup
### Rclone Installation & Configuration
- [SERVICE] Installed rclone on Unraid
- [CONFIG] Configured Google Drive remote (drive:)
- [SYNC] Initial sync completed for backup folders
### Flash Backup Updates
- [SCRIPT] Updated flash-backup script output path
- [PATH] Changed from /mnt/user/backup/unraid-flash to /mnt/user/Backup/unraid-flash
- [SYNC] Synced to drive:Backups/unraid-flash (371 MiB)
---
## 2026-01-21 - Pi-hole Version Sync Automation
### MikroTik Pi-hole Update
- [CONTAINER] Updated MikroTik Pi-hole to v6.4.1 (matching Unraid)
- [CONFIG] Enabled FTLCONF_webserver_api_app_sudo=true for nebula-sync
- [FIX] Resolved nebula-sync crash loop (was failing with HTTP 400)
### Version Sync Script
- [SCRIPT] Created pihole-version-sync User Script
- [SCHEDULE] Runs daily at 4:00 AM
- [PATH] /boot/config/plugins/user.scripts/scripts/pihole-version-sync/
---
## 2026-01-19 - Phase 8 Enhanced Network Mapping
### MikroTik DHCP Sync
- [SCRIPT] Created mikrotik_dhcp_to_netbox.sh
- [SYNC] 29 DHCP leases synced to NetBox IPs
### Slurpit Plugin Installation
- [PLUGIN] Installed slurpit_netbox v1.2.7
- [CONFIG] Plugin configuration at /mnt/user/appdata/netbox/config/plugins.py
---
## 2026-01-18 - Phase 7 Gitea & Woodpecker CI
### Gitea Setup
- [SERVICE] gitea container deployed
- [URL] https://git.xtrm-lab.org
- [AUTH] Integrated with Authentik OAuth2
### Woodpecker CI
- [SERVICE] woodpecker-server and woodpecker-agent deployed
- [URL] https://ci.xtrm-lab.org
- [AUTH] Integrated with Gitea OAuth2
---
## Previous Changes
See git history for earlier changes.
## 2026-01-22
- [CLEANUP] Removed Pi-hole container (binhex-official-pihole) from Unraid - using AdGuard Home on MikroTik as primary DNS
- [CLEANUP] Removed nebula-sync container - not in use
- [CLEANUP] Removed Traefik routes for ph1.xtrm-lab.org
- [SERVICE] adguardhome-sync: Added for syncing rules between MikroTik AdGuard Home and Unraid AdGuard
- [MONITORING] Added 27 monitors to Uptime Kuma covering all web services and infrastructure
- [ORGANIZATION] Updated Unraid container categories for better organization
- [NETBOX] Migrated NetBox to shared postgresql17 database and dockerproxy network
## 2026-01-23 - Diode Stack Consolidation
### Service Consolidation
- [POSTGRES] Removed dedicated diode-postgres container
- [REDIS] Removed dedicated diode-redis container
- [NETWORK] Migrated all Diode containers to dockerproxy network
- [SHARED] Using postgresql17 (172.18.0.13) for diode and hydra databases
- [SHARED] Using Redis (172.18.0.14) for queue management
### Static IP Assignments (dockerproxy)
- diode-ingress: 172.18.0.70
- diode-ingester: 172.18.0.71
- diode-reconciler: 172.18.0.72
- diode-hydra: 172.18.0.73
- diode-auth: 172.18.0.74
### Unraid Docker UI
- [LABELS] Added net.unraid.docker.managed=dockerman
- [ICONS] NetBox/Diode icon for all containers
- [FOLDERVIEW] Containers visible in Docker tab
### Configuration Updates
- [ENV] Updated .env to use shared service hostnames
- [NGINX] Updated nginx.conf with new container names
- [AGENT] Updated discovery agent config with diode-ingress IP
---
## 2026-01-23 - NetBox Redis Consolidation
### Service Consolidation
- [REDIS] Removed netbox-redis container (task queue)
- [SHARED] Using Redis (172.18.0.14) for NetBox task queue
- [CACHE] Kept netbox-redis-cache (172.18.0.64) for caching
### Configuration Changes
- [ENV] REDIS_HOST changed from 172.18.0.63 to 172.18.0.14
- [LABELS] Added Unraid labels and icons to NetBox containers
### Containers Removed
- netbox-redis (was 172.18.0.63)
---
## 2026-01-23 - Service Cleanup & Documentation Update
### Services Removed
- [REMOVED] Unbound - redundant (AdGuard has upstream DoH)
- [REMOVED] DoH-Server - redundant (AdGuard has built-in DoH)
- [REMOVED] stunnel-dot - redundant (AdGuard has built-in DoT)
- [REMOVED] Pangolin - not in use
### DNS Configuration
- [CONFIG] Unraid AdGuard: dns2.xtrm-lab.org (was dns.xtrm-lab.org)
- [CONFIG] MikroTik AdGuard: dns.xtrm-lab.org (primary)
### Container Management
- [LABELS] Added net.unraid.docker.managed to all containers
- [LABELS] Added WebUI URLs to containers with web interfaces
- [LABELS] Updated icons to PNG format (from SVG)
### FolderView2
- [CATEGORY] Added "Network Inventory" for NetBox/Diode/NetDisco
### Documentation
- [DOCS] Updated 00-CURRENT-STATE.md with current architecture
- [DOCS] Added Mermaid diagrams for network topology
- [DOCS] Added RAM usage statistics
- [DOCS] Documented removed services
---
Reference in New Issue View Git Blame Copy Permalink