diff --git a/docs/00-CURRENT-STATE.md b/docs/00-CURRENT-STATE.md index 59d889c..18ca13f 100644 --- a/docs/00-CURRENT-STATE.md +++ b/docs/00-CURRENT-STATE.md @@ -1,6 +1,6 @@ # Infrastructure Current State: xtrm-lab.org -## Document Updated: 2026-01-22 +## Document Updated: 2026-01-23 ## Target Domain: xtrm-lab.org --- @@ -49,7 +49,8 @@ graph TB end subgraph Monitoring["Monitoring"] UPTIME["Uptime Kuma
172.18.0.20"] - NETBOX["NetBox
172.24.0.5"] + NETBOX["NetBox
172.18.0.61"] + DIODE["NetBox Discovery
172.24.0.10"] end subgraph Media["Media"] PLEX["Plex"] @@ -113,7 +114,7 @@ graph TB **Configuration:** - Upstream: Quad9 DoH (https://dns10.quad9.net/dns-query) -- TLS Certificate: Let's Encrypt wildcard (\*.xtrm-lab.org) +- TLS Certificate: Let's Encrypt wildcard (*.xtrm-lab.org) - Server Name: dns.xtrm-lab.org - Certificate Expiry: 2026-04-02 - Credentials: jazzymc / 7RqWElENNbZnPW @@ -154,7 +155,7 @@ graph TB |---------|--------|---------| | br0 | 192.168.31.0/24 | LAN macvlan (AdGuard Home) | | dockerproxy | 172.18.0.0/16 | Traefik-accessible services | -| netbox | 172.24.0.0/16 | NetBox stack | +| diode_default | 172.24.0.0/16 | NetBox Discovery (Diode) | | bridge | 172.17.0.0/16 | Default Docker bridge | ### Key Services @@ -175,7 +176,7 @@ graph TB | CI/CD Server | woodpecker-server | 172.18.0.32 | ci.xtrm-lab.org | | **Monitoring** |||| | Uptime Kuma | UptimeKuma | 172.18.0.20 | uptime.xtrm-lab.org | -| NetBox | netbox | 172.24.0.5 | netbox.xtrm-lab.org | +| NetBox | netbox | 172.18.0.61 | netbox.xtrm-lab.org | | **Media** |||| | Plex | plex | host | plex.xtrm-lab.org | | Nextcloud | Nextcloud | 172.18.0.24 | nextcloud.xtrm-lab.org | @@ -298,47 +299,86 @@ flowchart TB ## Network Discovery & Management -### Slurp'it Stack +### NetBox (IPAM/DCIM) | Container | IP | Purpose | |-----------|-----|---------| -| slurpit-portal | 172.18.0.129 | Web UI (slurpit.xtrm-lab.org) | -| slurpit-scanner | 172.25.0.5 | SNMP network scanner | -| slurpit-scraper | 172.25.0.3 | Device data collector | -| slurpit-warehouse | 172.25.0.4 | Data storage API | -| slurpit-mariadb | 172.25.0.2 | Portal database | -| slurpit-mongodb | 172.25.0.6 | Discovery database | +| netbox | 172.18.0.61 | Web UI (netbox.xtrm-lab.org) | +| netbox-postgres | - | Database | +| netbox-redis | - | Cache | +| netbox-redis-cache | - | Redis cache | +| netbox-worker | - | Background tasks | -**Status:** Operational -**Discovered Devices:** 1 (MikroTik Router) -**SNMP Communities:** public, netdisco +**Plugins Installed:** +- netboxlabs-diode-netbox-plugin (NetBox Discovery integration) -**Configuration:** -- SNMP v2c credentials configured -- Scan target: 192.168.31.0/24 -- NetBox integration: Enabled (plugin_online: 1) +### NetBox Discovery (Diode) -**Pending Tasks:** -- Add SSH credentials to Vault for device scraping -- Troubleshoot SNMP discovery of switch and AP +NetBox Labs Diode provides automated network discovery and data ingestion into NetBox. + +| Container | IP | Purpose | +|-----------|-----|---------| +| diode-ingress-nginx-1 | 172.24.0.10 | API Gateway | +| diode-diode-auth-1 | - | OAuth2 authentication | +| diode-diode-ingester-1 | - | Data ingestion service | +| diode-diode-reconciler-1 | - | Data reconciliation | +| diode-hydra-1 | - | OAuth2 provider (Ory Hydra) | +| diode-postgres-1 | - | Database | +| diode-redis-1 | - | Cache | +| diode-discovery-agent | host network | Network scanner (orb-agent) | + +**Data Location:** /mnt/user/appdata/diode/ + +**Discovery Agent Configuration:** +- Schedule: Every 30 minutes +- Target: 192.168.31.0/24 +- Ports scanned: 22, 80, 161, 443 +- Site: Home + +**OAuth2 Credentials:** +- diode-ingest: For data ingestion +- netbox-to-diode: For NetBox plugin +- diode-to-netbox: For reconciler ### NetDisco +NetDisco provides SNMP-based network discovery and ARP table collection. + | Container | IP | Purpose | |-----------|-----|---------| | netdisco-web | 172.18.0.41 | Web UI (netdisco.xtrm-lab.org) | | netdisco-backend | 172.18.0.42 | SNMP poller | -### NetBox (IPAM/DCIM) +**Database:** postgresql17 (shared) +- Database: netdisco_db +- User: netdisco_user -| Container | IP | Purpose | -|-----------|-----|---------| -| netbox | 172.24.0.5 | Web UI (netbox.xtrm-lab.org) | -| netbox-postgres | 172.24.0.4 | Database | -| netbox-redis | 172.24.0.2 | Cache | -| netbox-worker | 172.24.0.6 | Background tasks | +**Discovered Data:** +- 4 SNMP-enabled devices +- 42 ARP entries (all network hosts) -**NetBox Slurp'it Plugin:** Installed and configured +### NetDisco to NetBox Sync + +A scheduled sync script pushes NetDisco data to NetBox via Diode. + +**Location:** /mnt/user/appdata/netdisco-netbox-sync/ + +| File | Purpose | +|------|---------| +| sync.py | Python sync script | +| Dockerfile | Container build file | +| docker-compose.yml | Deployment config | + +**Sync Configuration:** +- Source: NetDisco PostgreSQL database +- Target: NetBox via Diode gRPC API +- Data synced: Devices (with vendor, model, OS) and IP addresses (with MAC) + +**Run manually:** +```bash +cd /mnt/user/appdata/netdisco-netbox-sync +docker compose run --rm netdisco-netbox-sync +``` --- @@ -354,4 +394,3 @@ A dedicated service account `agent` was created for automated tools: | MikroTik Switch | N/A | No SSH (SwOS) | - | **Credentials:** See docs/AGENT-CREDENTIALS.md (gitignored, local only) - diff --git a/docs/06-CHANGELOG.md b/docs/06-CHANGELOG.md index e742a07..71be741 100644 --- a/docs/06-CHANGELOG.md +++ b/docs/06-CHANGELOG.md @@ -1,3 +1,35 @@ +## 2026-01-23 - NetBox Discovery (Diode) Setup & Slurp'it Removal + +### Slurp'it Removal +- [SLURPIT] Removed entire Slurp'it stack (portal, scanner, scraper, warehouse, mariadb, mongodb) +- [SLURPIT] Decision: Use NetBox Discovery + NetDisco instead for better NetBox integration + +### NetBox Discovery (Diode) Installation +- [DIODE] Installed Diode server stack via quickstart.sh +- [DIODE] Containers: ingress-nginx, diode-auth, diode-ingester, diode-reconciler, hydra, postgres, redis +- [NETBOX] Installed netboxlabs-diode-netbox-plugin via custom Dockerfile +- [NETBOX] Ran plugin migrations for netbox_diode_plugin +- [DIODE] Configured Nginx with public /auth/introspect endpoint for NetBox plugin +- [DIODE] Connected NetBox to diode_default network for inter-container communication + +### Discovery Agent Setup +- [DIODE] Deployed orb-agent container for network discovery +- [DIODE] Configured network_discovery policy: 192.168.31.0/24, ports 22/80/161/443 +- [DIODE] Schedule: Every 30 minutes (*/30 * * * *) +- [DIODE] Successfully discovered 26 hosts on first scan + +### NetDisco to NetBox Sync +- [SYNC] Created Python sync script at /mnt/user/appdata/netdisco-netbox-sync/ +- [SYNC] Uses Diode SDK to ingest devices and IP addresses from NetDisco +- [SYNC] Syncs: 4 devices (with vendor, model, OS) and 42 ARP entries (with MAC) +- [SYNC] Containerized with Docker for easy deployment + +### Documentation +- [DOCS] Updated 00-CURRENT-STATE.md with new discovery architecture +- [DOCS] Removed Slurp'it references, added Diode and sync script documentation + +--- + ## 2026-01-22 - NetBox Migration to Shared PostgreSQL ### Database Consolidation