jazzymc/infrastructure
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: Update for NetBox Discovery (Diode) setup and Slurpit removal

Browse Source 
- Removed Slurpit section from current state (stack removed)
- Added NetBox Discovery (Diode) architecture documentation
- Added NetDisco to NetBox sync script documentation
- Updated network diagram with Diode components
- Added changelog entries for 2026-01-23

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
XTRM-Unraid
2026-01-23 17:46:56 +02:00
parent 682aceb8e0
commit e870bddac8
2 changed files with 102 additions and 31 deletions
Download Patch File Download Diff File Expand all files Collapse all files

101
docs/00-CURRENT-STATE.md
View File

@@ -1,6 +1,6 @@
# Infrastructure Current State: xtrm-lab.org
## Document Updated: 2026-01-22
## Document Updated: 2026-01-23
## Target Domain: xtrm-lab.org
---
@@ -49,7 +49,8 @@ graph TB
end
subgraph Monitoring["Monitoring"]
UPTIME["Uptime Kuma<br/>172.18.0.20"]
NETBOX["NetBox<br/>172.24.0.5"]
NETBOX["NetBox<br/>172.18.0.61"]
DIODE["NetBox Discovery<br/>172.24.0.10"]
end
subgraph Media["Media"]
PLEX["Plex"]
@@ -113,7 +114,7 @@ graph TB
**Configuration:**
- Upstream: Quad9 DoH (https://dns10.quad9.net/dns-query)
- TLS Certificate: Let's Encrypt wildcard (\*.xtrm-lab.org)
- TLS Certificate: Let's Encrypt wildcard (*.xtrm-lab.org)
- Server Name: dns.xtrm-lab.org
- Certificate Expiry: 2026-04-02
- Credentials: jazzymc / 7RqWElENNbZnPW
@@ -154,7 +155,7 @@ graph TB
|---------|--------|---------|
| br0 | 192.168.31.0/24 | LAN macvlan (AdGuard Home) |
| dockerproxy | 172.18.0.0/16 | Traefik-accessible services |
| netbox | 172.24.0.0/16 | NetBox stack |
| diode_default | 172.24.0.0/16 | NetBox Discovery (Diode) |
| bridge | 172.17.0.0/16 | Default Docker bridge |
### Key Services
@@ -175,7 +176,7 @@ graph TB
| CI/CD Server | woodpecker-server | 172.18.0.32 | ci.xtrm-lab.org |
| **Monitoring** ||||
| Uptime Kuma | UptimeKuma | 172.18.0.20 | uptime.xtrm-lab.org |
| NetBox | netbox | 172.24.0.5 | netbox.xtrm-lab.org |
| NetBox | netbox | 172.18.0.61 | netbox.xtrm-lab.org |
| **Media** ||||
| Plex | plex | host | plex.xtrm-lab.org |
| Nextcloud | Nextcloud | 172.18.0.24 | nextcloud.xtrm-lab.org |
@@ -298,47 +299,86 @@ flowchart TB
## Network Discovery & Management
### Slurp'it Stack
### NetBox (IPAM/DCIM)
| Container | IP | Purpose |
|-----------|-----|---------|
| slurpit-portal | 172.18.0.129 | Web UI (slurpit.xtrm-lab.org) |
| slurpit-scanner | 172.25.0.5 | SNMP network scanner |
| slurpit-scraper | 172.25.0.3 | Device data collector |
| slurpit-warehouse | 172.25.0.4 | Data storage API |
| slurpit-mariadb | 172.25.0.2 | Portal database |
| slurpit-mongodb | 172.25.0.6 | Discovery database |
| netbox | 172.18.0.61 | Web UI (netbox.xtrm-lab.org) |
| netbox-postgres | - | Database |
| netbox-redis | - | Cache |
| netbox-redis-cache | - | Redis cache |
| netbox-worker | - | Background tasks |
**Status:** Operational
**Discovered Devices:** 1 (MikroTik Router)
**SNMP Communities:** public, netdisco
**Plugins Installed:**
- netboxlabs-diode-netbox-plugin (NetBox Discovery integration)
**Configuration:**
- SNMP v2c credentials configured
- Scan target: 192.168.31.0/24
- NetBox integration: Enabled (plugin_online: 1)
### NetBox Discovery (Diode)
**Pending Tasks:**
- Add SSH credentials to Vault for device scraping
- Troubleshoot SNMP discovery of switch and AP
NetBox Labs Diode provides automated network discovery and data ingestion into NetBox.
| Container | IP | Purpose |
|-----------|-----|---------|
| diode-ingress-nginx-1 | 172.24.0.10 | API Gateway |
| diode-diode-auth-1 | - | OAuth2 authentication |
| diode-diode-ingester-1 | - | Data ingestion service |
| diode-diode-reconciler-1 | - | Data reconciliation |
| diode-hydra-1 | - | OAuth2 provider (Ory Hydra) |
| diode-postgres-1 | - | Database |
| diode-redis-1 | - | Cache |
| diode-discovery-agent | host network | Network scanner (orb-agent) |
**Data Location:** /mnt/user/appdata/diode/
**Discovery Agent Configuration:**
- Schedule: Every 30 minutes
- Target: 192.168.31.0/24
- Ports scanned: 22, 80, 161, 443
- Site: Home
**OAuth2 Credentials:**
- diode-ingest: For data ingestion
- netbox-to-diode: For NetBox plugin
- diode-to-netbox: For reconciler
### NetDisco
NetDisco provides SNMP-based network discovery and ARP table collection.
| Container | IP | Purpose |
|-----------|-----|---------|
| netdisco-web | 172.18.0.41 | Web UI (netdisco.xtrm-lab.org) |
| netdisco-backend | 172.18.0.42 | SNMP poller |
### NetBox (IPAM/DCIM)
**Database:** postgresql17 (shared)
- Database: netdisco_db
- User: netdisco_user
| Container | IP | Purpose |
|-----------|-----|---------|
| netbox | 172.24.0.5 | Web UI (netbox.xtrm-lab.org) |
| netbox-postgres | 172.24.0.4 | Database |
| netbox-redis | 172.24.0.2 | Cache |
| netbox-worker | 172.24.0.6 | Background tasks |
**Discovered Data:**
- 4 SNMP-enabled devices
- 42 ARP entries (all network hosts)
**NetBox Slurp'it Plugin:** Installed and configured
### NetDisco to NetBox Sync
A scheduled sync script pushes NetDisco data to NetBox via Diode.
**Location:** /mnt/user/appdata/netdisco-netbox-sync/
| File | Purpose |
|------|---------|
| sync.py | Python sync script |
| Dockerfile | Container build file |
| docker-compose.yml | Deployment config |
**Sync Configuration:**
- Source: NetDisco PostgreSQL database
- Target: NetBox via Diode gRPC API
- Data synced: Devices (with vendor, model, OS) and IP addresses (with MAC)
**Run manually:**
```bash
cd /mnt/user/appdata/netdisco-netbox-sync
docker compose run --rm netdisco-netbox-sync
```
---
@@ -354,4 +394,3 @@ A dedicated service account `agent` was created for automated tools:
| MikroTik Switch | N/A | No SSH (SwOS) | - |
**Credentials:** See docs/AGENT-CREDENTIALS.md (gitignored, local only)

32
docs/06-CHANGELOG.md
View File

@@ -1,3 +1,35 @@
## 2026-01-23 - NetBox Discovery (Diode) Setup & Slurp'it Removal
### Slurp'it Removal
- [SLURPIT] Removed entire Slurp'it stack (portal, scanner, scraper, warehouse, mariadb, mongodb)
- [SLURPIT] Decision: Use NetBox Discovery + NetDisco instead for better NetBox integration
### NetBox Discovery (Diode) Installation
- [DIODE] Installed Diode server stack via quickstart.sh
- [DIODE] Containers: ingress-nginx, diode-auth, diode-ingester, diode-reconciler, hydra, postgres, redis
- [NETBOX] Installed netboxlabs-diode-netbox-plugin via custom Dockerfile
- [NETBOX] Ran plugin migrations for netbox_diode_plugin
- [DIODE] Configured Nginx with public /auth/introspect endpoint for NetBox plugin
- [DIODE] Connected NetBox to diode_default network for inter-container communication
### Discovery Agent Setup
- [DIODE] Deployed orb-agent container for network discovery
- [DIODE] Configured network_discovery policy: 192.168.31.0/24, ports 22/80/161/443
- [DIODE] Schedule: Every 30 minutes (*/30 * * * *)
- [DIODE] Successfully discovered 26 hosts on first scan
### NetDisco to NetBox Sync
- [SYNC] Created Python sync script at /mnt/user/appdata/netdisco-netbox-sync/
- [SYNC] Uses Diode SDK to ingest devices and IP addresses from NetDisco
- [SYNC] Syncs: 4 devices (with vendor, model, OS) and 42 ARP entries (with MAC)
- [SYNC] Containerized with Docker for easy deployment
### Documentation
- [DOCS] Updated 00-CURRENT-STATE.md with new discovery architecture
- [DOCS] Removed Slurp'it references, added Diode and sync script documentation
---
## 2026-01-22 - NetBox Migration to Shared PostgreSQL
### Database Consolidation