jazzymc/infrastructure
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: Update infrastructure state with diagrams and cleanup
All checks were successful
ci/woodpecker/push/woodpecker Pipeline was successful
Details

Browse Source
This commit is contained in:
XTRM-Unraid
2026-01-23 21:51:35 +02:00
parent 58cbc0d6a8
commit d8307cfdf3
2 changed files with 209 additions and 286 deletions
Download Patch File Download Diff File Expand all files Collapse all files

467
docs/00-CURRENT-STATE.md
View File

@@ -11,13 +11,12 @@
graph TB graph TB
subgraph Internet subgraph Internet
WAN["WAN: 62.73.120.142"] WAN["WAN: 62.73.120.142"]
DNS_EXT["dns.xtrm-lab.org<br/>DoH/DoT/DoQ"]
end end
subgraph MikroTik["MikroTik hAP ax³ (192.168.31.1)"] subgraph MikroTik["MikroTik hAP ax³ (192.168.31.1)"]
ROUTER["RouterOS 7.20.6"] ROUTER["RouterOS 7.20.6"]
subgraph MK_Containers["Docker Containers"] subgraph MK_Containers["Docker Containers"]
AGH_MK["AdGuard Home<br/>172.17.0.5:5355<br/>PRIMARY DNS"] AGH_MK["AdGuard Home<br/>172.17.0.5:5355<br/>dns.xtrm-lab.org"]
TS["Tailscale<br/>172.17.0.4"] TS["Tailscale<br/>172.17.0.4"]
end end
end end
@@ -31,6 +30,10 @@ graph TB
end end
subgraph Unraid["Unraid Server (192.168.31.2)"] subgraph Unraid["Unraid Server (192.168.31.2)"]
subgraph SharedServices["Shared Services"]
POSTGRES["PostgreSQL 17<br/>172.18.0.13"]
REDIS["Redis<br/>172.18.0.14"]
end
subgraph Core["Core Services"] subgraph Core["Core Services"]
TRAEFIK["Traefik<br/>172.18.0.3"] TRAEFIK["Traefik<br/>172.18.0.3"]
HOMARR["Homarr<br/>172.18.0.4"] HOMARR["Homarr<br/>172.18.0.4"]
@@ -40,171 +43,80 @@ graph TB
VAULT["Vaultwarden<br/>172.18.0.15"] VAULT["Vaultwarden<br/>172.18.0.15"]
end end
subgraph DNS_Unraid["DNS Services"] subgraph DNS_Unraid["DNS Services"]
AGH_UR["AdGuard Home<br/>192.168.31.4:53<br/>SECONDARY DNS"] AGH_UR["AdGuard Home<br/>192.168.31.4<br/>dns2.xtrm-lab.org"]
UNBOUND["Unbound<br/>192.168.31.5"]
end end
subgraph DevOps["DevOps"] subgraph DevOps["DevOps"]
GITEA["Gitea<br/>172.18.0.31"] GITEA["Gitea<br/>172.18.0.31"]
WOODPECKER["Woodpecker CI<br/>172.18.0.32"] WOODPECKER["Woodpecker CI<br/>172.18.0.32"]
end end
subgraph Monitoring["Monitoring"] subgraph NetBoxStack["Network Inventory"]
UPTIME["Uptime Kuma<br/>172.18.0.20"]
NETBOX["NetBox<br/>172.18.0.61"] NETBOX["NetBox<br/>172.18.0.61"]
DIODE["NetBox Discovery<br/>172.24.0.10"] DIODE["Diode Stack<br/>172.18.0.70-74"]
NETDISCO["NetDisco<br/>172.18.0.41-42"]
end end
subgraph Media["Media"]
PLEX["Plex"]
NEXTCLOUD["Nextcloud<br/>172.18.0.24"]
end
end
subgraph LAN["LAN Devices (192.168.31.x)"]
CLIENTS["Clients"]
end end
WAN --> ROUTER WAN --> ROUTER
DNS_EXT --> ROUTER
ROUTER --> AGH_MK ROUTER --> AGH_MK
ROUTER --> TS
ROUTER --> SW ROUTER --> SW
SW --> Unraid SW --> Unraid
SW --> AP SW --> AP
AP --> CLIENTS AGH_MK -.->|sync| AGH_UR
SW --> CLIENTS
AGH_MK -.->|"Upstream DoH"| QUAD9["Quad9 DNS"]
AGH_UR -.->|"Upstream DoH"| QUAD9
CLIENTS -->|"DNS Queries"| AGH_MK
CLIENTS -.->|"Failover"| AGH_UR
``` ```
--- ---
## MikroTik hAP ax³ Router (192.168.31.1) ## Service Architecture Diagram
| Parameter | Value | ```mermaid
|-----------|-------| flowchart TB
| RouterOS Version | 7.20.6 (stable) | subgraph SharedServices["Shared Infrastructure"]
| WAN IP (Static) | 62.73.120.142 | PG[("PostgreSQL 17<br/>172.18.0.13")]
| LAN Subnet | 192.168.31.0/24 | RD[("Redis<br/>172.18.0.14")]
| Docker Bridge | 172.17.0.0/24 | end
| SSH Access | Port 2222, user: jazzymc |
**Interfaces:** subgraph NetBoxStack["Network Inventory Stack"]
- `ether1` - WAN (62.73.120.142/23) NB["NetBox<br/>172.18.0.61"]
- `bridge` - LAN (192.168.31.1/24) NBW["NetBox Worker<br/>172.18.0.62"]
- `docker-bridge` - Container network (172.17.0.1/24) NBC[("Redis Cache<br/>172.18.0.64")]
- `back-to-home-vpn` - WireGuard VPN (192.168.216.1/24)
subgraph Diode["Diode Discovery"]
DI["Ingress<br/>172.18.0.70"]
DIN["Ingester<br/>172.18.0.71"]
DRE["Reconciler<br/>172.18.0.72"]
DHY["Hydra<br/>172.18.0.73"]
DAU["Auth<br/>172.18.0.74"]
DAG["Agent<br/>host network"]
end
subgraph NetDisco["NetDisco"]
NDW["Web<br/>172.18.0.41"]
NDB["Backend<br/>172.18.0.42"]
end
end
### Running Containers on MikroTik subgraph DevOps["DevOps Stack"]
GIT["Gitea<br/>172.18.0.31"]
WPS["Woodpecker Server<br/>172.18.0.32"]
WPA["Woodpecker Agent<br/>172.18.0.33"]
end
| Container | IP | Storage | Purpose | PG --> NB
|-----------|-----|---------|---------| PG --> GIT
| tailscale | 172.17.0.4 | usb1/tailscale/root | Tailscale VPN client | PG --> NDW
| adguardhome | 172.17.0.5 | disk1/agh-root + usb1 mount | DNS with DoH/DoT/DoQ | PG --> DRE
PG --> DHY
### AdGuard Home (MikroTik) - PRIMARY DNS RD --> DIN
RD --> DRE
| Service | Port | Protocol | Status | RD --> NBW
|---------|------|----------|--------| NBC --> NB
| DNS | 5355 (NAT from 53) | UDP/TCP | Active |
| Web UI | 80 | HTTP | Active | DAG -->|gRPC| DI
| DoH | 443 | HTTPS | Active | DI --> DIN
| DoT | 853 | TCP | Active | DIN --> RD
| DoQ | 8853 | UDP | Active | DRE --> NB
NDB --> NDW
**Configuration:** ```
- Upstream: Quad9 DoH (https://dns10.quad9.net/dns-query)
- TLS Certificate: Let's Encrypt wildcard (*.xtrm-lab.org)
- Server Name: dns.xtrm-lab.org
- Certificate Expiry: 2026-04-02
- Credentials: jazzymc / 7RqWElENNbZnPW
**Persistence:** root-dir on disk1 + data mount on usb1 (survives container restart)
---
## MikroTik CSS326-24G-2S+ Switch (192.168.31.9)
| Parameter | Value |
|-----------|-------|
| Role | Managed Layer 2 Switch |
| Ports | 24x Gigabit + 2x SFP |
| OS | SwOS |
| Web UI | https://sw.xtrm-lab.org |
---
## MikroTik cAP ac (192.168.31.6)
| Parameter | Value |
|-----------|-------|
| Role | CAPsMAN Managed Access Point |
| RouterOS Version | 7.20.1 (stable) |
| Identity | CAP XL ac |
---
## Unraid Server (192.168.31.2)
**Tailscale IP:** 100.100.208.70
**SSH Access:** `ssh -i ~/.ssh/id_ed25519_unraid root@192.168.31.2 -p 422`
### Docker Networks
| Network | Subnet | Purpose |
|---------|--------|---------|
| br0 | 192.168.31.0/24 | LAN macvlan (AdGuard Home) |
| dockerproxy | 172.18.0.0/16 | Traefik-accessible services |
| diode_default | 172.24.0.0/16 | NetBox Discovery (Diode) |
| bridge | 172.17.0.0/16 | Default Docker bridge |
### Key Services
| Service | Container | IP | External URL |
|---------|-----------|---|--------------|
| **Core** ||||
| Reverse Proxy | traefik | 172.18.0.3 | traefik.xtrm-lab.org |
| Dashboard | homarr | 172.18.0.4 | xtrm-lab.org |
| **Security** ||||
| Identity Provider | authentik | 172.18.0.11 | auth.xtrm-lab.org |
| Password Manager | vaultwarden | 172.18.0.15 | vault.xtrm-lab.org |
| **DNS** ||||
| AdGuard Home | adguardhome | 192.168.31.4 | - |
| Unbound | unbound | 192.168.31.5 | - |
| **DevOps** ||||
| Git Server | gitea | 172.18.0.31 | git.xtrm-lab.org |
| CI/CD Server | woodpecker-server | 172.18.0.32 | ci.xtrm-lab.org |
| **Monitoring** ||||
| Uptime Kuma | UptimeKuma | 172.18.0.20 | uptime.xtrm-lab.org |
| NetBox | netbox | 172.18.0.61 | netbox.xtrm-lab.org |
| **Media** ||||
| Plex | plex | host | plex.xtrm-lab.org |
| Nextcloud | Nextcloud | 172.18.0.24 | nextcloud.xtrm-lab.org |
| **Remote Access** ||||
| RustDesk | rustdesk-hbbs/hbbr | bridge | rustdesk.xtrm-lab.org |
### AdGuard Home (Unraid) - SECONDARY DNS
| Setting | Value |
|---------|-------|
| IP Address | 192.168.31.4 |
| Network | br0 (macvlan) |
| Web UI | http://192.168.31.4:3000 |
| DNS | 192.168.31.4:53 |
| DoT | 192.168.31.4:853 |
| Credentials | jazzymc / 7RqWElENNbZnPW |
**Configuration (synced with MikroTik):**
- Upstream: Quad9 DoH
- TLS Certificate: Let's Encrypt wildcard
- 6 Clients configured
- Custom filtering rules (SentinelOne, Jamf)
**Data Location:** /mnt/user/appdata/adguardhome/
**Stopped Services:**
- binhex-official-pihole (replaced by AdGuard Home)
- nebula-sync (incompatible with AdGuard Home)
--- ---
@@ -213,184 +125,167 @@ graph TB
```mermaid ```mermaid
flowchart TB flowchart TB
subgraph External["External Access"] subgraph External["External Access"]
DOH["DoH: https://dns.xtrm-lab.org/dns-query"] DOH1["DoH: dns.xtrm-lab.org"]
DOT["DoT: tls://dns.xtrm-lab.org:853"] DOT1["DoT: dns.xtrm-lab.org:853"]
DOQ["DoQ: quic://dns.xtrm-lab.org:8853"] DOH2["DoH: dns2.xtrm-lab.org"]
DOT2["DoT: dns2.xtrm-lab.org:853"]
end end
subgraph MikroTik["MikroTik Router"] subgraph MikroTik["MikroTik Router"]
NAT["NAT: 53 → 5355"] NAT["NAT: 53 → 5355"]
AGH1["AdGuard Home<br/>172.17.0.5:5355<br/>PRIMARY"] AGH1["AdGuard Home<br/>PRIMARY"]
end end
subgraph Unraid["Unraid Server"] subgraph Unraid["Unraid Server"]
AGH2["AdGuard Home<br/>192.168.31.4:53<br/>SECONDARY"] AGH2["AdGuard Home<br/>SECONDARY"]
end
subgraph Sync["Configuration Sync"]
AGHSYNC["adguardhome-sync<br/>Every 30 min"]
end end
subgraph Upstream["Upstream DNS"] subgraph Upstream["Upstream DNS"]
Q9["Quad9 DoH<br/>dns10.quad9.net"] Q9["Quad9 DoH"]
end end
subgraph Clients["LAN Clients"] DOH1 --> AGH1
C1["IPhone Dancho"] DOT1 --> AGH1
C2["IPhone Kimi"] DOH2 --> AGH2
C3["Laptop Dari"] DOT2 --> AGH2
C4["Laptop Kimi"]
C5["PC Dancho"]
C6["ROG Ally Teodor"]
end
External --> MikroTik
Clients -->|"Primary"| NAT
NAT --> AGH1 NAT --> AGH1
Clients -.->|"Failover"| AGH2
AGH1 --> Q9 AGH1 --> Q9
AGH2 --> Q9 AGH2 --> Q9
AGH1 <-.->|sync| AGHSYNC
AGHSYNC <-.->|sync| AGH2
``` ```
--- ---
## Configured Clients (Both AdGuard Instances) ## Container Summary
| Client | MAC Address | Tags | ### Shared Services
|--------|-------------|------|
| IPhone (Dancho) | f2:b8:14:61:c8:27 | - |
| IPhone (Kimi) | 2a:2b:ba:86:d4:af | user_child |
| Laptop (Dari) | 34:f6:4b:b3:14:83 | user_child |
| Laptop (Kimi) | 90:91:64:70:0d:86 | user_child |
| PC (Dancho) | 70:85:c2:75:64:e5 | - |
| ROG Ally (Teodor) | cc:5e:f8:d3:37:d3 | user_child |
--- | Container | IP | Purpose | Consumers |
|-----------|-----|---------|-----------|
| postgresql17 | 172.18.0.13 | PostgreSQL 17 | NetBox, Gitea, NetDisco, Authentik, Diode |
| Redis | 172.18.0.14 | Redis Queue | Diode, NetBox Worker |
## Custom Filtering Rules ### Network Inventory (NetBox & Discovery)
```
||dv-eu-prod.sentinelone.net^
||euce1-soc360.sentinelone.net^
||ampeco.jamfcloud.com^
||*.jamfcloud.com^
```
---
## NAT/Port Forwarding (MikroTik)
| Rule | Protocol | Port | Destination | Purpose |
|------|----------|------|-------------|---------|
| HTTP | TCP | 80 | 192.168.31.2:8001 | Traefik |
| HTTPS | TCP | 443 | 192.168.31.2:44301 | Traefik |
| DNS UDP | UDP | 53→5355 | 172.17.0.5 | AdGuard Home |
| DNS TCP | TCP | 53→5355 | 172.17.0.5 | AdGuard Home |
| DoT | TCP | 853 | 172.17.0.5 | DNS over TLS |
| DoQ | UDP | 8853 | 172.17.0.5 | DNS over QUIC |
| Plex | TCP | 32400 | 192.168.31.2 | Plex Media |
| RustDesk | TCP/UDP | 21115-21119 | 192.168.31.2 | RustDesk |
---
## Reference Documents
- [Phase 1: DNS Portability](./01-PHASE1-DNS-PORTABILITY.md)
- [Phase 7: Gitea GitOps](./08-PHASE7-GITEA-GITOPS.md)
- [Changelog](./06-CHANGELOG.md)
---
## Network Discovery & Management
### NetBox (IPAM/DCIM)
| Container | IP | Purpose | | Container | IP | Purpose |
|-----------|-----|---------| |-----------|-----|---------|
| netbox | 172.18.0.61 | Web UI (netbox.xtrm-lab.org) | | netbox | 172.18.0.61 | Web UI |
| netbox-postgres | - | Database | | netbox-worker | 172.18.0.62 | Background tasks |
| netbox-redis | - | Cache | | netbox-redis-cache | 172.18.0.64 | Query cache |
| netbox-redis-cache | - | Redis cache | | diode-ingress | 172.18.0.70 | API Gateway (nginx) |
| netbox-worker | - | Background tasks | | diode-ingester | 172.18.0.71 | Data ingestion |
| diode-reconciler | 172.18.0.72 | NetBox sync |
**Plugins Installed:** | diode-hydra | 172.18.0.73 | OAuth2 (Ory Hydra) |
- netboxlabs-diode-netbox-plugin (NetBox Discovery integration) | diode-auth | 172.18.0.74 | Token service |
| diode-agent | host | Network scanner |
### NetBox Discovery (Diode) | netdisco-web | 172.18.0.41 | Web UI |
NetBox Labs Diode provides automated network discovery and data ingestion into NetBox.
| Container | IP | Purpose |
|-----------|-----|---------|
| diode-ingress-nginx-1 | 172.24.0.10 | API Gateway |
| diode-diode-auth-1 | - | OAuth2 authentication |
| diode-diode-ingester-1 | - | Data ingestion service |
| diode-diode-reconciler-1 | - | Data reconciliation |
| diode-hydra-1 | - | OAuth2 provider (Ory Hydra) |
| diode-postgres-1 | - | Database |
| diode-redis-1 | - | Cache |
| diode-discovery-agent | host network | Network scanner (orb-agent) |
**Data Location:** /mnt/user/appdata/diode/
**Discovery Agent Configuration:**
- Schedule: Every 30 minutes
- Target: 192.168.31.0/24
- Ports scanned: 22, 80, 161, 443
- Site: Home
**OAuth2 Credentials:**
- diode-ingest: For data ingestion
- netbox-to-diode: For NetBox plugin
- diode-to-netbox: For reconciler
### NetDisco
NetDisco provides SNMP-based network discovery and ARP table collection.
| Container | IP | Purpose |
|-----------|-----|---------|
| netdisco-web | 172.18.0.41 | Web UI (netdisco.xtrm-lab.org) |
| netdisco-backend | 172.18.0.42 | SNMP poller | | netdisco-backend | 172.18.0.42 | SNMP poller |
**Database:** postgresql17 (shared) ### Infrastructure
- Database: netdisco_db
- User: netdisco_user
**Discovered Data:** | Container | IP | Purpose |
- 4 SNMP-enabled devices |-----------|-----|---------|
- 42 ARP entries (all network hosts) | traefik | 172.18.0.3 | Reverse proxy |
| dockersocket | - | Docker socket proxy |
| adguardhome | 192.168.31.4 | DNS (Secondary) |
| adguardhome-sync | 172.18.0.65 | Config sync |
### NetDisco to NetBox Sync ### DevOps
A scheduled sync script pushes NetDisco data to NetBox via Diode. | Container | IP | Purpose |
|-----------|-----|---------|
| gitea | 172.18.0.31 | Git hosting |
| woodpecker-server | 172.18.0.32 | CI/CD server |
| woodpecker-agent | 172.18.0.33 | CI/CD agent |
**Location:** /mnt/user/appdata/netdisco-netbox-sync/ ### Security
| File | Purpose | | Container | IP | Purpose |
|------|---------| |-----------|-----|---------|
| sync.py | Python sync script | | authentik | 172.18.0.11 | Identity provider |
| Dockerfile | Container build file | | authentik-worker | - | Background tasks |
| docker-compose.yml | Deployment config | | vaultwarden | 172.18.0.15 | Password manager |
**Sync Configuration:** ### Monitoring
- Source: NetDisco PostgreSQL database
- Target: NetBox via Diode gRPC API
- Data synced: Devices (with vendor, model, OS) and IP addresses (with MAC)
**Run manually:** | Container | IP | Purpose |
```bash |-----------|-----|---------|
cd /mnt/user/appdata/netdisco-netbox-sync | UptimeKuma | 172.18.0.20 | Uptime monitoring |
docker compose run --rm netdisco-netbox-sync | Uptime-Kuma-API | 172.18.0.18 | REST API |
``` | AutoKuma | 172.18.0.19 | Auto-monitor creation |
| NetAlertX | - | Network alerting |
| speedtest-tracker | - | Speed tests |
--- ---
## Agent Service Account ## RAM Usage (as of 2026-01-23)
A dedicated service account `agent` was created for automated tools: **Total: 15GB | Used: 12GB (80%) | Available: 2.7GB**
| Device | Username | Auth Method | Port | | Container | RAM | % |
|--------|----------|-------------|------| |-----------|-----|---|
| Unraid | agent | SSH Key + Password | 422 | | unimus | 1.62 GB | 10.5% |
| MikroTik Router | agent | SSH Key | 2222 | | karakeep | 664 MB | 4.2% |
| MikroTik AP | agent | Password | 2222 | | netdisco-web | 534 MB | 3.4% |
| MikroTik Switch | N/A | No SSH (SwOS) | - | | n8n | 293 MB | 1.9% |
| netdisco-backend | 281 MB | 1.8% |
| netbox-worker | 230 MB | 1.5% |
| plex | 161 MB | 1.0% |
| postgresql17 | 136 MB | 0.9% |
| All others | <130 MB each | <1% |
**Credentials:** See docs/AGENT-CREDENTIALS.md (gitignored, local only) ---
## Removed Services (2026-01-23)
The following services were removed as redundant (AdGuard Home provides DoH/DoT natively):
| Service | Reason |
|---------|--------|
| Unbound | AdGuard uses upstream DoH directly |
| DoH-Server | AdGuard has built-in DoH |
| stunnel-dot | AdGuard has built-in DoT |
| Pangolin | Not in use |
---
## External URLs
| Service | URL |
|---------|-----|
| Dashboard | https://xtrm-lab.org |
| Traefik | https://traefik.xtrm-lab.org |
| Authentik | https://auth.xtrm-lab.org |
| Gitea | https://git.xtrm-lab.org |
| Woodpecker CI | https://ci.xtrm-lab.org |
| NetBox | https://netbox.xtrm-lab.org |
| NetDisco | https://netdisco.xtrm-lab.org |
| Uptime Kuma | https://uptime.xtrm-lab.org |
| Plex | https://plex.xtrm-lab.org |
| Nextcloud | https://cloud.xtrm-lab.org |
| Vaultwarden | https://vault.xtrm-lab.org |
| DNS (Primary) | dns.xtrm-lab.org (MikroTik) |
| DNS (Secondary) | dns2.xtrm-lab.org (Unraid) |
---
## FolderView2 Categories
| Category | Containers |
|----------|------------|
| Infrastructure | traefik, dockersocket, adguardhome, adguardhome-sync |
| Security | authentik, authentik-worker, vaultwarden |
| Monitoring | UptimeKuma, Uptime-Kuma-API, AutoKuma, NetAlertX, speedtest-tracker |
| DevOps | gitea, woodpecker-server, woodpecker-agent, postgresql17, Redis |
| Media | plex, Libation, transmission |
| Storage/Backup | rustfs, UrBackup, TimeMachine, Nextcloud |
| Productivity | actual-budget, n8n, karakeep, homarr |
| Smart Home | HomeAssistant_inabox |
| Remote Access | rustdesk-hbbs, rustdesk-hbbr |
| Management | portainer, unimus |
| Network Inventory | netbox, netbox-worker, netbox-redis-cache, diode-*, netdisco-* |

28
docs/06-CHANGELOG.md
View File

@@ -251,3 +251,31 @@ See git history for earlier changes.
- netbox-redis (was 172.18.0.63) - netbox-redis (was 172.18.0.63)
--- ---
## 2026-01-23 - Service Cleanup & Documentation Update
### Services Removed
- [REMOVED] Unbound - redundant (AdGuard has upstream DoH)
- [REMOVED] DoH-Server - redundant (AdGuard has built-in DoH)
- [REMOVED] stunnel-dot - redundant (AdGuard has built-in DoT)
- [REMOVED] Pangolin - not in use
### DNS Configuration
- [CONFIG] Unraid AdGuard: dns2.xtrm-lab.org (was dns.xtrm-lab.org)
- [CONFIG] MikroTik AdGuard: dns.xtrm-lab.org (primary)
### Container Management
- [LABELS] Added net.unraid.docker.managed to all containers
- [LABELS] Added WebUI URLs to containers with web interfaces
- [LABELS] Updated icons to PNG format (from SVG)
### FolderView2
- [CATEGORY] Added "Network Inventory" for NetBox/Diode/NetDisco
### Documentation
- [DOCS] Updated 00-CURRENT-STATE.md with current architecture
- [DOCS] Added Mermaid diagrams for network topology
- [DOCS] Added RAM usage statistics
- [DOCS] Documented removed services
---