diff --git a/docs/incidents/2026-01-31-network-connectivity.md b/docs/incidents/2026-01-31-network-connectivity.md index ee0e18a..8a8c169 100644 --- a/docs/incidents/2026-01-31-network-connectivity.md +++ b/docs/incidents/2026-01-31-network-connectivity.md @@ -1,43 +1,41 @@ # Network Connectivity Issue **Date:** 2026-01-31 -**Status:** OPEN +**Status:** RESOLVED **Severity:** Medium -## Symptoms +## Symptoms (Initial) - DNS resolution failing (8.8.8.8 timeout) - Cannot connect to local services via 192.168.31.2 -- curl to Gitea (port 3005) returns HTTP 000 - Git push to Gitea failing -## Affected Services +## Root Causes Identified -- Git push to git.xtrm-lab.org -- Potentially other services using external DNS +1. **DHCP DNS misconfiguration** - All VLANs were pushing 8.8.8.8 instead of gateway IP +2. **Legacy IP references** - Old 192.168.31.x IPs no longer valid after VLAN migration +3. **Macvlan limitation** - AdGuard on macvlan (br0) unreachable from host -## Observations +## Resolution -- Gitea container is running (docker ps confirms) -- Ports are mapped correctly (0.0.0.0:3005->3000) -- Internal Docker IP: 172.18.0.31 -- Connections from Unraid shell to local IP timing out +1. Updated MikroTik DHCP network settings: + - VLAN 10: dns-server=192.168.10.1 + - VLAN 20: dns-server=192.168.20.1 + - VLAN 25: dns-server=192.168.25.1 + - VLAN 30: dns-server=192.168.30.1 -## Possible Causes +2. Renewed DHCP lease on Unraid: `dhcpcd -n br0` -- Docker bridge/macvlan network issue -- iptables/firewall blocking internal traffic -- VLAN configuration affecting local routing -- DNS upstream issue +3. Updated NetAlertX config with correct IP (192.168.10.20) -## Workaround +## Verification -- Pending git commit saved locally: `dbe4699` -- Push when network is restored: `cd /tmp/infrastructure && git push` +- DNS now resolves via 192.168.10.1 (MikroTik gateway → AdGuard) +- Git push to Gitea working +- All services accessible -## Next Steps +## Lessons Learned -- [ ] Check iptables rules -- [ ] Verify Docker network configuration -- [ ] Check AdGuard DNS upstream settings -- [ ] Test connectivity from different network paths +- After VLAN migration, update all hardcoded IPs +- DHCP DNS should point to gateway, not external servers +- Macvlan containers need shim interface for host communication