# Incident: DNS Outbound Blocked After MikroTik Restart

**Date:** 2026-01-25  
**Duration:** ~1 hour  
**Severity:** High (Complete DNS failure)  
**Status:** Resolved  

---

## Summary

After MikroTik router restart, AdGuardHome could not reach upstream DNS servers, causing complete DNS resolution failure for all network clients.

## Symptoms

- AdGuardHome container showing timeout errors to upstream DNS (8.8.8.8, Quad9)
- All DNS queries failing network-wide
- ICMP (ping) to external IPs worked, but port 53 (DNS) was unreachable
- Error pattern: `i/o timeout` when querying any external DNS server

## Root Cause

MikroTik NAT rules were redirecting ALL DNS traffic from `192.168.31.0/24` to AdGuardHome, including AdGuardHome's own outbound DNS queries. This created a loop where:

1. AdGuardHome (192.168.31.4) tries to query 8.8.8.8:53
2. MikroTik NAT rule 7 intercepts this traffic (src: 192.168.31.0/24, dst-port: 53)
3. Traffic redirected back to AdGuardHome (172.17.0.5:5355)
4. Query fails with timeout

**Problematic NAT Rules:**
```
Rule 7: Force DNS to AdGuard Home
  chain=dstnat action=dst-nat to-addresses=172.17.0.5 to-ports=5355 
  protocol=udp src-address=192.168.31.0/24 dst-port=53

Rule 9: Force DNS to AdGuard Home TCP
  chain=dstnat action=dst-nat to-addresses=172.17.0.5 to-ports=5355 
  protocol=tcp src-address=192.168.31.0/24 dst-port=53
```

## Resolution

Added exception rules BEFORE the redirect rules to allow AdGuardHome to reach external DNS:

```bash
# Added via SSH to MikroTik
/ip firewall nat add chain=dstnat protocol=udp src-address=192.168.31.4 dst-port=53 action=accept comment=Allow AdGuard outbound DNS place-before=7
/ip firewall nat add chain=dstnat protocol=tcp src-address=192.168.31.4 dst-port=53 action=accept comment=Allow AdGuard outbound DNS TCP place-before=8
```

**Additional Changes:**
1. Set MikroTik DNS to use only AdGuard: `/ip dns set servers=192.168.31.4`
2. Disabled ISP DNS from DHCP: `/ip dhcp-client set 0 use-peer-dns=no`
3. Updated Unraid resolv.conf to use 192.168.31.4 (for Tailscale access)

## Final NAT Rule Order (DNS-related)

| # | Comment | Action | Details |
|---|---------|--------|---------|
| 5 | Allow AdGuard outbound DNS | accept | src=192.168.31.4, udp/53 |
| 6 | Force DNS to AdGuard Home | dst-nat | src=192.168.31.0/24 → 172.17.0.5:5355 |
| 7 | Allow AdGuard outbound DNS TCP | accept | src=192.168.31.4, tcp/53 |
| 9 | Force DNS to AdGuard Home TCP | dst-nat | src=192.168.31.0/24 → 172.17.0.5:5355 |

## Verification

After fix:
- ✓ AdGuard can resolve external DNS (google.com, github.com)
- ✓ MikroTik using only AdGuard as DNS
- ✓ Internal domains resolving (git.xtrm-lab.org, cloud.xtrm-lab.org)
- ✓ External access to services working (Gitea, Woodpecker CI)

## Lessons Learned

1. NAT redirect rules need exceptions for the DNS server itself
2. After MikroTik restart, verify DNS flow end-to-end
3. Keep exception rules BEFORE redirect rules in firewall

## Related Configuration

- AdGuard IP: 192.168.31.4 (macvlan)
- AdGuard internal: 172.17.0.5:5355
- MikroTik: 192.168.31.1
- Upstream DNS: 8.8.8.8 (temporarily, was Quad9 DoH)

---

*Incident logged by Claude Code via Tailscale connection*

---

## Additional Issue: Devices No Internet (13:25)

### Symptom
- LG TV and other devices showing no internet connection
- DNS changes not reaching devices

### Root Cause
DHCP was pushing  as DNS to clients, but MikroTik NAT rules redirect all port 53 traffic to AdGuard. This created a mismatch.

**Before:**
```
DHCP DNS-SERVER: 8.8.8.8
```

### Fix
Changed DHCP to push MikroTik as DNS (which forwards to AdGuard):

```bash
/ip dhcp-server network set 0 dns-server=192.168.31.1
```

**After:**
```
DHCP DNS-SERVER: 192.168.31.1
```

### DNS Flow (Corrected)
```
Device → 192.168.31.1 (MikroTik DNS) → 192.168.31.4 (AdGuard) → 8.8.8.8 (upstream)
```

### Device Recovery
Devices need to renew DHCP lease to get new DNS:
- Wait for lease expiry (default 10 min)
- Reconnect to WiFi
- Reboot device

---

## Additional Issue: NAT Redirect Wrong IP/Port (13:35)

### Symptom
- TV showing DNS 192.168.31.1 but no internet
- DNS queries to MikroTik timing out

### Root Cause
NAT rules were redirecting DNS to wrong destination:

**Before (WRONG):**
```
to-addresses=172.17.0.5 to-ports=5355
```

But AdGuard:
- Is on macvlan IP: 192.168.31.4 (NOT 172.17.0.5)
- Listens on port: 53 (NOT 5355)

### Fix
```bash
/ip firewall nat set [find comment="Force DNS to AdGuard Home"] to-addresses=192.168.31.4 to-ports=53
/ip firewall nat set [find comment="Force DNS to AdGuard Home TCP"] to-addresses=192.168.31.4 to-ports=53
```

**After (CORRECT):**
```
to-addresses=192.168.31.4 to-ports=53
```

### Verification
- AdGuard container querying 192.168.31.1 → SUCCESS
- MikroTik resolve command → SUCCESS

---

## Additional Issue: Netflix Blocked by AdGuard Filter (13:45)

### Symptom
- TV reports no internet even though DNS is working
- Netflix app fails to connect

### Root Cause
AdGuard DNS filter (FilterListID:1) was blocking Netflix operational domains:
- `ichnaea.netflix.com` - Netflix connectivity check
- `customerevents.netflix.com` - Netflix events
- `*.logs.netflix.com` - Netflix logging

Netflix requires these domains to function, even though they're flagged as tracking.

### Fix
Added whitelist rules to AdGuard:
```
@@||ichnaea.netflix.com^
@@||customerevents.netflix.com^
@@||logs.netflix.com^
```

### Verification
Restarted AdGuard, TV should now connect to Netflix.

---

## Root Cause Found: Asymmetric Routing (13:55)

### Problem
DNS redirect (DNAT) was causing asymmetric routing:
1. Client sends DNS to 192.168.31.1:53
2. MikroTik DNATs to 192.168.31.4:53 (AdGuard)
3. AdGuard responds DIRECTLY to client (same L2 subnet)
4. Client receives response from .4 but expected it from .1
5. Client drops response → "no internet"

### Fix
Added srcnat/masquerade rule so AdGuard sees MikroTik as source:

```bash
/ip firewall nat add chain=srcnat action=masquerade protocol=udp \
  src-address=192.168.31.0/24 dst-address=192.168.31.4 dst-port=53 \
  comment="Masquerade DNS to AdGuard"
```

Now the flow is:
1. Client → 192.168.31.1:53
2. DNAT → 192.168.31.4:53 (src masqueraded to MikroTik)
3. AdGuard responds to MikroTik
4. MikroTik un-NATs response → Client
5. Client sees response from .1 ✓

### Verification
```
$ nslookup google.com 192.168.31.1
Server:   192.168.31.1
Name:     google.com
Address:  142.250.187.110
```