0fbb5b4cfcc852ca5a5278edd984e53713f2e01d
The self-heal script written on 2026-11-05 was never scheduled — it had run once, by hand. The hc binary had disappeared with no reboot at 25 days uptime, so the boot-time restore in the go file is not sufficient on its own. Now runs every 10 minutes, from the go file's crontab line because that line replaces the whole crontab at boot. Also documents hc put as the transport for Mac screenshots into a session here (verified end to end), the RAM-backed drop location, $HC_TERMINAL_ID, the new shot helper, and what reaching the Mac from this box still needs. Separately: the crontab comment's claimed hourly git-ref update of the AMPECO backend checkout corresponds to no actual job and has not been running. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
XTRM Home Lab Infrastructure
Domain: xtrm-lab.org Repository: https://git.xtrm-lab.org/jazzymc/infrastructure
Quick Reference
| Resource | Address |
|---|---|
| Dashboard | https://xtrm-lab.org |
| NetBox | https://netbox.xtrm-lab.org |
| Git | https://git.xtrm-lab.org |
| CI/CD | https://ci.xtrm-lab.org |
| DNS Primary | dns.xtrm-lab.org |
| DNS Secondary | dns2.xtrm-lab.org |
| Failover VIP | 192.168.10.250 |
Documentation Structure
docs/
├── 01-NETWORK-MAP.md # Network topology, IPs, Docker networks
├── 02-SERVICES-CRITICAL.md # DNS, Auth, Routing - must stay up
├── 03-SERVICES-OTHER.md # All other services
├── 04-HARDWARE-INVENTORY.md # Physical devices, specs, serials
├── 05-PORT-UTILIZATION.md # Device port assignments
├── 06-VLAN-DEVICE-ASSIGNMENT.md # VLAN device mapping
├── 07-WIFI-CAPSMAN-CONFIG.md # WiFi and CAPsMAN settings
├── 08-DNS-ARCHITECTURE.md # DNS failover architecture
├── 09-TAILSCALE-VPN.md # Tailscale VPN setup
├── 10-FAILOVER-NOBARA.md # VRRP failover to Nobara workstation
├── CHANGELOG.md # Change history
├── archive/ # Completed/legacy docs
│ └── vlan-migration/ # VLAN migration project artifacts
├── incidents/ # Incident reports
└── wip/ # Work in progress
Key Devices
| Device | IP | Role |
|---|---|---|
| HAP1 | 192.168.10.1 | Router, DNS, WiFi Controller |
| XTRM-U | 192.168.10.20 | Production Server (Unraid) |
| XTRM-Nobara | 192.168.10.103 | Failover Node (Nobara Linux) |
| CSS1 | 192.168.10.3 | Distribution Switch |
| ZX1 | 192.168.10.4 | Core Switch (2.5G) |
| CAP | 192.168.10.6 | Wireless Access Point |
SSH Access
# Unraid
ssh -i ~/.ssh/id_ed25519_unraid root@192.168.10.20 -p 422
# MikroTik Router
ssh -i ~/.ssh/mikrotik_key -p 2222 xtrm@192.168.10.1
# Nobara (failover node)
ssh nobara
Emergency Recovery
- DNS down? → Automatic failover to 192.168.10.10 (secondary), see
08-DNS-ARCHITECTURE.md - Internet down? → Check HAP1 at 192.168.10.1
- Services down? → Check Unraid at 192.168.10.20
- Unraid maintenance? → VRRP failover to Nobara (192.168.10.250 VIP), see
10-FAILOVER-NOBARA.md - Full outage? → See
02-SERVICES-CRITICAL.mdstartup order
Change Management
- Major changes: Document in
CHANGELOG.md - Minor changes: Git commit messages only
- Planned work: Create doc in
wip/folder
CI/CD
Woodpecker CI at https://ci.xtrm-lab.org
Pipelines trigger on push to this repository.
Languages
RouterOS Script
81.6%
Shell
18.4%