NDU — Network Discovery Utility

See every device on your LAN — which AP, which switch port, which VLAN, how it's connected, what it is.

NDU continuously scans your home network and correlates data from your existing infrastructure (MikroTik, OPNsense, Pi-hole, UniFi, OpenWRT) into a single, live topology view. Open the browser, see the whole house — no agents, no cloud.

Why NDU?

Most tools tell you what is connected. NDU tells you how.

NDU stitches together who (vendor, hostname, custom labels), where (AP / switch port / VLAN / interface), how (wifi vs ethernet, signal, link speed), and when (first/last seen, lease expiry, presence timeline) — from the devices you already run.

Live device table

Sortable, filterable, with per-device tags (infra / handheld / iot / printer / nas / …), presence badges, and last-seen timestamps — all updated in real time via WebSocket.

Per-device detail

Expand any row for full metadata, first/last seen, seen count, one-click ping and Wake-on-LAN, and on-demand nmap port scan with cached results.

Status: v0.6.0 running in production on a home LAN of ~50 devices. v0.7.0 (Docker image, audit cleanup) in progress — Docker works locally, GHCR multi-arch publishing pending. See CHANGELOG.md for full history.

What NDU does

• Discovers every device on the LAN via ARP scan / ip neigh / ping sweep.
• Enriches devices by calling provider plugins:
  • MikroTik — Wi-Fi clients, DHCP leases, DNS static records, ARP, bridge hosts, neighbors, routing, VLANs, interfaces, identity.
  • OPNsense — DHCP leases (dnsmasq / dhcpv4 / Kea), Unbound host overrides, ARP, gateway status, interfaces.
  • Pi-hole — DHCP leases and DNS hostnames.
  • UniFi — Wi-Fi clients with signal/channel, switch port mapping, device uplinks, LLDP neighbors.
• Merges the provider data into a unified device model (role-based, conflict-aware — see docs/architecture.md).
• Builds topology — firewall → switches → APs → clients, with per-AP Wi-Fi binding and bridge-host reattribution for wired clients behind APs.
• Scans ports on demand via nmap (throttled; stored per device with history).
• Notifies on new / offline devices via Home Assistant (webhook or HA API) and/or ntfy.
• Detects anomalies — MAC randomization, MAC spoofing (vendor mismatch), provider-only "phantom" MACs (gated by active ICMP verification).
• Serves a Svelte SPA with a live device table, topology map, compare view, admin config UI, privacy-blur toggle, WebSocket live updates.

For feature history see CHANGELOG.md.

Architecture at a glance

┌───────────────────────────────────────────────────────────────┐
│  Browser (Svelte 5 SPA, svelte-spa-router)                    │
│  Devices table • Topology • Compare • Admin • Login           │
│                 ▲                     ▲                       │
│                 │ REST /api/*         │ WebSocket /ws         │
└─────────────────┼─────────────────────┼───────────────────────┘
                  │                     │
┌─────────────────┴─────────────────────┴───────────────────────┐
│  ndu_api.py   —   FastAPI backend (REST + WS + static SPA)    │
├───────────────────────────────────────────────────────────────┤
│  ndu_monitor.py  —  scan loop orchestrator (subcommands)      │
│  ndu_scan.py     —  scan modes, ARP, provider envelope merge  │
│  topology/       —  capability-claim graph engine             │
│  ndu_topology.py —  persisted-topology loader (thin helper)   │
│  ndu_db.py       —  SQLite (WAL), devices / change_log / nmap │
│  ndu_common.py   —  shared models (DeviceSnapshot, helpers)   │
│  ndu_notify.py   —  HA webhook / HA API / ntfy dispatch       │
│  ndu_report.py   —  markdown / CSV / HTML / compare reports   │
├───────────────────────────────────────────────────────────────┤
│  providers/<name>/metadata_<name>.py + config.json            │
│    mikrotik  opnsense  openwrt  pihole  ubiquiti              │
└───────────────────────────────────────────────────────────────┘

All Python modules live at the repo root (not under backend/). The frontend is a separate tree in frontend/ that builds to frontend/dist/ — committed to git so the Pi can deploy without running Node.

Full architecture — provider envelope, capability model, merge rules, topology algorithm — is in docs/architecture.md.

Project layout

Requirements

• Python 3.11+ (tested on Raspberry Pi OS Bookworm).
• System tools installed from apt:
  • python3-venv — NDU runs in a project-local virtualenv.
  • arp-scan — for scan.mode: "arp_scan" (recommended; finds the most devices).
  • nmap — for port scanning from the UI.
  • sshpass — optional, MikroTik SSH password fallback when REST is closed.
• Node 20+ only on the build machine (Pi does not build — it consumes committed frontend/dist/).

sudo apt update
sudo apt install -y python3 python3-venv arp-scan nmap sshpass
sudo setcap cap_net_raw,cap_net_admin+eip "$(command -v arp-scan)"

Python dependencies (FastAPI, argon2, etc.) come from requirements.txt and are installed into /opt/ndu/.venv by scripts/install.sh — no global pip install.

Installation — Docker (recommended for non-Pi hosts)

Linux host only — network_mode: host is required (ARP broadcast, WOL, reverse DNS all need the real LAN, not a Docker bridge). Docker Desktop on macOS / Windows cannot join the host network — for those, run NDU on a Raspberry Pi or a small Linux VM.

git clone https://github.com/TursiThePanda/NDU.git ndu
cd ndu
mkdir -p config data
docker compose up -d

Open http://<host>:8787/ in a browser — the first-run wizard will configure everything.

What the compose file does:

• Builds the image from Dockerfile (multi-stage; Node compiles the SPA, Python runtime stays lean at ~270 MB)
• Runs on network_mode: host with NET_RAW + NET_ADMIN capabilities so arp-scan works without root
• Mounts ./config → /app/custom (config.json, wizard writes here) and ./data → /app/data (SQLite, OUI cache, runtime state)
• Healthcheck hits /api/health every 30 s

Once GitHub Actions publishes to GHCR, swap build: . for image: ghcr.io/tursithepanda/ndu:latest to skip the local build.

Known limitation for v0.7.0: the wizard writes the main config.json to the mounted ./config/ directory (persistent), but provider-specific credentials (MikroTik password, UniFi API key, etc.) are written inside the container filesystem and are lost on restart. Workaround: after the wizard, copy the provider configs out of the container to your host before stopping it, or pre-fill them before first start. A proper fix (wizard writes provider configs to the mounted volume) is tracked for v0.7.1.

Installation on a Raspberry Pi

1. Clone the repo

sudo mkdir -p /opt/ndu
sudo chown "$USER:$USER" /opt/ndu
git clone https://github.com/<your-account>/NDU-Network_Discovery_Utility.git /opt/ndu
cd /opt/ndu

2. Run the installer

sudo /opt/ndu/scripts/install.sh

Does everything in one pass:

• Checks prereqs (python3, python3-venv, arp-scan, nmap) and prints the exact apt install line if anything is missing.
• Optional (interactive Y/n): creates a dedicated ndu system user with /usr/sbin/nologin and no login shell, chowns /opt/ndu so the service cannot write elsewhere. Accept this on fresh installs; existing User=youruser deployments can keep their owner.
• Creates /opt/ndu/.venv and pip install -r requirements.txt inside it — no --break-system-packages, no global pollution.
• Hardens file permissions: chmod 600 on custom/config.json, providers/*/config.json, data/users.json; chmod 700 on data/.
• Prints a ready-to-copy /etc/systemd/system/ndu.service template with the resolved User= and ExecStart= pointing at the venv — operator saves it to disk and runs systemctl enable --now ndu.service.

Re-run anytime to refresh dependencies: sudo /opt/ndu/scripts/install.sh --deps-only (also what scripts/deploy-pi.sh calls after git pull).

See docs/secure-deployment.md for the threat model behind the dedicated user + chmods and the upgrade path from a legacy interactive-user install.

3. Bootstrap the config

Run the first-run wizard in your browser at http://<pi-ip>:8787/setup — it walks you through scan settings, provider detection, credentials, and admin account creation, then writes custom/config.json, per-provider configs, and data/users.json for you.

If you prefer to seed a config from the template manually:

cp custom/config.example.json custom/config.json

Then edit custom/config.json — at minimum set scan.network_cidr and scan.interface.

4. Enable the systemd unit

install.sh printed a unit file template at the end of step 2. Save it, reload, enable:

sudo nano /etc/systemd/system/ndu.service   # paste the template
sudo systemctl daemon-reload
sudo systemctl enable --now ndu.service
sudo systemctl status ndu.service

Open http://<pi-ip>:8787 in a browser and log in with the admin account you created in step 3.

Legacy installs (pre-v0.5.0)

Deployments created before v0.5.0 used a global pip3 install --break-system-packages and User=<your-interactive-user>. They keep working — the installer, deploy-pi.sh, and install.sh --deps-only all fall back gracefully when no venv / dedicated user exists. To migrate at your own pace, follow docs/secure-deployment.md.

Deploy workflow

Two paths, in order of preference:

Primary — git-based

1. Commit + push changes from your workstation to GitHub.

2. On the Pi:

   sudo /opt/ndu/scripts/deploy-pi.sh
   

   The script runs git pull as the youruser account (owner of /opt/ndu), then restarts ndu.service. Live configs (custom/, providers/*/config.json, data/, users.json) are gitignored — git pull does not touch them.

GitHub is the source of truth. /opt/ndu has origin set to the repo via a read-only SSH deploy key (github-ndu).

Secondary — Samba rsync (preview only)

For testing uncommitted changes before push:

1. scripts/deploy.ps1 robocopies Windows → Samba share.
2. On the Pi, rsync from the Samba path into /opt/ndu.

Use sparingly. Production always goes through git.

Scan modes

If arp-scan is missing or cannot open raw sockets, NDU falls back to active for that cycle.

Active verification for provider-only devices (ICMP ping before enrollment) is on by default — keeps DHCP-lease ghosts and UniFi "recently seen" phantoms out of state. See CHANGELOG.md entry for scan.require_active_verification.

Web UI

Routes in frontend/src/routes/:

• Devices (implicit /) — sortable table, filters, per-device expand row with metadata / topology location / nmap results.
• Topology (/topology) — pan/zoom/search graph with privacy-blur, offline retention, tag-based highlight.
• Compare (/compare) — change log for a time window (new / removed / renamed / IP changed).
• Report (/report) — markdown / HTML export.
• Login (/login) — required before any other route.
• Admin (/admin/*) — admin/Config.svelte (scan, identification, notifications, integrations, logging, classification), admin/Users.svelte (user CRUD). Admin role only.

Privacy blur (👁/🙈 in the top nav) toggles a CSS blur on sensitive cells (IP / MAC / hostname / SSID / WAN / nmap details). Good for screenshots and screen-share, not a security boundary — values stay in the DOM.

Notifications

Three channels, any combination:

Home Assistant webhook (recommended)

"home_assistant": {
  "webhook_url": "http://HA_IP:8123/api/webhook/ndu_new_device"
}

Automation in automations.yaml:

- id: ndu_new_device_push
  alias: NDU - New device discovered
  mode: queued
  trigger:
    - platform: webhook
      webhook_id: ndu_new_device
      allowed_methods: [POST]
      local_only: false
  action:
    - service: notify.mobile_app_YOUR_PHONE
      data:
        title: "NDU: New device on network"
        message: >
          {{ trigger.json.display_name or trigger.json.hostname or 'unknown' }}
          | {{ trigger.json.vendor or 'unknown vendor' }}
          | {{ trigger.json.ip }} / {{ trigger.json.mac }}
          {% if trigger.json.ssid %}| SSID: {{ trigger.json.ssid }}{% endif %}

Home Assistant notify service (direct API)

"home_assistant": {
  "api_url": "http://HA_IP:8123",
  "token": "LONG_LIVED_ACCESS_TOKEN",
  "notify_service": "notify.mobile_app_your_phone"
}

ntfy

"notifications": {
  "channels": {
    "ntfy": {
      "enabled": true,
      "server_url": "https://ntfy.sh",
      "topic": "my-ndu-topic",
      "priority": "default",
      "token": ""
    }
  }
}

Policy

"notifications": {
  "enabled": true,
  "cooldown_hours": 6,
  "cooldown_scope": "mac",
  "ignore": {
    "macs": ["aa:bb:cc:dd:ee:ff"],
    "oui_prefixes": ["48:8f:5a"],
    "tags": ["iot", "tv", "ap"],
    "connection_types": ["ethernet"],
    "vendors": ["routerboard"]
  },
  "offline": {
    "enabled": true,
    "grace_seconds": 900
  }
}

Ignored devices are still tracked and shown — only the notification is suppressed. Offline events fire on online → offline transition after the grace window.

NDU can also push aggregate sensor values to Home Assistant:

"home_assistant": {
  "api_url": "http://HA_IP:8123",
  "token": "...",
  "sensors": {
    "enabled": true,
    "interval_seconds": 300,
    "entity_prefix": "ndu_network"
  }
}

Exposed entities: total_devices, online_devices, offline_devices, unknown_devices, new_devices_24h.

Device identification

Display name fallback order:

1. hostname (reverse DNS or provider metadata).
2. Manual alias from identification.mac_aliases.
3. Provider-supplied display_name / name / device_name.
4. <vendor> <MAC suffix> from the OUI map.
5. Unknown (xx:yy).

OUI vendor map comes from three layers (later overrides earlier):

1. default/oui_vendors.bundle.json (shipped fallback).
2. identification.oui_vendor_file (auto-updated from IEEE if auto_update_oui: true).
3. Inline identification.oui_vendors in config.

Manual refresh:

python3 scripts/update_oui_vendors.py --output data/oui_vendors.json

Tags and classification

Tags can be set via config (classification.tags_by_mac, classification.tags_by_oui), from provider metadata, or matched by 23 default auto-tag rules (laptop, phone, AP, NAS, printer, camera, TV, IoT, …).

"classification": {
  "tags_by_mac": {
    "02:00:00:19:73:58": ["laptop", "trusted"]
  },
  "tags_by_oui": {
    "48:8f:5a": ["infra", "ap"]
  }
}

Tags drive: notification ignore rules, topology search highlight, device detail groups, node icons.

State retention

"scan": {
  "state_retention_hours": 24,
  "change_log_retention_hours": 168,
  "change_log_max_entries": 5000
}

Devices not seen within state_retention_hours are pruned from state. Known MAC history (known_macs) is kept separately so pruned devices do not re-trigger "new device" alerts when they reappear. To fully reset, delete data/ndu.db (and data/devices_state.json if present — legacy).

Reports and compare

# Markdown
python3 ndu_monitor.py --config custom/config.json report --format markdown --output reports/network.md

# HTML sorted by IP
python3 ndu_monitor.py --config custom/config.json report --format html --sort ip --output reports/network.html

# Change log for the last 24 hours
python3 ndu_monitor.py --config custom/config.json compare --hours 24 --format html --output reports/compare.html

--live-scan forces a fresh scan before generating the report. --desc reverses sort order.

Low-level entrypoints

The scan loop + API are one process under ndu.service. Other subcommands are available for scripting:

All accept --config <path>. Defaults to ./config.json.

Documentation index

• docs/architecture.md — provider model, merge rules, topology algorithm.
• docs/TODO & Notes.md — roadmap (Docker, OpenWRT provider, first-run wizard, MCP server, per-device pages).
• CHANGELOG.md — detailed per-version notes.

License and contributing

Licensed under AGPL v3 — see LICENSE. Derivative works, including ones made available over a network (SaaS / managed NDU deployments for others), must publish their modified source under the same terms. Self-hosting NDU on your own LAN carries no obligation.

The project welcomes new providers (the contract is in docs/architecture.md — "Contract for new provider authors"); adding one does not require any change to the scanner.