traceroute: Discovering the Path Hop by Hop

Every IP packet carries a TTL (hop limit in IPv6). Each router decrements it by one before forwarding; a router that decrements it to zero discards the packet and sends an ICMP Time Exceeded (type 11, code 0; ICMPv6 type 3) back to the source. The mechanism exists to kill packets caught in a routing loop. traceroute abuses it: send a probe with TTL 1 and the first router replies; TTL 2 and the second router replies; keep going until the destination itself answers — with an ICMP Port Unreachable for a UDP probe, an echo reply for an ICMP probe, or a SYN-ACK/RST for a TCP probe.

The source address of each Time Exceeded message is the router’s interface that received the probe (the ingress interface, usually), so what you see is a list of ingress addresses along the forward path, with the RTT to each measured as probe-out to ICMP-back. Three probes per TTL by default give three RTTs per hop; 30 TTLs is the default maximum.

Linux and macOS traceroute send UDP datagrams to high ports starting at 33434, incrementing per probe, so the destination answers with Port Unreachable. Windows tracert sends ICMP echo requests, so the destination answers with an echo reply. Linux traceroute -I uses ICMP echo and -T uses TCP SYN (needs root; tcptraceroute is the standalone version); -p 443 sets the port. The choice matters because firewalls treat them differently: a firewall that drops UDP to random high ports kills the Linux default at the last hop while tcptraceroute -p 443 sails through to the real service port. mtr (winmtr on Windows) runs the trace continuously and shows per-hop loss and RTT distributions — better than a single traceroute for intermittent problems.

The intermediate hops reply with Time Exceeded regardless of probe type, so the middle of the path looks the same with every variant; the probe type mainly decides whether the last hop answers.

Read RTT jumps between consecutive hops, not absolute values: the hop where RTT goes from 15 ms to 95 ms is where the packet crossed an ocean or a satellite link. Read * * * as "no reply from this hop", which is not the same as "this hop dropped my traffic" — if later hops reply, the silent one forwarded fine and merely declined to generate ICMP. A trace that ends in * * * forever means either the destination filters the probe type or the path really is broken; switch probe type before concluding.

Hostnames from reverse DNS often encode location and role (ae-3.r21.frankfurt.de style), which is how you learn that a Berlin-to-Munich packet went via Amsterdam. Private addresses (10.x, 172.16–31.x, 192.168.x) in the middle of a trace are normal inside carriers and clouds.

Silent hops. Routers generate ICMP on their control-plane CPU, not in forwarding hardware, and rate-limit it aggressively. A hop showing * * * or 30% loss in mtr while every later hop shows 0% loss is rate-limiting, not path loss. The rule: loss that does not persist to the destination is not real loss. Similarly a latency spike at one hop that disappears at the next means the router was slow to generate the ICMP, not slow to forward.

Asymmetric paths. Each RTT is forward-path-to-that-hop plus that hop’s return path to you, and the return path is chosen by the hop’s own routing, which may differ entirely from yours. A sudden RTT increase can come from a bad return route from that router, invisible in the forward trace. Tracing from the other end shows the other half; Internet Routing: Autonomous Systems and BGP explains why the halves differ.

MPLS and tunnels. Inside carrier networks packets are label-switched; providers frequently disable TTL propagation into the MPLS tunnel, so a dozen physical routers show up as one hop, or several hops show the same RTT because the ICMP travelled to the tunnel end first. ECMP. Load-balanced paths hash each probe differently, so probes for TTL n and n+1 may traverse different routers, producing a trace that interleaves two paths — paris-traceroute and mtr keep flow identifiers stable to avoid this.

None of this makes traceroute useless. It reliably shows: the rough geography of the path, the hop where a large RTT is added and persists, and the last hop that responded before silence. It does not reliably show per-hop loss, the return path, or the hops inside a carrier’s core.