Loading content...
While ping answers the binary question "Is the destination reachable?", network engineers often need more: Which routers does my traffic traverse? Where does the latency accumulate? At what point does the path fail?
These questions are answered by traceroute (Unix) or tracert (Windows)—a brilliant application of ICMP's TTL-exceeded mechanism to map network topology. First implemented by Van Jacobson in 1987, traceroute exploits a 'flaw' in IP's Time-To-Live handling to extract routing information that routers wouldn't normally reveal.
Traceroute has become indispensable for:
This page dissects the elegant mechanism behind traceroute, explores its protocol variations, and equips you with expert interpretation skills.
By the end of this page, you will fully understand the TTL-based discovery algorithm, distinguish between ICMP, UDP, and TCP traceroute variants, interpret traceroute output including anomalies like load balancing and asymmetric routing, and apply traceroute effectively in professional troubleshooting scenarios.
Traceroute's algorithm is elegant in its simplicity. It leverages a fundamental IP behavior: when a router decrements a packet's TTL to zero, it MUST discard the packet and send an ICMP Time Exceeded message back to the source.
By systematically probing with increasing TTL values, traceroute forces each router along the path to reveal itself:
The Core Algorithm:
123456789101112131415161718192021222324252627
function traceroute(destination): for ttl = 1 to MAX_HOPS: # typically 30 # Send probe packets with this TTL for probe = 1 to PROBES_PER_HOP: # typically 3 packet = create_probe_packet(destination, ttl) send_time = current_time() send(packet) response = wait_for_response(timeout=WAIT_TIME) if response is ICMP_TIME_EXCEEDED: # Intermediate router responded record(ttl, probe, response.source_ip, current_time() - send_time) elif response is ICMP_ECHO_REPLY or response is ICMP_DEST_UNREACHABLE(port): # Destination reached! record(ttl, probe, response.source_ip, current_time() - send_time) return SUCCESS elif no_response: record(ttl, probe, "*", timeout) display_hop(ttl, collected_results) return MAX_HOPS_EXCEEDEDStep-by-Step Execution:
Let's trace what happens when you run traceroute google.com:
TTL=1 Probe:
TTL=2 Probe:
...Continue incrementing TTL...
TTL=N (destination reached):
Traceroute defaults to 3 probes per TTL value. This redundancy serves multiple purposes: (1) Compensates for occasional packet loss, (2) Reveals load-balanced paths (different routers may respond to different probes), (3) Provides RTT variation data for each hop. Seeing consistent times across 3 probes suggests a stable link; wildly varying times suggest congestion or instability.
The traceroute concept works with any IP protocol—what matters is eliciting the ICMP Time Exceeded responses from intermediate routers. Different probe protocols are used based on destination behavior and firewall considerations:
1. ICMP Echo Traceroute (Windows default, Unix option)
Sends ICMP Echo Request packets with incrementing TTL:
1234567891011121314
# Windows tracert (always uses ICMP Echo)C:\> tracert google.com # Unix/Linux with ICMP option$ traceroute -I google.com # Advantages:# - Destination responds with ICMP Echo Reply (same as ping)# - Simple response interpretation# - Works through most NAT devices # Disadvantages:# - ICMP often blocked by firewalls# - Some routers rate-limit ICMP responses2. UDP Traceroute (Unix default)
Sends UDP packets to high-numbered ports (33434+):
123456789101112131415161718
# Default Unix traceroute (UDP)$ traceroute google.com # How it works:# - Sends UDP to port 33434 (incrementing per probe)# - Intermediate routers: ICMP Time Exceeded (same as ICMP method)# - Destination: ICMP Destination Unreachable (Port Unreachable)# Because nothing is listening on port 33434! # Advantages:# - Often traverses firewalls that block ICMP# - Different port per probe aids load-balancer analysis# - Raw socket not required on some systems # Disadvantages:# - Destination must return Port Unreachable# - Some firewalls block high UDP ports# - Response interpretation slightly more complex3. TCP Traceroute (Firewall-Friendly)
Sends TCP SYN packets to specific ports (often 80 or 443):
1234567891011121314151617181920212223242526
# TCP traceroute to port 80$ traceroute -T -p 80 google.com # Using tcptraceroute (specialized tool)$ tcptraceroute google.com 80 # Common options:$ traceroute -T -p 443 secure-server.com # HTTPS port$ traceroute -T -p 22 ssh-server.com # SSH port # How it works:# - Sends TCP SYN to specified port# - Intermediate routers: ICMP Time Exceeded (same as other methods)# - Destination: TCP SYN/ACK (if port open) or RST (if closed)# - Both responses confirm destination reached # Advantages:# - Traverses firewalls that allow web traffic# - Tests the exact path application traffic would take# - Often works when ICMP/UDP blocked# - Reaches hosts that filter ICMP # Disadvantages:# - Requires raw socket privileges# - May trigger IDS/IPS alerts# - Some firewalls do stateful inspection that blocks probes| Aspect | ICMP Echo | UDP | TCP SYN |
|---|---|---|---|
| Unix command | traceroute -I | traceroute (default) | traceroute -T |
| Windows command | tracert (default) | Not built-in | Not built-in |
| Destination response | Echo Reply | Port Unreachable | SYN/ACK or RST |
| Firewall traversal | Often blocked | Usually passes | Best (if port 80/443) |
| Requires root | Yes | Sometimes | Yes |
| Best use case | Simple networks | General purpose | Through strict firewalls |
Start with the default (UDP on Unix, ICMP on Windows). If you see * * * at the destination or final hops, the destination may be blocking your probe type—try TCP to port 80/443. If intermediate hops show * * *, those routers may not respond to ICMP Time Exceeded at all (some are configured to silently discard expired packets).
Traceroute output contains dense information. Let's analyze a real-world trace:
1234567891011121314151617
$ traceroute google.comtraceroute to google.com (142.250.185.46), 30 hops max, 60 byte packets 1 router.local (192.168.1.1) 0.421 ms 0.385 ms 0.361 ms 2 10.0.0.1 (10.0.0.1) 8.234 ms 8.198 ms 8.156 ms 3 ae-5.r21.lsanca07.us.bb.gin.ntt.net (129.250.2.89) 12.543 ms 12.521 ms 12.498 ms 4 ae-1.a01.lsanca07.us.bb.gin.ntt.net (129.250.6.114) 11.987 ms 12.003 ms 11.976 ms 5 72.14.196.226 (72.14.196.226) 12.134 ms 12.112 ms 12.089 ms 6 * * * 7 142.251.247.99 (142.251.247.99) 12.456 ms 12.441 ms 12.419 ms 8 lax17s51-in-f14.1e100.net (142.250.185.46) 11.234 ms 11.198 ms 11.156 ms # Breaking down each column:# # Hop# Hostname (IP Address) RTT1 RTT2 RTT3# # The RTT times are cumulative from source to that hop (round-trip).# Difference between hops indicates that link's contribution to latency.Field-by-Field Analysis:
Hop Number: Sequential position in the path. Hop 1 is your first router (usually default gateway).
Hostname: Reverse DNS (PTR record) of the responding IP. Reveals:
router.local — Local network namingae-5.r21.lsanca07.us.bb.gin.ntt.net — ISP router naming convention:
ae-5: Interface (Aggregated Ethernet #5)r21: Router designationlsanca07: Location code (Los Angeles, CA, site 07)us.bb.gin.ntt.net: NTT's US backbone Global IP NetworkIP Address: The source IP of the ICMP Time Exceeded message—the router's interface facing you (ingress interface).
RTT Values: Three probe times showing round-trip from source to this hop. In our example:
Latency Analysis:
| Hop | RTT | Link Contribution | Interpretation |
|---|---|---|---|
| 1 | 0.4 ms | 0.4 ms | LAN (excellent) |
| 2 | 8.2 ms | +7.8 ms | DSL/Cable modem (typical) |
| 3 | 12.5 ms | +4.3 ms | Metro fiber (good) |
| 4 | 12.0 ms | -0.5 ms | Same POP (negligible) |
| 5 | 12.1 ms | +0.1 ms | Peering point (fast) |
| 7 | 12.5 ms | +0.4 ms | Within Google (fast) |
| 8 | 11.2 ms | -1.3 ms | Destination (negative = normal variance) |
Asterisks mean 'no response.' This does NOT necessarily indicate a problem. Many routers are configured to not respond to ICMP (rate-limited or disabled). If subsequent hops respond, the path is working—the non-responding router is just silent. Only worry if ALL remaining hops show * * * (true black hole).
Real-world traceroutes often show patterns that confuse beginners. Understanding these anomalies is key to accurate interpretation:
12345678910111213141516171819202122232425262728293031
# Load Balancing (Multiple routers at same hop) 5 router-a.isp.net (10.0.1.1) 12.456 ms router-b.isp.net (10.0.1.2) 12.789 ms router-a.isp.net (10.0.1.1) 12.123 ms# Interpretation: ISP has parallel routers; traffic is load-balanced # MPLS Tunnel (Same IP at multiple hops) 4 mpls-edge.carrier.net (172.16.0.1) 20.123 ms 20.456 ms 20.789 ms 5 mpls-edge.carrier.net (172.16.0.1) 21.234 ms 21.567 ms 21.890 ms 6 mpls-edge.carrier.net (172.16.0.1) 22.345 ms 22.678 ms 22.901 ms# Interpretation: MPLS tunnel hides internal hops; only edge router responds # Asymmetric Path (IPs from wrong direction) 5 chicago-to-newyork.isp.net (10.5.6.7) 45.123 ms 6 losangeles-to-chicago.isp.net (10.8.9.10) 48.456 ms# Interpretation: Probe goes one way, ICMP returns via different path# The router names suggest geographic locations that don't make sense forward # Path Failure 7 last-good-router.isp.net (10.20.30.40) 25.678 ms 25.789 ms 25.890 ms 8 * * * 9 * * *10 * * *# Interpretation: Path dies after hop 7. Problem is beyond that router. # Routing Loop 5 router-x.isp.net (10.1.1.1) 30.123 ms 30.456 ms 30.789 ms 6 router-y.isp.net (10.2.2.2) 35.234 ms 35.567 ms 35.890 ms 7 router-x.isp.net (10.1.1.1) 40.345 ms 40.678 ms 40.901 ms 8 router-y.isp.net (10.2.2.2) 45.456 ms 45.789 ms 46.012 ms# Interpretation: Packets bouncing between two routers until TTL expiresAsymmetric Routing: The Hidden Complexity
A critical concept often overlooked: traceroute only shows the forward path. The ICMP Time Exceeded messages return via the reverse path, which may be completely different.
This means:
To truly diagnose asymmetric paths, you need traceroutes from BOTH ends.
Not every anomaly requires investigation. If you're troubleshooting 'why is my connection to X slow?' focus on large RTT jumps at specific hops. If investigating 'why can't I reach Y?', focus on where the path terminates (last responding hop). Asterisks in the middle of an otherwise working traceroute are usually irrelevant.
Expert network engineers leverage advanced options to extract maximum diagnostic value from traceroute:
1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253
# === PROBE CONFIGURATION === # Send 5 probes per hop (more data, slower)traceroute -q 5 google.com # Send only 1 probe per hop (faster, less reliable)traceroute -q 1 google.com # Increase maximum hops (default 30)traceroute -m 60 google.com # Set initial TTL (skip early hops)traceroute -f 5 google.com # Start at hop 5 # === TIMING CONTROL === # Set probe timeout (seconds)traceroute -w 2 google.com # 2-second wait per probe # Set minimum time between probes (seconds)traceroute -z 0.5 google.com # 500ms between probes # Simultaneous probes (Paris traceroute style)traceroute -N 16 google.com # 16 probes in parallel # === PROTOCOL OPTIONS === # ICMP modetraceroute -I google.com # TCP mode with specific porttraceroute -T -p 443 google.com # UDP with specific source porttraceroute -U --sport=53 google.com # Appear as DNS response # === OUTPUT CONTROL === # Don't resolve hostnames (faster, raw IPs only)traceroute -n google.com # Show AS numbers (Autonomous System)traceroute -A google.com # Show TTL values in repliestraceroute -v google.com # Verbose mode # === LOAD BALANCER ANALYSIS === # Paris traceroute: maintain flow identity across hops# Uses same src/dst port for all probes to same hoptraceroute --help | grep paris# or use dedicated tool: paris-tracerouteMTR: Traceroute + Ping Combined
MTR (My TraceRoute) combines traceroute path discovery with continuous ping statistics. It's often more useful than traditional traceroute for diagnosing intermittent issues:
1234567891011121314151617181920212223
# Install MTR# Ubuntu/Debian: apt install mtr# macOS: brew install mtr# Windows: WinMTR (graphical) # Basic MTRmtr google.com # Report mode (non-interactive, good for saving/sharing)mtr -r -c 100 google.com # Example MTR output:# My tracance to google.com# Host Loss% Snt Last Avg Best Wrst StDev# 1. router.local 0.0% 100 0.5 0.6 0.3 1.2 0.2# 2. 10.0.0.1 0.0% 100 8.2 8.5 7.9 12.3 0.8# 3. ae-5.r21.lsanca07... 0.0% 100 12.1 12.4 11.8 15.6 0.6# 4. ??? 0.0% 100 12.3 12.5 12.0 16.2 0.7# 5. 72.14.196.226 0.0% 100 12.8 13.1 12.4 18.9 1.2# 6. 142.251.247.99 0.5% 100 13.2 13.8 12.9 45.3 4.5 # <-- some loss!# 7. google.com 0.0% 100 12.1 12.5 11.8 16.7 0.9 # The Loss% column reveals intermittent problems that single traceroute misses| Column | Meaning | What to Look For |
|---|---|---|
| Loss% | Packet loss at this hop | 1% at any hop is concerning; >5% is problematic |
| Snt | Packets sent | More samples = more reliable statistics |
| Last | Most recent RTT | Current latency |
| Avg | Average RTT | Compare to expected for distance |
| Best | Minimum RTT | Network capability under ideal conditions |
| Wrst | Maximum RTT | Worst case—correlates with congestion |
| StDev | Standard deviation | High value indicates jitter/instability |
Some routers rate-limit ICMP responses, showing apparent loss in MTR even when no real loss exists. The key insight: If hop N shows loss but all subsequent hops show 0% loss, hop N is just rate-limiting responses—there's no actual packet loss. Only investigate loss that carries through to the destination.
Let's examine how experienced network engineers apply traceroute in real troubleshooting scenarios:
Scenario 1: User reports "the website is slow"
123456789101112131415161718192021
# Step 1: Baseline traceroute$ traceroute -n slow-website.com 1 192.168.1.1 0.5 ms 0.4 ms 0.4 ms 2 10.0.0.1 8.2 ms 8.1 ms 8.0 ms 3 68.86.85.1 12.3 ms 12.2 ms 12.1 ms 4 68.86.84.41 15.6 ms 15.5 ms 15.4 ms 5 173.167.57.105 89.7 ms 92.3 ms 95.6 ms # <-- HUGE JUMP! 6 104.198.214.1 92.4 ms 91.8 ms 93.1 ms 7 slow-website.com 93.2 ms 92.7 ms 93.5 ms # Analysis: # - Hops 1-4: Normal latency progression (~15ms total)# - Hop 5: Sudden +75ms jump!# - Hops 5-7: Latency stabilizes around 92ms # Diagnosis: The problem is the link between hop 4 and hop 5# - Hop 4 is ISP peering (68.86.x.x = Comcast)# - Hop 5 is server hosting (173.167.x.x)# - This is a peering congestion issue between networks # Solution: Report to hosting provider with traceroute dataScenario 2: Server unreachable from certain locations
12345678910111213141516171819202122232425262728
# From New York office - works fine$ traceroute server.company.com 1 192.168.1.1 0.3 ms ... 8 server.company.com 25.6 ms # From London office - fails$ traceroute server.company.com 1 192.168.10.1 0.4 ms 2 10.10.0.1 2.1 ms 3 213.242.89.1 12.3 ms 4 89.96.200.41 22.4 ms 5 89.96.200.42 24.1 ms !H !H !H # <-- Host Unreachable! # Analysis:# - London path fails at hop 5 with !H (Host Unreachable)# - Hop 5 is the same ISP as hop 4 (89.96.x.x)# - The ISP is sending back Host Unreachable # Diagnosis: Possible causes:# 1. Server firewall blocking London IP range# 2. ISP has blackholed server's IP range# 3. Routing misconfiguration at ISP level # Investigation steps:# - Check server firewall for geo-blocking# - Try traceroute from different London ISP# - Contact ISP (89.96.x.x) with traceroute evidenceScenario 3: Verifying traffic path for compliance
123456789101112131415161718192021
# Requirement: Traffic to EU servers must not transit US (GDPR)# Verify traffic from Frankfurt to London stays in EU $ traceroute -A datacenter-london.company.eu 1 192.168.1.1 [AS -] 0.3 ms 2 10.0.0.1 [AS -] 1.2 ms 3 de-fra04a.ix.de [AS8365] 2.3 ms # Frankfurt IX - good 4 ae-5.r01.frnkge09.de [AS3320] 3.1 ms # Deutsche Telekom - good 5 ae-3.r02.londen12.uk [AS3320] 8.7 ms # London - still AS3320 6 185.203.15.1 [AS12876] 9.2 ms # Scaleway London 7 datacenter-london.company.eu [AS12876] 10.1 ms # Analysis with AS numbers:# - AS3320: Deutsche Telekom (German carrier)# - Traffic path: Frankfurt IX → Deutsche Telekom → London# - No US-based AS numbers in path# - GDPR compliance: ✓ VERIFIED # If this appeared instead:# 4 ae-7.cr01.nyc01.us [AS3356] 78.3 ms # Level3 NYC!# That would indicate traffic transiting US - compliance failure!When reporting issues to ISPs or hosting providers, always include: (1) Your source IP or location, (2) Complete traceroute output with timestamps, (3) Comparison traceroute if behavior differs by time/location. Tools like mtr -r provide concise, shareable reports. Many ISPs have web portals (looking glasses) to run traceroutes from their network—use these to show the problem from both ends.
Traceroute is powerful but imperfect. Understanding its limitations prevents misdiagnosis:
The ICMP Generation Latency Problem
A particularly confusing issue: sometimes later hops show lower latency than earlier hops. This isn't time travel—it's ICMP generation priority:
3 router-a.isp.net 15.2 ms 15.4 ms 15.3 ms
4 router-b.isp.net 35.6 ms 38.2 ms 34.8 ms # <-- seems slow
5 router-c.isp.net 18.1 ms 17.9 ms 18.3 ms # <-- faster than hop 4?!
Interpretation: Hop 4's router is busy or deprioritizes ICMP—it takes extra time to generate the Time Exceeded message, making its reported RTT artificially high. The actual forwarding latency between hops is normal.
Rule of thumb: If a suspicious latency spike at hop N doesn't carry through to subsequent hops, it's probably ICMP generation delay, not real network latency.
Professional diagnosis requires: (1) Multiple traceroutes over time to establish patterns, (2) Traceroutes from both ends if possible, (3) Different protocol modes (ICMP, UDP, TCP) to rule out filtering, (4) MTR for packet loss statistics, (5) Comparison to known-good baseline traces. A single traceroute is a snapshot, not a diagnosis.
We've covered the complete traceroute mechanism, from underlying algorithm to expert-level interpretation. Let's consolidate the essential knowledge:
What's Next:
With the traceroute mechanism understood, we'll next explore TTL Manipulation—the deliberate control of Time-To-Live for diagnostic purposes beyond standard traceroute. You'll learn how TTL is used in packet scoping, stealth reconnaissance, and advanced network analysis techniques.
You now understand traceroute at a professional level—from its elegant algorithm exploiting ICMP, through protocol variations and output interpretation, to practical troubleshooting application. Combined with your ping expertise, you possess the two fundamental tools that form the foundation of network path diagnosis.