Ping and Traceroute — Network Troubleshooting | OneNoughtOne

0/107

Computer NetworksTroubleshooting

Network Troubleshooting

LevelIntermediate

Duration75 mins

TopicTroubleshooting

3 / 5

Ping and Traceroute

Mastering the Fundamentals

If you had to troubleshoot networks with only two tools, you'd choose ping and traceroute . These deceptively simple utilities answer the two most fundamental questions in network troubleshooting: Ping: Can I reach the destination, and how long does it take? Traceroute: What path do packets take to get there, and where might they fail? Most engineers use these tools daily but never develop true mastery. They know how to run the commands but not how to extract deep diagnostic insights from the results. This page takes you from basic usage to expert-level interpretation.

What You Will Learn

Deep understanding of ICMP protocol mechanics, advanced ping techniques (including MTU discovery and timing analysis), traceroute variants and their tradeoffs, interpreting complex results, and combining both tools for comprehensive path analysis. You'll develop intuition for what results mean and what they don't.

ICMP - The Protocol Behind the Tools

Both ping and traceroute rely on the Internet Control Message Protocol (ICMP) . Understanding ICMP is essential for interpreting results correctly and understanding tool limitations. ICMP Overview: ICMP is a Layer 3 protocol encapsulated directly in IP (IP protocol number 1). Unlike TCP and UDP, ICMP doesn't carry application data—it carries control and error messages about network conditions. ICMP Message Types Used in Troubleshooting:

Key ICMP Message Types
Type	Code	Name	Purpose
0	0	Echo Reply	Response to ping (Echo Request)
3	0	Net Unreachable	Router can't reach destination network
3	1	Host Unreachable	Router can reach network but not host
3	3	Port Unreachable	Host reached, but UDP port closed
3	4	Fragmentation Needed	Packet too big, but DF flag set (MTU discovery)
3	13	Administratively Prohibited	Firewall blocked the packet
8	0	Echo Request	Ping request
11	0	TTL Exceeded	Packet's TTL reached zero (used by traceroute)

Converting Mermaid diagram...

ICMP Echo Request/Reply Structure: +-------------------+-------------------+ | Type (1 byte) | Code (1 byte) | +-------------------+-------------------+ | Checksum (2 bytes) | +------------------+--------------------+ | Identifier (2) | Sequence (2) | +------------------+--------------------+ | Data (variable) | +---------------------------------------+ Type/Code: 8/0 for request, 0/0 for reply Identifier: Matches requests with replies (usually process ID) Sequence Number: Increments with each ping, correlates specific packets Data: Optional payload (used for timing and size testing) ICMP and Firewalls: ICMP is frequently filtered by firewalls. This creates complications:

Filtering Approach	Result
Block all ICMP	Ping fails, traceroute fails, Path MTU Discovery breaks
Allow Echo only	Ping works, traceroute may partially work
Block Echo, allow TTL Exceeded	Ping fails, traceroute works
Rate-limit ICMP	Ping/traceroute slow or show packet loss that isn't real

Always consider that ICMP tools may be affected by filtering, not just network issues.

ICMP Can Lie

ICMP traffic is often treated differently than application traffic. A successful ping doesn't guarantee the application will work—firewalls may allow ICMP but block TCP. Conversely, a failed ping doesn't mean the host is unreachable—the host might just block ICMP while happily serving HTTP.

Deep Dive into Ping

Beyond basic reachability testing, ping provides detailed metrics for network analysis. Ping Metrics Explained: 1. Round-Trip Time (RTT): RTT measures the time from sending the Echo Request to receiving the Echo Reply. It includes: Propagation delay (speed of light through medium) Transmission delay (time to push bits onto wire) Processing delay (router/host processing) Queuing delay (waiting in buffers) Typical baseline RTT values:

Path Type	Typical RTT
Local LAN	< 1ms
Metropolitan WAN	5-20ms
Continental (Europe, NA)	20-60ms
Intercontinental	100-200ms
Satellite (GEO)	500-700ms

2. Packet Loss: Percentage of pings that don't receive replies. Causes include: Congestion (buffers overflow) Physical issues (bad cables, interference) Firewall filtering (sometimes rate-limited) Host overload (can't respond to all pings)

Packet Loss	Quality Level
0%	Excellent
< 1%	Good (acceptable for most applications)
1-5%	Poor (VoIP/video affected)
5-20%	Very poor (noticeable performance impact)
20%	Severe (investigate immediately)

3. Jitter (RTT Variation): The difference between minimum and maximum RTT, or the variance across pings. High jitter indicates inconsistent network performance. # Example ping statistics analysis 64 bytes from 8.8.8.8: icmp_seq=1 ttl=117 time=12.3 ms 64 bytes from 8.8.8.8: icmp_seq=2 ttl=117 time=14.1 ms 64 bytes from 8.8.8.8: icmp_seq=3 ttl=117 time=11.8 ms 64 bytes from 8.8.8.8: icmp_seq=4 ttl=117 time=45.2 ms <- Spike! 64 bytes from 8.8.8.8: icmp_seq=5 ttl=117 time=12.0 ms Statistics: - Min: 11.8ms, Max: 45.2ms, Avg: 19.1ms - Jitter: 45.2 - 11.8 = 33.4ms (concerning) 4. TTL (Time to Live): The TTL in the reply indicates how many hops are left after traversing the path. Different OS default TTL values:

OS	Default TTL
Linux	64
Windows	128
macOS	64
Network devices	Usually 64 or 255

If you ping a host and get TTL=52, and Linux uses TTL=64, the host is approximately 12 hops away (64 - 52 = 12).

Advanced Ping Techniques

# MTU Path Discovery - Find maximum packet size without fragmentation
# Start with 1472 (1500 - 20 IP - 8 ICMP)
ping -f -l 1472 target.example.com
 
# If "Packet needs to be fragmented", reduce size:
ping -f -l 1400 target.example.com
 
# Binary search to find exact MTU
# Success at 1400, fail at 1472
ping -f -l 1436 target.example.com
 
# Ping with specific TTL (test reachability to specific hop)
ping -i 5 target.example.com  # TTL=5
 
# Record route option (limited to 9 hops)
ping -r 9 target.example.com
 
# Timestamp option
ping -s 4 target.example.com
 
# Resolve hostnames for each hop IP
ping -a target.example.com

MTU Discovery Technique

Path MTU discovery using ping is invaluable for troubleshooting fragmentation issues. Start at 1472 (Ethernet MTU minus IP/ICMP headers). If it fails, binary search downward. Common break points: 1400 (VPN overhead), 1380 (PPPoE), 1280 (IPv6 minimum).

Interpreting Ping Results

Expert interpretation goes beyond 'it works' or 'it doesn't.' Each result pattern tells a story. Pattern 1: Complete Failure C:\> ping 192.168.1.100 Pinging 192.168.1.100 with 32 bytes of data: Request timed out. Request timed out. Request timed out. Request timed out. Analysis: No ICMP errors returned, just silence Could be: host down, firewall dropping silently, routing black hole Next step: traceroute to see where packets stop, check ARP for local hosts Pattern 2: Destination Unreachable C:\> ping 10.99.99.99 Pinging 10.99.99.99 with 32 bytes of data: Reply from 192.168.1.1: Destination host unreachable. Analysis: A router (192.168.1.1) is responding with ICMP error Router can't find valid route or ARP for destination The router IP tells you where the problem is Next step: Check routing on that router Pattern 3: Intermittent Loss 64 bytes from 8.8.8.8: icmp_seq=1 ttl=117 time=12.3 ms 64 bytes from 8.8.8.8: icmp_seq=2 ttl=117 time=11.8 ms Request timed out. 64 bytes from 8.8.8.8: icmp_seq=4 ttl=117 time=12.1 ms Request timed out. 64 bytes from 8.8.8.8: icmp_seq=6 ttl=117 time=11.9 ms Analysis: Some packets succeed, some fail Could be: congestion, flapping link, ICMP rate limiting If loss is regular (every Nth packet), likely rate limiting If random, likely congestion or physical issue Next step: mtr for continuous path analysis

Concerning Patterns

•Consistently high RTT — Congestion or suboptimal routing
•Sudden RTT spikes — Intermittent congestion or queue buildup
•Gradually increasing RTT — Buffer bloat or worsening congestion
•Regular packet loss — ICMP rate limiting or firewall
•Variable TTL values — Load balancing across different paths
•Different reply IP than destination — Middlebox, NAT, or proxy

Healthy Patterns

•Consistent low RTT — Stable, well-provisioned path
•Zero packet loss — Reliable connectivity
•Stable TTL — Consistent path (no flapping)
•RTT matches distance — Expected latency for path length
•Symmetric RTT — Consistent in both directions
•Low jitter — RTT variance under 10% of average

Pattern 4: High Latency Spikes 64 bytes from target: icmp_seq=1 ttl=117 time=11.2 ms 64 bytes from target: icmp_seq=2 ttl=117 time=11.5 ms 64 bytes from target: icmp_seq=3 ttl=117 time=312.6 ms <- Spike! 64 bytes from target: icmp_seq=4 ttl=117 time=11.3 ms Analysis: Occasional massive latency spikes Classic symptom of buffer bloat or momentary congestion One packet queued behind large transfers Common in residential connections during uploads Next step: Test with background load (simulates real usage) Pattern 5: TTL Changes 64 bytes from target: icmp_seq=1 ttl=117 time=12.1 ms 64 bytes from target: icmp_seq=2 ttl=116 time=13.2 ms 64 bytes from target: icmp_seq=3 ttl=117 time=11.9 ms 64 bytes from target: icmp_seq=4 ttl=116 time=14.1 ms Analysis: TTL alternates between 117 and 116 Different paths with different hop counts (load balancing) ECMP (Equal-Cost Multi-Path) routing in effect If RTT also alternates, different path characteristics

Traceroute Fundamentals

Traceroute reveals the path packets take through the network. It exploits the TTL field in IP headers—each router decrements TTL and, when it hits zero, returns an ICMP TTL Exceeded message. How Traceroute Works: Send packet with TTL=1 First router decrements to 0, returns ICMP TTL Exceeded Note router IP and round-trip time Send packet with TTL=2 Second router returns TTL Exceeded Continue until destination responds (or max hops reached)

Converting Mermaid diagram...

Traceroute Variants:

Command	Platform	Probe Type	Default Port
`tracert`	Windows	ICMP Echo	N/A
`traceroute`	Linux	UDP	33434+
`traceroute -I`	Linux	ICMP Echo	N/A
`traceroute -T`	Linux	TCP SYN	80 (configurable)
`mtr`	Linux	Configurable	Configurable

The probe type matters because different protocols are filtered differently by firewalls.

Traceroute Commands

# Basic traceroute (uses ICMP)
tracert example.com
 
# Don't resolve hostnames (faster)
tracert -d example.com
 
# Set maximum hops
tracert -h 20 example.com
 
# Set timeout per hop (milliseconds)
tracert -w 2000 example.com
 
# Force IPv4
tracert -4 example.com
 
# Force IPv6
tracert -6 example.com
 
# PowerShell equivalent
Test-NetConnection example.com -TraceRoute

Use mtr for Intermittent Issues

mtr (My Traceroute) combines continuous ping with traceroute, showing packet loss and latency at each hop over time. It's invaluable for diagnosing intermittent problems that a single traceroute might miss.

Interpreting Traceroute Output

Traceroute output requires careful interpretation. Many patterns that look problematic are actually normal. Sample Traceroute Analysis: traceroute to example.com (93.184.216.34), 30 hops max 1 gateway (192.168.1.1) 1.234 ms 0.987 ms 0.923 ms 2 10.0.0.1 12.432 ms 11.876 ms 12.001 ms 3 isp-edge.example.net 15.234 ms 14.987 ms 15.123 ms 4 core-router.peer.net 25.432 ms 24.876 ms 25.001 ms 5 * * * 6 cdn-edge.example.net 23.234 ms 22.987 ms 23.123 ms 7 93.184.216.34 24.432 ms 23.876 ms 24.001 ms Understanding Each Line:

Hop	IP/Hostname	RTT Values	Meaning
1	gateway (192.168.1.1)	~1ms	Local router, fast
2	10.0.0.1	~12ms	ISP first hop, normal
3	isp-edge.example.net	~15ms	ISP edge, slight increase
4	core-router.peer.net	~25ms	Peering point, 10ms jump
5		No response (normal!)
6	cdn-edge.example.net	~23ms	Response returned (faster than hop 4!)
7	Destination	~24ms	Final destination

Key Insight: Hop 5's asterisks don't indicate a problem! That router simply doesn't respond to traceroute probes (ICMP filtered). Since hops 6 and 7 respond, connectivity is fine.

Common Traceroute Patterns: Pattern 1: * * * at Intermediate Hops (Usually Normal) 4 core-router.isp.net 25.1 ms 24.8 ms 25.0 ms 5 * * * 6 * * * 7 edge-server.cdn.net 28.4 ms 27.9 ms 28.2 ms Interpretation: Hops 5-6 are routers that don't respond to traceroute. Since the trace continues and completes, this is normal filtering, not a problem. Pattern 2: * * * to End (Potential Problem) 4 core-router.isp.net 25.1 ms 24.8 ms 25.0 ms 5 peering.exchange.net 35.2 ms 34.9 ms 35.1 ms 6 * * * 7 * * * 8 * * * ... Interpretation: After hop 5, nothing responds. Could be: Destination is down Firewall blocking all probes Routing issue beyond hop 5 Try ICMP vs TCP probes to confirm Pattern 3: Sudden Latency Jump 3 isp-core.example.net 15.2 ms 14.9 ms 15.1 ms 4 trans-atlantic.net 95.3 ms 94.8 ms 95.0 ms <- 80ms jump! 5 eu-router.example.net 98.4 ms 97.9 ms 98.1 ms Interpretation: 80ms jump at hop 4 indicates a long-distance link (continental hop). This is normal physics (speed of light across Atlantic). RTT stays in that range afterward.

Normal Patterns

•Intermediate * * * hops — Routers filtering ICMP
•Gradual RTT increase — Each hop adds latency
•Sudden 60-100ms jump — Intercontinental link
•Multiple IPs for one hop — Load-balanced path
•Higher RTT at mid-path — Router prioritizes data over ICMP
•Private IPs (10.x, 172.x) — ISP internal addressing

Problem Indicators

•All * * * to end — Likely reachability issue
•High RTT that stays high — Congestion at that hop
•Inconsistent RTT (jitter) — Unstable link or congestion
•Time exceeds for ICMP — Different from normal * * *
•Packet loss at specific hop — Check with mtr
•RTT decreases significantly — Asymmetric routing (replies take shorter path)

The Asymmetric Path Trap

RTT at a hop includes the round-trip: outbound to hop N, plus the ICMP reply returning. But the reply may take a completely different path! A hop showing 100ms might be 50ms outbound + 50ms return via a different route. Don't assume the outbound path determines total latency.

mtr - Continuous Path Analysis

mtr (My Traceroute) combines the functionality of ping and traceroute, providing continuous statistics for each hop. It's the gold standard for diagnosing intermittent path issues. Sample mtr Output: My traceroute [v0.95] host Loss% Snt Last Avg Best Wrst StDev 1. gateway 0.0% 50 0.5 0.4 0.3 0.8 0.1 2. isp-access 0.0% 50 11.2 10.8 9.5 15.2 1.2 3. isp-core 0.0% 50 14.5 14.2 12.8 18.9 1.5 4. peering 0.0% 50 25.3 24.8 22.5 32.1 2.1 5. ??? ??? 0 0.0 0.0 0.0 0.0 0.0 6. cdn-edge 0.0% 50 28.2 27.5 25.8 35.4 2.3 7. destination 0.0% 50 29.1 28.4 26.5 36.2 2.4 Reading mtr Output:

Column	Meaning
Loss%	Percentage of packets lost at this hop
Snt	Number of probes sent
Last	Last probe RTT (ms)
Avg	Average RTT (ms)
Best	Minimum RTT seen
Wrst	Maximum RTT seen
StDev	Standard deviation (jitter indicator)

Interpreting mtr Statistics:

The Loss Pattern Analysis: Scenario 1: Loss at intermediate hop, not at destination 4. problematic-router 15.0% 50 25.1 24.8 22.5 35.1 2.5 5. next-router 0.0% 50 28.2 27.5 25.8 35.4 2.3 6. destination 0.0% 50 29.1 28.4 26.5 36.2 2.4 Interpretation: Hop 4 shows 15% loss, but destination shows 0%. This router likely deprioritizes or rate-limits ICMP responses. It's NOT actually dropping traffic—if it were, the destination would also show loss. Scenario 2: Loss starts at hop and continues to end 4. problematic-router 12.0% 50 25.1 24.8 22.5 35.1 2.5 5. next-router 12.0% 50 28.2 27.5 25.8 35.4 2.3 6. destination 12.0% 50 29.1 28.4 26.5 36.2 2.4 Interpretation: Loss begins at hop 4 and persists (same percentage) through all subsequent hops. Hop 4 is genuinely dropping packets. This is a real problem. Scenario 3: High jitter (StDev) at specific hop 3. stable-router 0.0% 50 14.5 14.2 12.8 16.9 0.8 4. unstable-router 0.0% 50 75.1 44.8 22.5 195.1 45.2 <- High StDev! 5. next-router 0.0% 50 78.2 47.5 25.8 198.4 46.3 Interpretation: Hop 4 has 0% loss but massive jitter (StDev=45.2). This indicates buffer bloat or intermittent congestion at that router. The jitter propagates to subsequent hops.

mtr Best Practices
bash
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
# Standard report (100 probes, shows summary)
mtr -r -c 100 example.com
 
# Wide report with full hostnames
mtr -rwc 100 example.com
 
# Both directions (if you have access to remote host)
# From your machine:
mtr -rwc 100 remote-host
# From remote host:
ssh remote-host "mtr -rwc 100 your-machine"
 
# TCP mode (bypasses ICMP filtering)
mtr -T -P 443 -c 100 example.com
 
# Include AS numbers (identify networks)
mtr -rzw example.com
 
# Export to file for sharing
mtr -rwc 100 example.com > mtr-report.txt
 
# Best practice: run from both ends during issues
# and run for at least 100 cycles (about 2 minutes)

Bidirectional mtr

Network issues can be asymmetric—the problem might be in the return path, not the outbound path. If possible, run mtr from both ends simultaneously. Compare the results to distinguish outbound vs return path issues.

Combining Ping and Traceroute

Maximum diagnostic power comes from using both tools strategically together. Diagnostic Workflow: 1. Symptom: "Can't reach server X" | v 2. [ping server-x] | +----+----+ | | Success Failure | | v v 3. ping works, but [traceroute server-x] app doesn't: | - Check ports Where does it stop? - Check app | - Check DNS +-----+-----+ | | Mid-path Never reaches death destination (hop N * * * | to end) Routing/firewall | issue v [ping hop-N directly] | 4. Does it respond? Yes: routing issue beyond hop N No: hop N device issue

Workflow Example - Diagnosing 'Slow Website': Step 1: Baseline ping $ ping website.com ROund-trip min/avg/max = 11.2/145.8/312.4 ms Packet loss = 5% High average, high variance, some loss. Problem confirmed. Step 2: Traceroute to find where $ traceroute website.com 1 gateway 0.5 ms 2 isp-access 11.2 ms 3 isp-core 14.5 ms 4 peering 25.3 ms 5 congested-hop 312.8 ms <- HUGE jump! 6 cdn-edge 315.2 ms 7 website.com 318.4 ms Hop 5 shows 290ms jump. Problem isolated. Step 3: Ping problematic hop directly $ ping 10.0.5.1 -c 100 5% packet loss, avg=285ms, max=450ms Confirmed: hop 5 is the problem. Step 4: Identify ownership $ whois 10.0.5.1 [Shows ISP/upstream provider info] Now you know who to contact. Step 5: Use mtr for evidence $ mtr -rwc 100 website.com > mtr-report.txt Statistical evidence to share with provider.

Troubleshooting Decision Tree

•Ping fails, traceroute shows local hops only → Local configuration issue (check gateway, routes)
•Ping fails, traceroute dies at ISP edge → ISP issue or destination unreachable
•Ping fails, traceroute completes → ICMP blocked but destination reachable; try TCP tests
•Ping works with loss, traceroute shows * * * mid-path → Use mtr to find loss-introducing hop
•Ping slow, traceroute shows latency jump at hop N → Congestion or distance at hop N
•Ping jittery, traceroute normal → May need mtr for continuous analysis to catch intermittent issues

Document Everything

When reporting issues to ISPs or upstream providers, include: (1) mtr reports from both directions if possible, (2) timestamps correlating with problem occurrence, (3) what traffic is affected (all traffic vs specific destinations), (4) any relevant changes you made. Good evidence gets faster resolution.

Computer NetworksTroubleshooting

Network Troubleshooting

LevelIntermediate

Duration75 mins

TopicTroubleshooting

3 / 5

Ping and Traceroute

Mastering the Fundamentals

If you had to troubleshoot networks with only two tools, you'd choose ping and traceroute . These deceptively simple utilities answer the two most fundamental questions in network troubleshooting: Ping: Can I reach the destination, and how long does it take? Traceroute: What path do packets take to get there, and where might they fail? Most engineers use these tools daily but never develop true mastery. They know how to run the commands but not how to extract deep diagnostic insights from the results. This page takes you from basic usage to expert-level interpretation.

What You Will Learn

Deep understanding of ICMP protocol mechanics, advanced ping techniques (including MTU discovery and timing analysis), traceroute variants and their tradeoffs, interpreting complex results, and combining both tools for comprehensive path analysis. You'll develop intuition for what results mean and what they don't.

ICMP - The Protocol Behind the Tools

Both ping and traceroute rely on the Internet Control Message Protocol (ICMP) . Understanding ICMP is essential for interpreting results correctly and understanding tool limitations. ICMP Overview: ICMP is a Layer 3 protocol encapsulated directly in IP (IP protocol number 1). Unlike TCP and UDP, ICMP doesn't carry application data—it carries control and error messages about network conditions. ICMP Message Types Used in Troubleshooting:

Key ICMP Message Types
Type	Code	Name	Purpose
0	0	Echo Reply	Response to ping (Echo Request)
3	0	Net Unreachable	Router can't reach destination network
3	1	Host Unreachable	Router can reach network but not host
3	3	Port Unreachable	Host reached, but UDP port closed
3	4	Fragmentation Needed	Packet too big, but DF flag set (MTU discovery)
3	13	Administratively Prohibited	Firewall blocked the packet
8	0	Echo Request	Ping request
11	0	TTL Exceeded	Packet's TTL reached zero (used by traceroute)

Converting Mermaid diagram...

ICMP Echo Request/Reply Structure: +-------------------+-------------------+ | Type (1 byte) | Code (1 byte) | +-------------------+-------------------+ | Checksum (2 bytes) | +------------------+--------------------+ | Identifier (2) | Sequence (2) | +------------------+--------------------+ | Data (variable) | +---------------------------------------+ Type/Code: 8/0 for request, 0/0 for reply Identifier: Matches requests with replies (usually process ID) Sequence Number: Increments with each ping, correlates specific packets Data: Optional payload (used for timing and size testing) ICMP and Firewalls: ICMP is frequently filtered by firewalls. This creates complications:

Filtering Approach	Result
Block all ICMP	Ping fails, traceroute fails, Path MTU Discovery breaks
Allow Echo only	Ping works, traceroute may partially work
Block Echo, allow TTL Exceeded	Ping fails, traceroute works
Rate-limit ICMP	Ping/traceroute slow or show packet loss that isn't real

Always consider that ICMP tools may be affected by filtering, not just network issues.

ICMP Can Lie

ICMP traffic is often treated differently than application traffic. A successful ping doesn't guarantee the application will work—firewalls may allow ICMP but block TCP. Conversely, a failed ping doesn't mean the host is unreachable—the host might just block ICMP while happily serving HTTP.

Deep Dive into Ping

Beyond basic reachability testing, ping provides detailed metrics for network analysis. Ping Metrics Explained: 1. Round-Trip Time (RTT): RTT measures the time from sending the Echo Request to receiving the Echo Reply. It includes: Propagation delay (speed of light through medium) Transmission delay (time to push bits onto wire) Processing delay (router/host processing) Queuing delay (waiting in buffers) Typical baseline RTT values:

Path Type	Typical RTT
Local LAN	< 1ms
Metropolitan WAN	5-20ms
Continental (Europe, NA)	20-60ms
Intercontinental	100-200ms
Satellite (GEO)	500-700ms

2. Packet Loss: Percentage of pings that don't receive replies. Causes include: Congestion (buffers overflow) Physical issues (bad cables, interference) Firewall filtering (sometimes rate-limited) Host overload (can't respond to all pings)

Packet Loss	Quality Level
0%	Excellent
< 1%	Good (acceptable for most applications)
1-5%	Poor (VoIP/video affected)
5-20%	Very poor (noticeable performance impact)
20%	Severe (investigate immediately)

3. Jitter (RTT Variation): The difference between minimum and maximum RTT, or the variance across pings. High jitter indicates inconsistent network performance. # Example ping statistics analysis 64 bytes from 8.8.8.8: icmp_seq=1 ttl=117 time=12.3 ms 64 bytes from 8.8.8.8: icmp_seq=2 ttl=117 time=14.1 ms 64 bytes from 8.8.8.8: icmp_seq=3 ttl=117 time=11.8 ms 64 bytes from 8.8.8.8: icmp_seq=4 ttl=117 time=45.2 ms <- Spike! 64 bytes from 8.8.8.8: icmp_seq=5 ttl=117 time=12.0 ms Statistics: - Min: 11.8ms, Max: 45.2ms, Avg: 19.1ms - Jitter: 45.2 - 11.8 = 33.4ms (concerning) 4. TTL (Time to Live): The TTL in the reply indicates how many hops are left after traversing the path. Different OS default TTL values:

OS	Default TTL
Linux	64
Windows	128
macOS	64
Network devices	Usually 64 or 255

If you ping a host and get TTL=52, and Linux uses TTL=64, the host is approximately 12 hops away (64 - 52 = 12).

Advanced Ping Techniques

# MTU Path Discovery - Find maximum packet size without fragmentation
# Start with 1472 (1500 - 20 IP - 8 ICMP)
ping -f -l 1472 target.example.com
 
# If "Packet needs to be fragmented", reduce size:
ping -f -l 1400 target.example.com
 
# Binary search to find exact MTU
# Success at 1400, fail at 1472
ping -f -l 1436 target.example.com
 
# Ping with specific TTL (test reachability to specific hop)
ping -i 5 target.example.com  # TTL=5
 
# Record route option (limited to 9 hops)
ping -r 9 target.example.com
 
# Timestamp option
ping -s 4 target.example.com
 
# Resolve hostnames for each hop IP
ping -a target.example.com

MTU Discovery Technique

Path MTU discovery using ping is invaluable for troubleshooting fragmentation issues. Start at 1472 (Ethernet MTU minus IP/ICMP headers). If it fails, binary search downward. Common break points: 1400 (VPN overhead), 1380 (PPPoE), 1280 (IPv6 minimum).

Interpreting Ping Results

Expert interpretation goes beyond 'it works' or 'it doesn't.' Each result pattern tells a story. Pattern 1: Complete Failure C:\> ping 192.168.1.100 Pinging 192.168.1.100 with 32 bytes of data: Request timed out. Request timed out. Request timed out. Request timed out. Analysis: No ICMP errors returned, just silence Could be: host down, firewall dropping silently, routing black hole Next step: traceroute to see where packets stop, check ARP for local hosts Pattern 2: Destination Unreachable C:\> ping 10.99.99.99 Pinging 10.99.99.99 with 32 bytes of data: Reply from 192.168.1.1: Destination host unreachable. Analysis: A router (192.168.1.1) is responding with ICMP error Router can't find valid route or ARP for destination The router IP tells you where the problem is Next step: Check routing on that router Pattern 3: Intermittent Loss 64 bytes from 8.8.8.8: icmp_seq=1 ttl=117 time=12.3 ms 64 bytes from 8.8.8.8: icmp_seq=2 ttl=117 time=11.8 ms Request timed out. 64 bytes from 8.8.8.8: icmp_seq=4 ttl=117 time=12.1 ms Request timed out. 64 bytes from 8.8.8.8: icmp_seq=6 ttl=117 time=11.9 ms Analysis: Some packets succeed, some fail Could be: congestion, flapping link, ICMP rate limiting If loss is regular (every Nth packet), likely rate limiting If random, likely congestion or physical issue Next step: mtr for continuous path analysis

Concerning Patterns

•Consistently high RTT — Congestion or suboptimal routing
•Sudden RTT spikes — Intermittent congestion or queue buildup
•Gradually increasing RTT — Buffer bloat or worsening congestion
•Regular packet loss — ICMP rate limiting or firewall
•Variable TTL values — Load balancing across different paths
•Different reply IP than destination — Middlebox, NAT, or proxy

Healthy Patterns

•Consistent low RTT — Stable, well-provisioned path
•Zero packet loss — Reliable connectivity
•Stable TTL — Consistent path (no flapping)
•RTT matches distance — Expected latency for path length
•Symmetric RTT — Consistent in both directions
•Low jitter — RTT variance under 10% of average

Pattern 4: High Latency Spikes 64 bytes from target: icmp_seq=1 ttl=117 time=11.2 ms 64 bytes from target: icmp_seq=2 ttl=117 time=11.5 ms 64 bytes from target: icmp_seq=3 ttl=117 time=312.6 ms <- Spike! 64 bytes from target: icmp_seq=4 ttl=117 time=11.3 ms Analysis: Occasional massive latency spikes Classic symptom of buffer bloat or momentary congestion One packet queued behind large transfers Common in residential connections during uploads Next step: Test with background load (simulates real usage) Pattern 5: TTL Changes 64 bytes from target: icmp_seq=1 ttl=117 time=12.1 ms 64 bytes from target: icmp_seq=2 ttl=116 time=13.2 ms 64 bytes from target: icmp_seq=3 ttl=117 time=11.9 ms 64 bytes from target: icmp_seq=4 ttl=116 time=14.1 ms Analysis: TTL alternates between 117 and 116 Different paths with different hop counts (load balancing) ECMP (Equal-Cost Multi-Path) routing in effect If RTT also alternates, different path characteristics

Traceroute Fundamentals

Traceroute reveals the path packets take through the network. It exploits the TTL field in IP headers—each router decrements TTL and, when it hits zero, returns an ICMP TTL Exceeded message. How Traceroute Works: Send packet with TTL=1 First router decrements to 0, returns ICMP TTL Exceeded Note router IP and round-trip time Send packet with TTL=2 Second router returns TTL Exceeded Continue until destination responds (or max hops reached)

Converting Mermaid diagram...

Traceroute Variants:

Command	Platform	Probe Type	Default Port
`tracert`	Windows	ICMP Echo	N/A
`traceroute`	Linux	UDP	33434+
`traceroute -I`	Linux	ICMP Echo	N/A
`traceroute -T`	Linux	TCP SYN	80 (configurable)
`mtr`	Linux	Configurable	Configurable

The probe type matters because different protocols are filtered differently by firewalls.

Traceroute Commands

# Basic traceroute (uses ICMP)
tracert example.com
 
# Don't resolve hostnames (faster)
tracert -d example.com
 
# Set maximum hops
tracert -h 20 example.com
 
# Set timeout per hop (milliseconds)
tracert -w 2000 example.com
 
# Force IPv4
tracert -4 example.com
 
# Force IPv6
tracert -6 example.com
 
# PowerShell equivalent
Test-NetConnection example.com -TraceRoute

Use mtr for Intermittent Issues

mtr (My Traceroute) combines continuous ping with traceroute, showing packet loss and latency at each hop over time. It's invaluable for diagnosing intermittent problems that a single traceroute might miss.

Interpreting Traceroute Output

Traceroute output requires careful interpretation. Many patterns that look problematic are actually normal. Sample Traceroute Analysis: traceroute to example.com (93.184.216.34), 30 hops max 1 gateway (192.168.1.1) 1.234 ms 0.987 ms 0.923 ms 2 10.0.0.1 12.432 ms 11.876 ms 12.001 ms 3 isp-edge.example.net 15.234 ms 14.987 ms 15.123 ms 4 core-router.peer.net 25.432 ms 24.876 ms 25.001 ms 5 * * * 6 cdn-edge.example.net 23.234 ms 22.987 ms 23.123 ms 7 93.184.216.34 24.432 ms 23.876 ms 24.001 ms Understanding Each Line:

Hop	IP/Hostname	RTT Values	Meaning
1	gateway (192.168.1.1)	~1ms	Local router, fast
2	10.0.0.1	~12ms	ISP first hop, normal
3	isp-edge.example.net	~15ms	ISP edge, slight increase
4	core-router.peer.net	~25ms	Peering point, 10ms jump
5		No response (normal!)
6	cdn-edge.example.net	~23ms	Response returned (faster than hop 4!)
7	Destination	~24ms	Final destination

Key Insight: Hop 5's asterisks don't indicate a problem! That router simply doesn't respond to traceroute probes (ICMP filtered). Since hops 6 and 7 respond, connectivity is fine.

Common Traceroute Patterns: Pattern 1: * * * at Intermediate Hops (Usually Normal) 4 core-router.isp.net 25.1 ms 24.8 ms 25.0 ms 5 * * * 6 * * * 7 edge-server.cdn.net 28.4 ms 27.9 ms 28.2 ms Interpretation: Hops 5-6 are routers that don't respond to traceroute. Since the trace continues and completes, this is normal filtering, not a problem. Pattern 2: * * * to End (Potential Problem) 4 core-router.isp.net 25.1 ms 24.8 ms 25.0 ms 5 peering.exchange.net 35.2 ms 34.9 ms 35.1 ms 6 * * * 7 * * * 8 * * * ... Interpretation: After hop 5, nothing responds. Could be: Destination is down Firewall blocking all probes Routing issue beyond hop 5 Try ICMP vs TCP probes to confirm Pattern 3: Sudden Latency Jump 3 isp-core.example.net 15.2 ms 14.9 ms 15.1 ms 4 trans-atlantic.net 95.3 ms 94.8 ms 95.0 ms <- 80ms jump! 5 eu-router.example.net 98.4 ms 97.9 ms 98.1 ms Interpretation: 80ms jump at hop 4 indicates a long-distance link (continental hop). This is normal physics (speed of light across Atlantic). RTT stays in that range afterward.

Normal Patterns

•Intermediate * * * hops — Routers filtering ICMP
•Gradual RTT increase — Each hop adds latency
•Sudden 60-100ms jump — Intercontinental link
•Multiple IPs for one hop — Load-balanced path
•Higher RTT at mid-path — Router prioritizes data over ICMP
•Private IPs (10.x, 172.x) — ISP internal addressing

Problem Indicators

•All * * * to end — Likely reachability issue
•High RTT that stays high — Congestion at that hop
•Inconsistent RTT (jitter) — Unstable link or congestion
•Time exceeds for ICMP — Different from normal * * *
•Packet loss at specific hop — Check with mtr
•RTT decreases significantly — Asymmetric routing (replies take shorter path)

The Asymmetric Path Trap

RTT at a hop includes the round-trip: outbound to hop N, plus the ICMP reply returning. But the reply may take a completely different path! A hop showing 100ms might be 50ms outbound + 50ms return via a different route. Don't assume the outbound path determines total latency.

mtr - Continuous Path Analysis

mtr (My Traceroute) combines the functionality of ping and traceroute, providing continuous statistics for each hop. It's the gold standard for diagnosing intermittent path issues. Sample mtr Output: My traceroute [v0.95] host Loss% Snt Last Avg Best Wrst StDev 1. gateway 0.0% 50 0.5 0.4 0.3 0.8 0.1 2. isp-access 0.0% 50 11.2 10.8 9.5 15.2 1.2 3. isp-core 0.0% 50 14.5 14.2 12.8 18.9 1.5 4. peering 0.0% 50 25.3 24.8 22.5 32.1 2.1 5. ??? ??? 0 0.0 0.0 0.0 0.0 0.0 6. cdn-edge 0.0% 50 28.2 27.5 25.8 35.4 2.3 7. destination 0.0% 50 29.1 28.4 26.5 36.2 2.4 Reading mtr Output:

Column	Meaning
Loss%	Percentage of packets lost at this hop
Snt	Number of probes sent
Last	Last probe RTT (ms)
Avg	Average RTT (ms)
Best	Minimum RTT seen
Wrst	Maximum RTT seen
StDev	Standard deviation (jitter indicator)

Interpreting mtr Statistics:

The Loss Pattern Analysis: Scenario 1: Loss at intermediate hop, not at destination 4. problematic-router 15.0% 50 25.1 24.8 22.5 35.1 2.5 5. next-router 0.0% 50 28.2 27.5 25.8 35.4 2.3 6. destination 0.0% 50 29.1 28.4 26.5 36.2 2.4 Interpretation: Hop 4 shows 15% loss, but destination shows 0%. This router likely deprioritizes or rate-limits ICMP responses. It's NOT actually dropping traffic—if it were, the destination would also show loss. Scenario 2: Loss starts at hop and continues to end 4. problematic-router 12.0% 50 25.1 24.8 22.5 35.1 2.5 5. next-router 12.0% 50 28.2 27.5 25.8 35.4 2.3 6. destination 12.0% 50 29.1 28.4 26.5 36.2 2.4 Interpretation: Loss begins at hop 4 and persists (same percentage) through all subsequent hops. Hop 4 is genuinely dropping packets. This is a real problem. Scenario 3: High jitter (StDev) at specific hop 3. stable-router 0.0% 50 14.5 14.2 12.8 16.9 0.8 4. unstable-router 0.0% 50 75.1 44.8 22.5 195.1 45.2 <- High StDev! 5. next-router 0.0% 50 78.2 47.5 25.8 198.4 46.3 Interpretation: Hop 4 has 0% loss but massive jitter (StDev=45.2). This indicates buffer bloat or intermittent congestion at that router. The jitter propagates to subsequent hops.

mtr Best Practices
bash
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
# Standard report (100 probes, shows summary)
mtr -r -c 100 example.com
 
# Wide report with full hostnames
mtr -rwc 100 example.com
 
# Both directions (if you have access to remote host)
# From your machine:
mtr -rwc 100 remote-host
# From remote host:
ssh remote-host "mtr -rwc 100 your-machine"
 
# TCP mode (bypasses ICMP filtering)
mtr -T -P 443 -c 100 example.com
 
# Include AS numbers (identify networks)
mtr -rzw example.com
 
# Export to file for sharing
mtr -rwc 100 example.com > mtr-report.txt
 
# Best practice: run from both ends during issues
# and run for at least 100 cycles (about 2 minutes)

Bidirectional mtr

Network issues can be asymmetric—the problem might be in the return path, not the outbound path. If possible, run mtr from both ends simultaneously. Compare the results to distinguish outbound vs return path issues.

Combining Ping and Traceroute

Maximum diagnostic power comes from using both tools strategically together. Diagnostic Workflow: 1. Symptom: "Can't reach server X" | v 2. [ping server-x] | +----+----+ | | Success Failure | | v v 3. ping works, but [traceroute server-x] app doesn't: | - Check ports Where does it stop? - Check app | - Check DNS +-----+-----+ | | Mid-path Never reaches death destination (hop N * * * | to end) Routing/firewall | issue v [ping hop-N directly] | 4. Does it respond? Yes: routing issue beyond hop N No: hop N device issue

Workflow Example - Diagnosing 'Slow Website': Step 1: Baseline ping $ ping website.com ROund-trip min/avg/max = 11.2/145.8/312.4 ms Packet loss = 5% High average, high variance, some loss. Problem confirmed. Step 2: Traceroute to find where $ traceroute website.com 1 gateway 0.5 ms 2 isp-access 11.2 ms 3 isp-core 14.5 ms 4 peering 25.3 ms 5 congested-hop 312.8 ms <- HUGE jump! 6 cdn-edge 315.2 ms 7 website.com 318.4 ms Hop 5 shows 290ms jump. Problem isolated. Step 3: Ping problematic hop directly $ ping 10.0.5.1 -c 100 5% packet loss, avg=285ms, max=450ms Confirmed: hop 5 is the problem. Step 4: Identify ownership $ whois 10.0.5.1 [Shows ISP/upstream provider info] Now you know who to contact. Step 5: Use mtr for evidence $ mtr -rwc 100 website.com > mtr-report.txt Statistical evidence to share with provider.

Troubleshooting Decision Tree

•Ping fails, traceroute shows local hops only → Local configuration issue (check gateway, routes)
•Ping fails, traceroute dies at ISP edge → ISP issue or destination unreachable
•Ping fails, traceroute completes → ICMP blocked but destination reachable; try TCP tests
•Ping works with loss, traceroute shows * * * mid-path → Use mtr to find loss-introducing hop
•Ping slow, traceroute shows latency jump at hop N → Congestion or distance at hop N
•Ping jittery, traceroute normal → May need mtr for continuous analysis to catch intermittent issues

Document Everything

When reporting issues to ISPs or upstream providers, include: (1) mtr reports from both directions if possible, (2) timestamps correlating with problem occurrence, (3) what traffic is affected (all traffic vs specific destinations), (4) any relevant changes you made. Good evidence gets faster resolution.

Ping and Traceroute

Mastering the Fundamentals

ICMP - The Protocol Behind the Tools

Deep Dive into Ping

Interpreting Ping Results

Traceroute Fundamentals

Interpreting Traceroute Output

mtr - Continuous Path Analysis

Combining Ping and Traceroute

There’s more to understand.

Ping and Traceroute

Mastering the Fundamentals

ICMP - The Protocol Behind the Tools

Deep Dive into Ping

Interpreting Ping Results

Traceroute Fundamentals

Interpreting Traceroute Output

mtr - Continuous Path Analysis

Combining Ping and Traceroute

There’s more to understand.