BGP Operation

Every time you access a website hosted on a different continent, stream video from a content delivery network, or use a cloud service, your traffic traverses multiple autonomous systems—each operated by different organizations with their own policies and priorities. The Border Gateway Protocol (BGP) is the glue that binds these disparate networks into a cohesive, functioning Internet.

At the heart of BGP's operation lies the concept of the BGP session—a persistent, reliable connection between two BGP speakers that forms the foundation for all routing information exchange. Unlike interior gateway protocols that may use broadcast or multicast mechanisms, BGP relies on explicit, point-to-point sessions established over TCP. Understanding BGP sessions is fundamental to understanding how the Internet routes traffic at a global scale.

BGP's choice of TCP as its transport protocol is a deliberate architectural decision that distinguishes it from most other routing protocols. While protocols like OSPF and IS-IS implement their own reliability mechanisms directly over IP, and RIP uses UDP with application-layer timers, BGP leverages TCP's mature, battle-tested reliability services.

The Rationale for TCP:

BGP operates in an environment where routing updates must be delivered reliably across potentially unreliable network paths. When a BGP router announces a new prefix or withdraws an existing one, that information must reach its peer without corruption, duplication, or loss. TCP provides this reliability without BGP needing to implement its own acknowledgment and retransmission logic.

Moreover, BGP sessions often traverse multiple router hops—particularly in eBGP scenarios where peers may be separated by intermediate networks. TCP's end-to-end reliability guarantees work regardless of the underlying network topology, making it ideal for inter-domain routing.

TCP Connection Establishment:

BGP session establishment begins with a standard TCP three-way handshake. One BGP speaker initiates a TCP connection to port 179 on its peer. The BGP implementation can be configured to either:

1. Active Mode: Initiate TCP connections to configured peers
2. Passive Mode: Listen for incoming connections from peers
3. Both: Attempt connections while also accepting inbound connections

In practice, both peers typically attempt to connect simultaneously. BGP resolves this collision by selecting the connection initiated by the peer with the higher BGP identifier (router ID). The other connection is closed gracefully.

Once the TCP connection is established, BGP-level session establishment begins with the exchange of OPEN messages. The TCP connection remains established for the lifetime of the BGP session—which can persist for days, weeks, or even months in stable networks.

BGP sessions are established between BGP peers (also called BGP neighbors). The nature of the peering relationship profoundly affects BGP's behavior, including route propagation rules, attribute handling, and operational requirements. Understanding peer relationships is essential for both network design and troubleshooting.

External BGP (eBGP) Sessions:

eBGP sessions connect BGP speakers in different Autonomous Systems (ASes). These sessions form the inter-domain routing fabric of the Internet. Key characteristics include:

• Different AS Numbers: Peers belong to different ASes
• Typically Directly Connected: Standard eBGP requires peers to share a common network segment
• TTL = 1 by Default: Packets limited to single hop (security measure)
• Full Route Propagation: Routes learned from eBGP peers are advertised to all other peers
• Next-Hop Changes: The next-hop attribute is modified to the eBGP speaker's address
• AS-Path Modification: The local AS number is prepended to advertised routes

Internal BGP (iBGP) Sessions:

iBGP sessions connect BGP speakers within the same Autonomous System. These sessions distribute external routes throughout the AS and enable consistent egress point selection. Key characteristics include:

• Same AS Number: Both peers belong to the same AS
• No Direct Connectivity Required: iBGP peers can be anywhere in the AS
• High TTL: Packets traverse multiple internal hops
• Split Horizon: Routes learned from iBGP peers are NOT re-advertised to other iBGP peers
• Next-Hop Preserved: The next-hop attribute from eBGP is retained
• AS-Path Unchanged: No AS prepending for iBGP advertisements

The iBGP Full Mesh Requirement:

The split-horizon rule for iBGP (routes from one iBGP peer not advertised to other iBGP peers) creates a critical design requirement: every BGP speaker in an AS must have iBGP sessions with every other BGP speaker. This full mesh ensures that routing information is distributed completely.

For an AS with n BGP speakers, this requires n(n-1)/2 iBGP sessions. With 5 routers, you need 10 sessions. With 100 routers, you need 4,950 sessions. This O(n²) scaling becomes prohibitive in large networks.

Solutions to the Full Mesh Problem:

1. Route Reflectors: Designated routers that re-advertise iBGP routes, eliminating the need for full mesh
2. Confederations: Subdivide the AS into sub-ASes with eBGP-like behavior between them

Both mechanisms will be explored in the policy routing page, as they fundamentally alter route propagation behavior.

BGP session establishment follows a rigorous state machine defined in RFC 4271. Understanding this state machine is crucial for diagnosing session establishment failures and interpreting BGP status outputs.

The BGP Finite State Machine (FSM) defines six states that every BGP session transitions through:

1. Idle State

The initial state for all BGP sessions. In this state:

• No resources allocated to the peer
• Waiting for a "start" event (administrative enable, configuration change)
• A failed session returns here after error processing
• An exponentially increasing retry timer prevents rapid reconnection attempts after failures

2. Connect State

The router is attempting to establish a TCP connection:

• TCP connection actively being initiated
• If TCP succeeds → Send OPEN message → Transition to OpenSent
• If TCP fails → Transition to Active state
• ConnectRetry timer running (typically 120 seconds)

3. Active State

TCP connection attempt failed; trying again:

• Listening for incoming TCP connections
• Periodically retrying outbound connection
• If incoming connection succeeds → Send OPEN → Transition to OpenSent
• If outbound connection succeeds → Send OPEN → Transition to OpenSent
• If ConnectRetry timer expires → Try again

4. OpenSent State

TCP established; OPEN message sent; waiting for peer's OPEN:

• Waiting for OPEN message from peer
• If valid OPEN received → Send KEEPALIVE → Transition to OpenConfirm
• If invalid OPEN → Send NOTIFICATION → Transition to Idle
• HoldTimer running (typically 90 seconds initially)

5. OpenConfirm State

OPEN exchange complete; waiting for KEEPALIVE confirmation:

• Waiting for KEEPALIVE from peer
• If KEEPALIVE received → Transition to Established
• If NOTIFICATION received → Transition to Idle
• If HoldTimer expires → Send NOTIFICATION → Transition to Idle

6. Established State

The session is fully operational:

• Routes can be exchanged (UPDATE messages)
• KEEPALIVE messages exchanged periodically
• NOTIFICATION terminates session immediately
• This is the normal operational state

Troubleshooting with the FSM:

Understanding where a session is stuck reveals the nature of the problem:

Session Oscillation:

A session that repeatedly transitions between states (flapping) causes route instability affecting all downstream routers. BGP implementations include dampening mechanisms:

• Idle Hold Timer: Exponentially increases after each failure (30s → 60s → 120s...)
• Penalty-Based Dampening: Flapping routes receive penalties; suppressed when threshold exceeded

BGP uses four message types to establish sessions, exchange routes, and maintain connectivity. Each message has a common header followed by message-specific fields.

Common BGP Header (19 bytes minimum):

+-------------------------------------------+
| Marker (16 bytes - all 1s for auth/sync) |
+-------------------------------------------+
| Length (2 bytes) | Type (1 byte)          |
+-------------------------------------------+
| Message Body (variable)                   |
+-------------------------------------------+

The marker field was originally designed for authentication but is now always all 1s. The length field includes the header itself. The type field identifies one of four message types:

1. OPEN Message

The OPEN message initiates the BGP session and contains critical parameters:

• Version: BGP version number (currently 4)
• My Autonomous System: The sender's AS number (2 bytes, extended to 4 bytes via capability)
• Hold Time: Maximum time between messages before session is declared dead
• BGP Identifier: Router ID (typically highest loopback IP or configured value)
• Optional Parameters: Capabilities (multiprotocol, 4-byte AS, route refresh, etc.)

The Hold Time negotiation is critical: both peers send their preferred hold time, and the minimum of the two values is used. A hold time of 0 means keepalives are disabled (used only in special cases).

2. UPDATE Message

The UPDATE message carries routing information:

• Withdrawn Routes Length: Size of withdrawn routes field
• Withdrawn Routes: NLRI (prefix/length) of routes being removed
• Total Path Attribute Length: Size of path attributes
• Path Attributes: Attributes applying to the announced routes (AS_PATH, NEXT_HOP, etc.)
• Network Layer Reachability Information (NLRI): Prefixes being announced

A single UPDATE can announce multiple prefixes (sharing the same attributes) and withdraw multiple other prefixes. BGP implementations optimize by batching related updates.

3. NOTIFICATION Message

The NOTIFICATION message reports errors and always terminates the session:

• Error Code: Major error category
• Error Subcode: Specific error within the category
• Data: Optional diagnostic data (variable length)

Common error codes include Message Header Error, OPEN Message Error, UPDATE Message Error, Hold Timer Expired, Finite State Machine Error, and Cease (administrative shutdown).

4. KEEPALIVE Message

The simplest BGP message—just the 19-byte header with type 4:

• Sent periodically (default: 1/3 of the negotiated hold time)
• Resets the hold timer on the receiving peer
• If no message (UPDATE or KEEPALIVE) received within the hold time, session is terminated

KEEPALIVEs serve as a heartbeat, detecting TCP connectivity issues that might not trigger immediate TCP disconnection.

Let's trace through the complete sequence of establishing a BGP session between two routers—Router A (AS 65001, IP 10.0.0.1) and Router B (AS 65002, IP 10.0.0.2).

Phase 1: TCP Connection Establishment

Router A (10.0.0.1)          Router B (10.0.0.2)
      |                              |
      |------- SYN (dst:179) ------->|
      |<------ SYN-ACK ------------- |
      |------- ACK ----------------->|
      |                              |
      | TCP Connection Established   |

Both routers may initiate connections simultaneously. If both succeed, the router with the higher BGP identifier keeps the connection it initiated; the other is closed.

Phase 2: OPEN Message Exchange

Router A                     Router B
      |                              |
      |------- OPEN ----------------->|
      |  Version: 4                   |
      |  My AS: 65001                 |
      |  Hold Time: 90                |
      |  BGP ID: 1.1.1.1              |
      |  Capabilities: [...]          |
      |                              |
      |<------ OPEN -----------------||
      |  Version: 4                   |
      |  My AS: 65002                 |
      |  Hold Time: 180               |
      |  BGP ID: 2.2.2.2              |
      |  Capabilities: [...]          |

Each router validates the peer's OPEN message:

• Version must be 4
• AS number must match configuration (what we expect from this peer)
• Hold time must be 0 or at least 3 seconds
• Capabilities must be compatible

The negotiated hold time is min(90, 180) = 90 seconds.

Phase 3: KEEPALIVE Confirmation

Router A                     Router B
      |                              |
      |------- KEEPALIVE ------------>|
      |<------ KEEPALIVE -------------||
      |                              |
      | Session Established!          |

The KEEPALIVE messages confirm that both sides successfully processed the OPEN messages and are ready for UPDATE exchange.

Phase 4: Initial Route Exchange

Router A                     Router B
      |                              |
      |------- UPDATE --------------->|
      |  (Full BGP table from A)      |
      |<------ UPDATE ----------------||
      |  (Full BGP table from B)      |
      |                              |
      | Steady State: Incremental     |
      | Updates and Periodic          |
      | KEEPALIVEs                    |

After establishment, each router sends its complete BGP table (subject to outbound policies). This initial exchange can take significant time for routers with large tables (hundreds of thousands of routes). Once complete, only incremental updates are exchanged.

Session Termination Scenarios:

BGP sessions terminate through one of several mechanisms:

Graceful Restart (RFC 4724):

Traditional BGP session loss causes immediate withdrawal of all routes learned from that peer—potentially causing significant traffic disruption. Graceful Restart addresses this:

1. During session establishment, routers advertise Graceful Restart capability
2. If a peer fails but had GR capability, the local router marks routes as "stale" but keeps them in the routing table
3. A restart timer begins (typically 120 seconds)
4. If the peer re-establishes and resends the routes, traffic continues uninterrupted
5. If the timer expires, stale routes are withdrawn

Graceful Restart is critical for planned maintenance—restarting BGP processes without causing traffic loss.

Proper BGP session configuration is essential for security, stability, and optimal performance. This section covers key configuration parameters and best practices.

Essential Configuration Parameters:

1. Neighbor Specification

BGP requires explicit configuration of each peer:

router bgp 65001
  neighbor 10.0.0.2 remote-as 65002    ! eBGP peer
  neighbor 192.168.1.2 remote-as 65001  ! iBGP peer

2. Authentication (MD5)

BGP session authentication prevents unauthorized peers:

neighbor 10.0.0.2 password SECRET_KEY

MD5 authentication signs each TCP segment with a shared secret. While not cryptographically strong by modern standards, it remains widely used and provides protection against casual attacks and misconfigurations.

3. Source Address Binding

For iBGP and multihop eBGP, binding sessions to loopback addresses improves resilience:

neighbor 10.0.0.2 update-source Loopback0
neighbor 10.0.0.2 ebgp-multihop 2

Loopback-based sessions survive individual link failures as long as any path exists.

4. Timer Tuning

neighbor 10.0.0.2 timers 10 30  ! keepalive 10s, hold 30s

Shorter timers detect failures faster but increase false positive risk.

5. Maximum Prefix Protection

Prevents a peer from overwhelming your router:

neighbor 10.0.0.2 maximum-prefix 100000 80 restart 15

This configuration:

• Warns at 80% of 100,000 prefixes
• Terminates session if 100,000 exceeded
• Automatically restores session after 15 minutes

Production Best Practices:

1. Always Use Authentication: Even simple MD5 prevents accidental peering and basic attacks

2. Use Loopback Addresses for iBGP: Multiple physical paths = resilient sessions

3. Set Maximum Prefix Limits: Protect against misconfigurations and hijacking attempts

4. Configure Appropriate Timers: Balance failure detection speed against stability

5. Enable Logging and SNMP Traps: Monitor session state changes

6. Document Peering Configurations: Track AS numbers, IP addresses, and policies

7. Use Route Filtering: Never accept or advertise default routes accidentally

8. Consider BFD for Fast Failure Detection: Sub-second failure detection without aggressive BGP timers

Effective BGP session monitoring enables proactive maintenance and rapid problem resolution. This section covers diagnostic approaches for common session issues.

Key Metrics to Monitor:

1. Session State: Should always be Established for operational peers
2. Session Uptime: Long uptime indicates stability; frequent resets indicate problems
3. Prefix Counts: Prefixes received, accepted, advertised—sudden changes signal issues
4. Message Counters: Opens, Updates, Notifications—notification counts indicate errors
5. Memory Usage: Large BGP tables consume significant memory

Diagnostic Command Patterns (Cisco IOS):

! View all BGP neighbors and their states
show ip bgp summary

! Detailed information about a specific neighbor  
show ip bgp neighbor 10.0.0.2

! View BGP table for specific prefix
show ip bgp 192.0.2.0/24

! Debug BGP session establishment (use carefully!)
debug ip bgp

Common Problems and Solutions:

Understanding NOTIFICATION Error Codes:

NOTIFICATION messages provide diagnostic information through error codes:

Example: Diagnosing a Session Problem

Scenario: BGP session between R1 (AS 65001) and R2 (AS 65002) won't establish.

1. Check session state on R1: show ip bgp neighbor 10.0.0.2 → Shows "Active"
2. Verify TCP connectivity: telnet 10.0.0.2 179 → Connection refused
3. Check for ACLs: show access-list → Found ACL blocking TCP 179
4. Remove ACL or add permit statement
5. Verify session now Established

BGP sessions form the foundation of all inter-domain routing communication. Understanding session mechanics is essential for designing, operating, and troubleshooting networks that participate in Internet routing.

What's Next:

With a solid understanding of BGP session mechanics, we'll explore how BGP advertises routes in the next page. You'll learn about UPDATE message structure, path attributes, route announcement and withdrawal, and how BGP efficiently propagates routing information across the Internet.