Thursday, June 13, 2013

BGP disable-connected-check and ebgp-multihop

For eBGP neighborship to form between two routers, by default; the routers send a TCP ACK with a TTL of 1. then an Open message is sent to negotiate some parameters that BGP neighbors must know about each other like version number and capabilities etc.

For directly connected neighbors, it's pretty straight forward. But when using the loopback for form the the neighborship, there's an extra hop for that TCP session to be established.

Lets consider the following topology here



Forming neighborship through directly connected interfaces between R1 and R2 wouldn't be a problem with default configuration, all you need is TCP packets with TTL of 1. We can check that using the following command

R1#show ip bgp neighbors
BGP neighbor is 10.1.2.2,  remote AS 2, external link
  BGP version 4, remote router ID 2.2.2.2
  BGP state = Established, up for 00:06:07
  Last read 00:00:47, last write 00:00:49, hold time is 180, keepalive interval is 60 seconds
  Neighbor sessions:
    1 active, is multisession capable
  Neighbor capabilities:
    Route refresh: advertised and received(new)
    Four-octets ASN Capability: advertised and received
    Address family IPv4 Unicast: advertised and received
    Multisession Capability: advertised and received
  Message statistics, state Established:
    InQ depth is 0
    OutQ depth is 0

                         Sent       Rcvd
    Opens:                  1          1
    Notifications:          0          0
    Updates:                1          1
    Keepalives:             8          8
    Route Refresh:          0          0
    Total:                 10         10
  Default minimum time between advertisement runs is 30 seconds

  Address tracking is enabled, the RIB does have a route to 10.1.2.2
  Connections established 1; dropped 0
  Last reset never
  Transport(tcp) path-mtu-discovery is enabled
  Graceful-Restart is disabled
Connection state is ESTAB, I/O status: 1, unread input bytes: 0
Connection is ECN Disabled, Mininum incoming TTL 0, Outgoing TTL 1
Local host: 10.1.2.1, Local port: 29870
Foreign host: 10.1.2.2, Foreign port: 179
Connection tableid (VRF): 0
Maximum output segment queue size: 50

Enqueued packets for retransmit: 0, input: 0  mis-ordered: 0 (0 bytes)

Event Timers (current time is 0x296C9C):
Timer          Starts    Wakeups            Next
Retrans             9          0             0x0
TimeWait            0          0             0x0
AckHold             9          8             0x0
SendWnd             0          0             0x0
KeepAlive           0          0             0x0
GiveUp              0          0             0x0
PmtuAger            1          0        0x2CD508
DeadWait            0          0             0x0
Linger              0          0             0x0
ProcessQ            0          0             0x0

iss: 3161157976  snduna: 3161158210  sndnxt: 3161158210     sndwnd:  16151
irs: 1964609928  rcvnxt: 1964610181  rcvwnd:      16132  delrcvwnd:    252

SRTT: 210 ms, RTTO: 904 ms, RTV: 694 ms, KRTT: 0 ms
minRTT: 64 ms, maxRTT: 300 ms, ACK hold: 200 ms
Status Flags: active open
Option Flags: nagle, path mtu capable
IP Precedence value : 6

Datagrams (max data segment is 1460 bytes):
Rcvd: 18 (out of order: 0), with data: 11, total data bytes: 252
Sent: 21 (retransmit: 0, fastretransmit: 0, partialack: 0, Second Congestion: 0), with data: 11, total data bytes: 252
 Packets received in fast path: 0, fast processed: 0, slow path: 0
 fast lock acquisition failures: 0, slow path: 0


Now, we can form this neighborship over loopbacks. we'll just change the configuration a little bit and see what will happen.

Configuration has been changed so that both routers use their loopback to establish the BGP peering


router bgp 1
 no synchronization
 bgp log-neighbor-changes
 neighbor 2.2.2.2 remote-as 2
 neighbor 2.2.2.2 update-source Loopback0
 no auto-summary

R1#show ip bgp neighbors 2.2.2.2
BGP neighbor is 2.2.2.2,  remote AS 2, external link
  BGP version 4, remote router ID 0.0.0.0
  BGP state = Idle
  Neighbor sessions:
    0 active, is multisession capable
  Default minimum time between advertisement runs is 30 seconds
 For address family: IPv4 Unicast
  BGP table version 1, neighbor version 1/0
  Output queue size : 0
  Index 0
                                 Sent       Rcvd
  Prefix activity:               ----       ----
    Prefixes Current:               0          0
    Prefixes Total:                 0          0
    Implicit Withdraw:              0          0
    Explicit Withdraw:              0          0
    Used as bestpath:             n/a          0
    Used as multipath:            n/a          0
                                   Outbound    Inbound
  Local Policy Denied Prefixes:    --------    -------
    Total:                                0          0
  Number of NLRIs in the update sent: max 0, min 0
  Address tracking is enabled, the RIB does not have a route to 2.2.2.2
  Connections established 0; dropped 0
  Last reset never
  External BGP neighbor not directly connected.
  Transport(tcp) path-mtu-discovery is enabled
  Graceful-Restart is disabled
  No active TCP connection
 
 Now there's no active TCP connection between the two routers, and the router knows that this IP isn't on any of it's connected interfaces, so it must be "not directly connected". let's make it reachable for both routers by configuring static routes to each other loopbacks.

Note: I'll remove some of the unneeded outputs to make the output more to the point of discussion


R1(config)#ip route 2.2.2.2 255.255.255.255 10.1.2.2
R1#show ip bgp neighbors
BGP neighbor is 2.2.2.2,  remote AS 2, external link
  BGP version 4, remote router ID 0.0.0.0
  BGP state = Idle
  Neighbor sessions:
    0 active, is multisession capable
  Default minimum time between advertisement runs is 30 seconds
 For address family: IPv4 Unicast
  BGP table version 1, neighbor version 1/0
  Output queue size : 0
  Index 0

  Number of NLRIs in the update sent: max 0, min 0
  Address tracking is enabled, the RIB does have a route to 2.2.2.2
  Connections established 0; dropped 0
  Last reset never
  External BGP neighbor not directly connected.
  Transport(tcp) path-mtu-discovery is enabled
  Graceful-Restart is disabled
  No active TCP connection



Now the router seems to have a route to it's neighbor, but the TCP connection won't come up. well the reason is, as we mentioned before , the default behavior for EBGP neighborship is to send packets with TTL of 1.  

For that to work, there are two solutions that might look very similar, but actually they are intended for different purposes, disable-connected-check and ebgp-multihop commands.

 disable-connected-check still sends packets with TTL of 1, but the directly connected router must have the peering address configured on it. while ebgp-multihop can allow the packet to traverse through it to a router behind it (up to 255 hops). 


here's what it looks like on R1

R1#sho ip bgp neighbors
BGP neighbor is 2.2.2.2,  remote AS 2, external link
  BGP version 4, remote router ID 2.2.2.2
  BGP state = Established, up for 00:17:07
  Last read 00:00:12, last write 00:00:17, hold time is 180, keepalive interval is 60 seconds
  Neighbor sessions:
    1 active, is multisession capable
  Address tracking is enabled, the RIB does have a route to 2.2.2.2
  Connections established 2; dropped 1
  Last reset 00:17:29, due to User reset of session 1
  External BGP neighbor not directly connected.
  Transport(tcp) path-mtu-discovery is enabled
  Graceful-Restart is disabled
Connection state is ESTAB, I/O status: 1, unread input bytes: 0
Connection is ECN Disabled, Mininum incoming TTL 0, Outgoing TTL 1

as you can see, the router sends packets with TTL of 1, now let's see how ebgp-multihop will work



router bgp 1
 no synchronization
 bgp log-neighbor-changes
 neighbor 2.2.2.2 remote-as 2
 neighbor 2.2.2.2 ebgp-multihop 2
 neighbor 2.2.2.2 update-source Loopback0
 no auto-summary

R1#show ip bgp neighbors
BGP neighbor is 2.2.2.2,  remote AS 2, external link
  BGP version 4, remote router ID 2.2.2.2
  BGP state = Established, up for 00:37:21
  Last read 00:00:20, last write 00:00:38, hold time is 180, keepalive interval is 60 seconds
  Address tracking is enabled, the RIB does have a route to 2.2.2.2
  Connections established 2; dropped 1
  Last reset 00:37:43, due to User reset of session 1
  External BGP neighbor may be up to 2 hops away.
  Transport(tcp) path-mtu-discovery is enabled
  Graceful-Restart is disabled
Connection state is ESTAB, I/O status: 1, unread input bytes: 0
Connection is ECN Disabled, Mininum incoming TTL 0, Outgoing TTL 2
Local host: 1.1.1.1, Local port: 39820
Foreign host: 2.2.2.2, Foreign port: 179