[c-nsp] 2851 and full BGP
Jay Nakamura
zeusdadog at gmail.com
Mon Aug 11 10:01:56 EDT 2008
To answer couple people's questions,
MTU on the routers are 1500. I have tested with ping and df-bit set.
Provider has higher frame size to cover that MTU over the WAN link and our
switches that connects to them on both ends have higher frame size. (1526
frame size or higher)
While I am at it, I noticed 12.4 line IOS for 28xx is MD release. Which,
cisco's link doesn't tell you what that means. I know GD, ED, etc releases
but wasn't sure what MD relase meant. Mainline deployment?
Anyway, is 12.4 the most stable way to go on 28xx? We are not using any
fancy features. One router is using NM-1T3/E3 card but that's about it.
Here are some output from both routers while exchanging just internal
routes.
border2-col#sh ip bgp neighbors Y.Y.Y.Y
BGP neighbor is Y.Y.Y.Y, remote AS ZZZZ, internal link
BGP version 4, remote router ID Y.Y.Y.Y
BGP state = Established, up for 3d03h
Last read 00:00:41, last write 00:00:49, hold time is 180, keepalive
interval is 60 seconds
Neighbor capabilities:
Route refresh: advertised and received(new)
Address family IPv4 Unicast: advertised and received
Message statistics:
InQ depth is 0
OutQ depth is 0
Sent Rcvd
Opens: 7 7
Notifications: 3 1
Updates: 171196 105628
Keepalives: 4581 4586
Route Refresh: 0 0
Total: 175787 110226
Default minimum time between advertisement runs is 0 seconds
For address family: IPv4 Unicast
BGP table version 887105, neighbor version 887105/0
Output queue size : 0
Index 3, Offset 0, Mask 0x8
3 update-group member
Inbound soft reconfiguration allowed
Outgoing update prefix filter list is COLUMBUS_NET
Sent Rcvd
Prefix activity: ---- ----
Prefixes Current: 7 9 (Consumes 468 bytes)
Prefixes Total: 8 9
Implicit Withdraw: 0 0
Explicit Withdraw: 1 0
Used as bestpath: n/a 9
Used as multipath: n/a 0
Outbound Inbound
Local Policy Denied Prefixes: -------- -------
prefix-list 535265 0
Bestpath from this peer: 9 n/a
Total: 535274 0
Number of NLRIs in the update sent: max 1024, min 0
Address tracking is enabled, the RIB does have a route to Y.Y.Y.Y
Connections established 7; dropped 6
Last reset 3d03h, due to BGP Notification received, illegal header length
Transport(tcp) path-mtu-discovery is enabled
Connection state is ESTAB, I/O status: 1, unread input bytes: 0
Connection is ECN Disabled, Mininum incoming TTL 0, Outgoing TTL 255
Local host: X.X.X.X, Local port: 51918
Foreign host: Y.Y.Y.Y, Foreign port: 179
Connection tableid (VRF): 0
Enqueued packets for retransmit: 0, input: 0 mis-ordered: 0 (0 bytes)
Event Timers (current time is 0x15C86EE0):
Timer Starts Wakeups Next
Retrans 4563 31 0x0
TimeWait 0 0 0x0
AckHold 4529 4183 0x0
SendWnd 0 0 0x0
KeepAlive 0 0 0x0
GiveUp 0 0 0x0
PmtuAger 1 1 0x0
DeadWait 0 0 0x0
Linger 0 0 0x0
ProcessQ 0 0 0x0
iss: 3264861958 snduna: 3264948248 sndnxt: 3264948248 sndwnd: 16023
irs: 3518332904 rcvnxt: 3518419120 rcvwnd: 16118 delrcvwnd: 266
SRTT: 301 ms, RTTO: 308 ms, RTV: 7 ms, KRTT: 0 ms
minRTT: 4 ms, maxRTT: 2824 ms, ACK hold: 200 ms
Status Flags: active open
Option Flags: nagle, path mtu capable
IP Precedence value : 6
Datagrams (max data segment is 536 bytes):
Rcvd: 8963 (out of order: 0), with data: 4530, total data bytes: 86215
Sent: 8919 (retransmit: 31, fastretransmit: 0, partialack: 0, Second
Congestion: 0), with data: 4532, total data bytes: 86289
Packets received in fast path: 0, fast processed: 0, slow path: 0
fast lock acquisition failures: 0, slow path: 0
border2-indy#sh ip bgp neighbors X.X.X.X
BGP neighbor is X.X.X.X, remote AS ZZZZ, internal link
BGP version 4, remote router ID X.X.X.X
BGP state = Established, up for 3d04h
Last read 00:00:39, last write 00:00:31, hold time is 180, keepalive
interval is 60 seconds
Neighbor capabilities:
Route refresh: advertised and received(new)
Address family IPv4 Unicast: advertised and received
Message statistics:
InQ depth is 0
OutQ depth is 0
Sent Rcvd
Opens: 9 9
Notifications: 1 4
Updates: 144559 224571
Keepalives: 4590 4585
Route Refresh: 0 0
Total: 149155 229172
Default minimum time between advertisement runs is 0 seconds
For address family: IPv4 Unicast
BGP table version 2377206, neighbor version 2377206/0
Output queue size : 0
Index 2, Offset 0, Mask 0x4
2 update-group member
Inbound soft reconfiguration allowed
Outgoing update prefix filter list is INDY_NET
Sent Rcvd
Prefix activity: ---- ----
Prefixes Current: 9 7 (Consumes 364 bytes)
Prefixes Total: 9 8
Implicit Withdraw: 0 0
Explicit Withdraw: 0 1
Used as bestpath: n/a 7
Used as multipath: n/a 0
Outbound Inbound
Local Policy Denied Prefixes: -------- -------
prefix-list 458047 0
Bestpath from this peer: 9 n/a
Total: 458056 0
Number of NLRIs in the update sent: max 1135, min 0
Address tracking is enabled, the RIB does have a route to X.X.X.X
Connections established 9; dropped 8
Last reset 3d04h, due to BGP Notification sent, illegal header length
Transport(tcp) path-mtu-discovery is enabled
Connection state is ESTAB, I/O status: 1, unread input bytes: 0
Connection is ECN Disabled, Mininum incoming TTL 0, Outgoing TTL 255
Local host: Y.Y.Y.Y, Local port: 179
Foreign host: X.X.X.X, Foreign port: 51918
Connection tableid (VRF): 0
Enqueued packets for retransmit: 0, input: 0 mis-ordered: 0 (0 bytes)
Event Timers (current time is 0x10A0F458):
Timer Starts Wakeups Next
Retrans 4578 46 0x0
TimeWait 0 0 0x0
AckHold 4532 4200 0x0
SendWnd 0 0 0x0
KeepAlive 0 0 0x0
GiveUp 0 0 0x0
PmtuAger 0 0 0x0
DeadWait 0 0 0x0
Linger 0 0 0x0
ProcessQ 0 0 0x0
iss: 3518332904 snduna: 3518419158 sndnxt: 3518419158 sndwnd: 16080
irs: 3264861958 rcvnxt: 3264948267 rcvwnd: 16004 delrcvwnd: 380
SRTT: 304 ms, RTTO: 335 ms, RTV: 31 ms, KRTT: 0 ms
minRTT: 4 ms, maxRTT: 468 ms, ACK hold: 200 ms
Status Flags: passive open, gen tcbs
Option Flags: nagle, path mtu capable
IP Precedence value : 6
Datagrams (max data segment is 536 bytes):
Rcvd: 8953 (out of order: 0), with data: 4533, total data bytes: 86308
Sent: 8920 (retransmit: 46, fastretransmit: 0, partialack: 0, Second
Congestion: 0), with data: 4532, total data bytes: 86253
Packets received in fast path: 0, fast processed: 0, slow path: 0
fast lock acquisition failures: 0, slow path: 0
On Mon, Aug 11, 2008 at 9:18 AM, Church, Charles <cchurc05 at harris.com>wrote:
> Oh, yeah. Sorry, I didn't catch the 'WAN' part of it the first time.
> That does make MTU a possibility. But didn't he get like 20% of his
> routes before the error message? Since it was 12.4(20)T (pretty
> bleeding edge), I'd lean towards that still. I'd think that an MTU
> problem would show up way before you got to 20%. Does BGP set the DF
> bit?
>
> Chuck
>
> -----Original Message-----
> From: Paul Cosgrove [mailto:paul.cosgrove at heanet.ie]
> Sent: Monday, August 11, 2008 4:33 AM
> To: Church, Charles
> Cc: mtinka at globaltransit.net; cisco-nsp at puck.nether.net
> Subject: Re: [c-nsp] 2851 and full BGP
>
>
> Hi Chuck,
>
> Jay will be able to clarify, but I took the following to mean that the
> two are separated via third party infrastructure: "two 2851s connected
> to each other over gigabit Ethernet WAN".
>
> May well be a bug though.
>
> Paul.
>
> Church, Charles wrote:
> > Wasn't the original problem the iBGP connection over his own network?
> Sounds like a bug more than anything else.
> >
> > Chuck
> >
> > ----- Original Message -----
> > From: cisco-nsp-bounces at puck.nether.net
> <cisco-nsp-bounces at puck.nether.net>
> > To: mtinka at globaltransit.net <mtinka at globaltransit.net>
> > Cc: cisco-nsp at puck.nether.net <cisco-nsp at puck.nether.net>
> > Sent: Sun Aug 10 15:52:03 2008
> > Subject: Re: [c-nsp] 2851 and full BGP
> >
> >
> > Keep in mind that if the peerings are not between directly connected
> IP,
> > disabling PMTUd for BGP will cause it to use an MSS of 536 bytes.
> >
> > You could check the achievable MTU using extended pings with the DF
> bit
> > set, and compare it with the segment size listed by BGP before you
> > decide whether to make that change.
> >
> > Paul.
> >
> > Mark Tinka wrote:
> >> On Saturday 09 August 2008 10:28:40 Jay Nakamura wrote:
> >>
> >>
> >>> Any ideas on what could be causing this issue? Is there
> >>> a better IOS version to use?
> >>>
> >> Sounds like an MTU issue.
> >>
> >> Try disabling TCP PMTUd for BGP and see if that helps:
> >>
> >> router bgp 1234
> >> no bgp transport path-mtu-discovery
> >>
> >> If that works, consider checking with your provider on the
> >> supported MTU, end-to-end, and adjust your interface MTU if
> >> it helps.
> >>
> >> Cheers,
> >>
> >> Mark.
> >>
> >>
> ------------------------------------------------------------------------
> >>
> >> _______________________________________________
> >> cisco-nsp mailing list cisco-nsp at puck.nether.net
> >> https://puck.nether.net/mailman/listinfo/cisco-nsp
> >> archive at http://puck.nether.net/pipermail/cisco-nsp/
> >
> > _______________________________________________
> > cisco-nsp mailing list cisco-nsp at puck.nether.net
> > https://puck.nether.net/mailman/listinfo/cisco-nsp
> > archive at http://puck.nether.net/pipermail/cisco-nsp/
>
>
> --
> HEAnet Limited
> Ireland's Education & Research Network
> 5 George's Dock, IFSC, Dublin 1, Ireland
> Tel: +353.1.6609040
> Web: http://www.heanet.ie
> Company registered in Ireland: 275301
>
> Please consider the environment before printing this e-mail.
> _______________________________________________
> cisco-nsp mailing list cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>
More information about the cisco-nsp
mailing list