[nsp] eBGP routes not balancing
Jeff Chan
cisco-nsp at jeffchan.com
Mon Nov 17 02:50:31 EST 2003
Hi all,
We're running multihomed with two DS3s and have been for a while.
After we reset our peering session with one upstream our outbound
traffic no longer balances correctly. Both are large providers
and we're getting similarly full routes from both. Our route maps
are stamping identical MEDs and localprefs on all the incoming
routes but the bestpath selection is clearly skewed towards one
(AT&T which was reset least recently). I'm wondering if anyone
has any ideas how to correct this.
A couple possibilities are that the BGP decision algorithm is
reaching near the bottom where peer addresses are considered
and the peer with the lower numerical IP address is chosen,
or there is some kind of historical precedence issue where the
older route is favored.
I'm hoping this is a fairly easy one for you guys, but hope
it's not too obvious. :) Here are some configs. This is
on a 7513 with CX-HIPs for the DS3s. IOS is 12.2(14)S3.
CEF is currently disabled for other compatibility issues
but presumably is irrelevant for CX boards, and the balancing
was a lot closer to even before with the same hardware, IOS,
and configs....
interface Hssi4/0
description AT&T DS3
bandwidth 10000
ip address 12.118.243.166 255.255.255.252
serial restart-delay 0
no cdp enable
hold-queue 512 in
hold-queue 512 out
interface Hssi11/0
description Sprint DS3
bandwidth 12000
ip address 144.228.52.182 255.255.255.252
serial restart-delay 0
no cdp enable
hold-queue 512 in
hold-queue 512 out
router bgp 4150
no synchronization
bgp always-compare-med
bgp log-neighbor-changes
bgp dampening
network nnn...
neighbor 12.118.243.165 remote-as 7018
neighbor 12.118.243.165 description AT&T
neighbor 12.118.243.165 version 4
neighbor 12.118.243.165 prefix-list SUPRANET-NETWORKS out
neighbor 12.118.243.165 route-map STANDARD-ROUTE in
neighbor 12.118.243.165 filter-list 11 out
neighbor 144.228.52.181 remote-as 1239
neighbor 144.228.52.181 description Sprint
neighbor 144.228.52.181 version 4
neighbor 144.228.52.181 prefix-list SUPRANET-NETWORKS out
neighbor 144.228.52.181 route-map STANDARD-ROUTE in
neighbor 144.228.52.181 filter-list 11 out
no auto-summary
route-map STANDARD-ROUTE permit 10
match ip address 10
set metric 10
set local-preference 100
supranet01>sh ip bgp sum
BGP router identifier 66.170.0.1, local AS number 4150
BGP table version is 14615826, main routing table version 14615826
126321 network entries using 12758421 bytes of memory
378215 path entries using 18154320 bytes of memory
142988 BGP path attribute entries using 8579640 bytes of memory
63915 BGP AS-PATH entries using 1673176 bytes of memory
1 BGP extended community entries using 24 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
1 BGP filter-list cache entries using 12 bytes of memory
BGP using 41165593 total bytes of memory
Dampening enabled. 189 history paths, 176 dampened paths
BGP activity 387767/261446 prefixes, 5443140/5064925 paths, scan interval 60 secs
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
12.118.243.165 4 7018 2264780 95888 14615812 0 0 1d23h 125820
144.228.52.181 4 1239 5113516 172568 14615826 0 0 1d23h 126054
[...]
supranet01>sh ip bgp 4.0.0.0
BGP routing table entry for 4.0.0.0/8, version 14369789
Paths: (3 available, best #2, table Default-IP-Routing-Table)
Not advertised to any peer
1239 3356, (aggregated by 3356 4.68.0.3)
144.228.52.181 from 144.228.52.181 (144.228.243.27)
Origin IGP, metric 10, localpref 100, valid, external, atomic-aggregate
7018 3356, (aggregated by 3356 4.68.0.3)
12.118.243.165 from 12.118.243.165 (12.123.216.79)
Origin IGP, metric 10, localpref 100, valid, external, atomic-aggregate, best
701 3356, (aggregated by 3356 4.68.0.3)
137.39.7.236 from 137.39.7.236 (137.39.7.236)
Origin IGP, metric 200, localpref 50, valid, external, atomic-aggregate
(UUNet (AS701) is used as a backup connection. With a localpref of 50
their routes won't have any effect unless the others peers are down.)
The bestpath stats show Sprint routes being selected much less of
the time (nearly 1 in 3):
AT&T:
Used as bestpath: n/a 89915
Sprint:
Used as bestpath: n/a 36170
supranet01>sh ip bgp neigh
BGP neighbor is 12.118.243.165, remote AS 7018, external link
Description: AT&T
BGP version 4, remote router ID 12.123.216.79
BGP state = Established, up for 1d23h
Last read 00:00:16, hold time is 180, keepalive interval is 60 seconds
Neighbor capabilities:
Route refresh: advertised and received(old & new)
Address family IPv4 Unicast: advertised and received
Message statistics:
InQ depth is 0
OutQ depth is 0
Sent Rcvd
Opens: 4 4
Notifications: 0 0
Updates: 4 2168915
Keepalives: 95882 95876
Route Refresh: 0 0
Total: 95890 2264795
Default minimum time between advertisement runs is 30 seconds
For address family: IPv4 Unicast
BGP table version 14615847, neighbor version 14615836
Index 5, Offset 0, Mask 0x20
Inbound path policy configured
Outbound path policy configured
Outgoing update prefix filter list is SUPRANET-NETWORKS
Outgoing update AS path filter list is 11
Route map for incoming advertisements is STANDARD-ROUTE
Sent Rcvd
Prefix activity: ---- ----
Prefixes Current: 3 125820 (Consumes 6042912 bytes)
Prefixes Total: 3 229063
Implicit Withdraw: 0 84939
Explicit Withdraw: 0 18304
Used as bestpath: n/a 89915
Used as multipath: n/a 0
History paths: n/a 74 (Consumes 3552 bytes)
Outbound Inbound
Local Policy Denied Prefixes: -------- -------
prefix-list 205140 0
AS_PATH loop: n/a 1
Bestpath from this peer: 201224 n/a
No source path table ID: 205143 n/a
Total: 611507 1
Number of NLRIs in the update sent: max 3, min 0
Connections established 4; dropped 3
Last reset 1d23h, due to User reset
Connection state is ESTAB, I/O status: 1, unread input bytes: 0
Local host: 12.118.243.166, Local port: 13032
Foreign host: 12.118.243.165, Foreign port: 179
Enqueued packets for retransmit: 0, input: 0 mis-ordered: 0 (0 bytes)
Event Timers (current time is 0x265BC2814):
Timer Starts Wakeups Next
Retrans 2957 94 0x0
TimeWait 0 0 0x0
AckHold 15343 13121 0x0
SendWnd 0 0 0x0
KeepAlive 0 0 0x0
GiveUp 0 0 0x0
PmtuAger 0 0 0x0
DeadWait 0 0 0x0
iss: 1043433219 snduna: 1043487684 sndnxt: 1043487684 sndwnd: 15263
irs: 3775095062 rcvnxt: 3779964682 rcvwnd: 16384 delrcvwnd: 0
SRTT: 300 ms, RTTO: 303 ms, RTV: 3 ms, KRTT: 0 ms
minRTT: 0 ms, maxRTT: 736 ms, ACK hold: 200 ms
Flags: higher precedence, nagle
Datagrams (max data segment is 4430 bytes):
Rcvd: 17875 (out of order: 0), with data: 15671, total data bytes: 4869619
Sent: 17053 (retransmit: 94, fastretransmit: 0), with data: 2862, total data byt
es: 54464
BGP neighbor is 144.228.52.181, remote AS 1239, external link
Description: Sprint
BGP version 4, remote router ID 144.228.243.27
BGP state = Established, up for 1d23h
Last read 00:00:11, hold time is 135, keepalive interval is 45 seconds
Neighbor capabilities:
Route refresh: advertised and received(old & new)
Address family IPv4 Unicast: advertised and received
Message statistics:
InQ depth is 0
OutQ depth is 0
Sent Rcvd
Opens: 3 3
Notifications: 0 0
Updates: 3 5113425
Keepalives: 172565 110
Route Refresh: 0 0
Total: 172571 5113538
Default minimum time between advertisement runs is 30 seconds
For address family: IPv4 Unicast
BGP table version 14615850, neighbor version 14615850
Index 4, Offset 0, Mask 0x10
Inbound path policy configured
Outbound path policy configured
Outgoing update prefix filter list is SUPRANET-NETWORKS
Outgoing update AS path filter list is 11
Route map for incoming advertisements is STANDARD-ROUTE
Sent Rcvd
Prefix activity: ---- ----
Prefixes Current: 3 126056 (Consumes 6052560 bytes)
Prefixes Total: 3 243574
Implicit Withdraw: 0 102838
Explicit Withdraw: 0 14680
Used as bestpath: n/a 36170
Used as multipath: n/a 0
History paths: n/a 39 (Consumes 1872 bytes)
Outbound Inbound
Local Policy Denied Prefixes: -------- -------
route-map: 0 1
prefix-list 177193 0
Bestpath from this peer: 59669 n/a
No source path table ID: 177196 n/a
Total: 414058 1
Number of NLRIs in the update sent: max 3, min 0
Connections established 3; dropped 2
Last reset 1d23h, due to User reset
Connection state is ESTAB, I/O status: 1, unread input bytes: 0
Local host: 144.228.52.182, Local port: 13033
Foreign host: 144.228.52.181, Foreign port: 179
Enqueued packets for retransmit: 0, input: 0 mis-ordered: 0 (0 bytes)
Event Timers (current time is 0x265BC2E70):
Timer Starts Wakeups Next
Retrans 4035 236 0x0
TimeWait 0 0 0x0
AckHold 12478 11328 0x0
SendWnd 0 0 0x0
KeepAlive 0 0 0x0
GiveUp 0 0 0x0
PmtuAger 0 0 0x0
DeadWait 0 0 0x0
iss: 3382838677 snduna: 3382910907 sndnxt: 3382910907 sndwnd: 31666
irs: 4244662677 rcvnxt: 4250898248 rcvwnd: 12709 delrcvwnd: 3675
SRTT: 322 ms, RTTO: 478 ms, RTV: 156 ms, KRTT: 0 ms
minRTT: 4 ms, maxRTT: 596 ms, ACK hold: 200 ms
Flags: higher precedence, nagle
Datagrams (max data segment is 4430 bytes):
Rcvd: 16731 (out of order: 0), with data: 13026, total data bytes: 6235570
Sent: 16408 (retransmit: 236, fastretransmit: 0), with data: 3798, total data by
tes: 72229
Any ideas or suggestions would be welcomed with gratitude.
Cheers,
Jeff C.
--
Jeff Chan
mailto:cisco-nsp at jeffchan.com
http://www.supranet.net/
More information about the cisco-nsp
mailing list