[c-nsp] Offering bgp services from the L3 access edge with relatively low-spec devices using bgp selective route download ?

[Spyros Kakaroukas]

Hey all,

Lately I’ve been pondering the idea of offering full bgp feed to customers from our L3 access edge ( consisting mostly of ME3600Xs ) using bgp selective route download ( http://www.cisco.com/en/US/docs/ios-xml/ios/iproute_bgp/configuration/15-s/irg-selective-download.html ) , instead of xconnecting said customers to something beefier.

The main idea is as follows. BGP SRD ( abbreviated as such from now on ) allows us to get a full table ( as long as it can fit in the RAM ) and filter it from populating the RIB/FIB . It was introduced mostly for high RR scalability in dedicated RR deployments. What if we load the full bgp table on a 3600 and then use said 3600 to offer it to directly connected customers, while we prevent the TCAM from getting overflowed by utilizing BGP SRD ?

Potential benefits : One xconnect less. One label less. Less config at the core ( or whatever’s the next level in your hierarchy ) / internet edge / wherever you terminate your bgp customers, more at the access edge. The ability to do maintenance on a big box without either disrupting customers that would terminate their bgp session to it or setting up a more complex deployment with multiple peering sessions. The look of your coworkers’ faces when you feed a full table on a 3600 and it doesn’t blow up ;)

Potential gotchas : We’re advertising routes we possibly can’t reach, which could lead to us blackholing traffic. However, if we design it right ( filter only transit routes and potentially peer ones ) , we’re not really worse than just using defaults, which we would anyway. Quite a bit higher RAM utilization, though in most deployment scenarios you’ll probably run out of TCAM space before you run out of RAM.

The config :

Fairly simple. Peer as you normal would. Filter as follows:

route-map BGP_FILTER deny 10
match <however you want to classify routes your want to filter. As-path, communities, whatever>
route-map BGP_FILTER permit 20
router bgp <AS>
address-family ipv4
table-map BGP_FILTER filter

The test:

ME3600X peered to two BGP speakers that have the full table plus a simulated customer. Customer peering is not visible in the following output because it was running on my laptop, which wasn’t in the network at the time of this writing.

BGPTEST-PE#sh ver
Cisco IOS Software, ME360x Software (ME360x-UNIVERSALK9-M), Version 15.3(1)S1, RELEASE SOFTWARE (fc1)


BGPTEST-PE#sh ip bgp summ
BGP router identifier 172.18.0.23, local AS number x
BGP table version is 1722298, main routing table version 1722298
486834 network entries using 70104096 bytes of memory
504699 path entries using 40375920 bytes of memory
83639/79684 BGP path/bestpath attribute entries using 12044016 bytes of memory
73474 BGP AS-PATH entries using 2664012 bytes of memory
324 BGP route-map cache entries using 11664 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
BGP using 125199708 total bytes of memory
BGP activity 1467160/980326 prefixes, 1593903/1089204 paths, scan interval 60 secs
Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
172.16.0.0      4       x 47388     516  1722298    0    0 07:42:30    35354
172.16.0.1      4       x 136609     519  1722298    0    0 07:42:30   469345

A 3600 shouldn’t be able to handle that…or should it ?

BGPTEST-PE#sh ip route summary
IP routing table name is default (0x0)
IP routing table maximum-paths is 4
Route Source    Networks    Subnets     Replicates  Overhead    Memory (bytes)
connected       0           3           0           180         540
static          0           0           0           0           0
isis            3           210         0           12780       38340
  Level 1: 0 Level 2: 213 Inter-area: 0
bgp x      1           14          0           900         2700
  External: 0 Internal: 15 Local: 0
internal        12                                              13040
Total           16          227         0           13860       54620

Not that many BGP nets made it in the RIB after all.

BGPTEST-PE#sh platform tcam utilization ucastv4
Nile Tcam Utilization per Application & Region:
ES == Entry size == Number of 80 bit TCAM words
==================================================================
App/Region            Start  Num Avail  ES    Used Range  Num Used
==================================================================
UCASTV4                   0     20480   1
    nile0                                                      238
    nile1                                                      238

They didn’t make it into the FIB either.

BGPTEST-PE#sh proc memory sorted
Processor Pool Total:  878140880 Used:  290813580 Free:  587327300
      I/O Pool Total:   33546240 Used:   20264056 Free:   13282184
 PID TTY  Allocated      Freed    Holding    Getbufs    Retbufs Process
326   0  480323644  276217088  154612944          0          0 BGP Router
   0   0 1386240456 1257666092  124974456          0          0 *Init*
330   0   11084372          0   11094532          0          0 BGP Event

RAM utilization is increased as expected.

BGPTEST-PE#sh proc cpu
CPU utilization for five seconds: 5%/0%; one minute: 3%; five minutes: 3%

CPU utilization is also pretty low. There was the expected spike there while bgp was converging though.

Reachability was fine. NLRI processing was fine. Simulated a customer advertised a /24 from another ASN and it was reachable from the internet.

It’s going to take a bit more labbing before we decide whether we want to actually implement this or not, but it seemed like an interesting idea so I thought I’d share.

Feedback would be appreciated!



My thoughts and words are my own.

Kind Regards,

Spyros



This e-mail and any attachment(s) contained within are confidential and are intended only for the use of the individual to whom they are addressed. The information contained in this communication may be privileged, or exempt from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this communication in error, please notify the sender and delete the communication without retaining any copies. Rolaware Hellas SA is not responsible for, nor endorses, any opinion, recommendation, conclusion, solicitation, offer or agreement or any information contained in this communication.