[c-nsp] Unusual behavior with CSR virtual Router and customer-triggered RTBHs
Matthew Melbourne
matt at melbourne.org.uk
Mon Nov 17 19:28:23 EST 2014
Hi Ryan,
Does it make a difference if you reconfigure your customer eBGP session as
an ebgp-multihop session, or with "neighbor x.x.x.x
disable-connected-check". I have seen issues when re-writing next-hop to a
destination which isn't expected on the eBGP transfer-net. The behaviour I
saw was that the /32 route in the BGP table was 'inaccessible' due to
next-hop re-write.
Cheers,
Matt
------------------------------
Message: 3
Date: Mon, 17 Nov 2014 11:43:26 -0800
From: Ryan McHugh <ryan.mchugh at gnomishthoughts.com>
To: cisco-nsp at puck.nether.net
Subject: [c-nsp] Unusual behavior with CSR virtual Router and
customer-triggered RTBHs
Message-ID:
<CAJqS+6B_HVeGmphAFZ4JdL9XVvaVHwQH7mA-ZhQcyb4QaFrdag at mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
I am attempting to configure (currently in a test environment with some IPs
masked in output) a Customer-Triggered RTBH with a CSR1000v as my test
bed. I am running in to an unusual problem.
To start, I have a working RTBH configuration that works when I generate a
local static route with tag using a redistribute static route-map in my bgp
configuration (based on
http://packetlife.net/blog/2009/jul/6/remotely-triggered-black-hole-rtbh-rou
ting/
):
route-map Redistribute-Static-to-BGP permit 1000
description Redistribute static blocks with tag 666 to BGP for RTBH and
forward to RTBH capable Peers
match tag 666
set origin igp
set community 64512:666
set ip next-hop 192.0.2.1
!
route-map Redistribute-Static-to-BGP permit 2000
description Redistribute static blocks with tag 6666 to BGP for
Source-based RTBH, does not get sent to peers
match tag 6666
set origin igp
set community 64512:6666
set ip next-hop 192.0.2.1
!
route-map Redistribute-Static-to-BGP permit 50000
description Redistribute Static with tag 2000 to BGP, add
Carry-Route-In-non-Full-Table-Routers Comm
match tag 2000
set origin igp
set community 64512:800 64512:30000 64512:31000
!
route-map Redistribute-Static-to-BGP permit 55000
description Redistribute Static with tag 64512 to OSPF AND BGP, add
Carry-Route-In-non-Full-Table-Routers Comm
match tag 64512
set origin igp
set community 64512:800 64512:30000 64512:31000
!
There are some additional tags that are not related to the RTBH but
included for completeness. This setup fully works as expected. The odd
behavior occurs when I try to allow my ?customer? to send a /32 route with
a RTBH request community.
The router (CSR0) receives the /32 route from the ?customer? device
properly, I convert the customer RTBH request community (64512:9999) to my
internal RTBH community (64512:666) and set next hop to 192.0.2.1 in the
following route-map:
route-map Customer-In deny 1000
description Block BOGONS from coming in
match ip address prefix-list BOGON
!
route-map Customer-In permit 50000
description Accept RTBH request from customer and pass along to carriers
that accept them
match ip address prefix-list RTBH-Customer-In
match community GT-Customer-Request-RTBH-Community
set community 64512:666
set ip next-hop 192.0.2.1
!
route-map Customer-In permit 55000
description default Accept rule for routes from 'customer'
set comm-list Remove-GT-Internal-Tags-From-Customer delete
set community 64512:30000 64512:31000 64512:31010 additive
!
The route properly hits the correct match prefix-list (locks announcements
down to only accept /32s) and appears to complete its task correctly,
however the Routes are not accepted into RIB and do not properly direct the
traffic to NULL0.
csr0#show ip bgp
BGP table version is 5, local router ID is D.E.F.2
Status codes: s suppressed, d damped, h history, * valid, > best, i -
internal,
r RIB-failure, S Stale, m multipath, b backup-path, f
RT-Filter,
x best-external, a additional-path, c RIB-compressed,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found
Network Next Hop Metric LocPrf Weight Path
0.0.0.0 0.0.0.0 0 i
*> A.B.C.0/23 <cust peer ip> 0 0 64555 i
* A.B.C.125/32 192.0.2.1 0 0 64555 i
*> D.E.F.0/23 0.0.0.0 0 32768 i
Customer AS 64555 is announcing A.B.C.0/23 which is properly received. As
you see, the A.B.C.125/32 route is not marked as ?best? (D.E.F.0/23 is a
local announced block from CSR0)
BGP routing table entry for A.B.C.125/32, version 0
Paths: (2 available, no best path)
Flag: 0x820
Not advertised to any peer
Refresh Epoch 1
64555
192.0.2.1 (inaccessible) from <cust peer ip> (<customer routerid>)
Origin IGP, metric 0, localpref 100, valid, external
Community: 64512:666
rx pathid: 0, tx pathid: 0
Refresh Epoch 1
64555, (received-only)
<cust peer ip> from <cust peer ip> (<customer routerid>)
Origin IGP, metric 0, localpref 100, valid, external
Community: 64512:9999
rx pathid: 0, tx pathid: 0
This of course prevents the RTBH function from working correctly. However,
if I add a local static route, for either the Customer A.B.C.0/23 or the
?carrier? block of D.E.F.0/23, initially I get nothing exciting, except the
?local? RTBH works:
csr0(config)#ip route D.E.F.244 255.255.255.255 null 0 tag 666 name
RTBH-Sacrificial-Helper
csr0(config)#end
csr0#show ip bgp
BGP table version is 6, local router ID is D.E.F.2
Status codes: s suppressed, d damped, h history, * valid, > best, i -
internal,
r RIB-failure, S Stale, m multipath, b backup-path, f
RT-Filter,
x best-external, a additional-path, c RIB-compressed,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found
Network Next Hop Metric LocPrf Weight Path
0.0.0.0 0.0.0.0 0 i
*> A.B.C.0/23 <cust peer ip> 0 0 64555 i
* A.B.C.125/32 192.0.2.1 0 0 64555 i
*> D.E.F.0/23 0.0.0.0 0 32768 i
*> D.E.F.244/32 192.0.2.1 0 32768 i
However, if I reset the bgp peering with the customer (hard clear, soft
does not work), the customer RTBH route works:
csr0#clear ip bgp <cust peer ip>
csr0#show ip bgp
BGP table version is 9, local router ID is D.E.F.2
Status codes: s suppressed, d damped, h history, * valid, > best, i -
internal,
r RIB-failure, S Stale, m multipath, b backup-path, f
RT-Filter,
x best-external, a additional-path, c RIB-compressed,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found
Network Next Hop Metric LocPrf Weight Path
0.0.0.0 0.0.0.0 0 i
*> A.B.C.0/23 <cust peer ip> 0 0 64555 i
*> A.B.C.125/32 192.0.2.1 0 0 64555 i
*> D.E.F.0/23 0.0.0.0 0 32768 i
*> D.E.F.244/32 192.0.2.1 0 32768 i
csr0#show ip bgp A.B.C.125/32
BGP routing table entry for A.B.C.125/32, version 8
Paths: (2 available, best #1, table default)
Advertised to update-groups:
54
Refresh Epoch 1
64555
192.0.2.1 from <cust peer ip> (<customer routerid>)
Origin IGP, metric 0, localpref 100, valid, external, best
Community: 64512:666
rx pathid: 0, tx pathid: 0x0
Refresh Epoch 1
64555, (received-only)
<cust peer ip> from <cust peer ip> (<customer routerid>)
Origin IGP, metric 0, localpref 100, valid, external
Community: 64512:9999
rx pathid: 0, tx pathid: 0
At this point the RTBH works as expected, until I remove the Sacrificial
route, shortly after words (at next pass through the table checking for
next-hop reachable I assume) the customer /32 route is removed from RIB and
the RTBH fails again.
Additional info:
csr0#show ver
Cisco IOS XE Software, Version 03.12.00.S - Standard Support Release
Cisco IOS Software, CSR1000V Software (X86_64_LINUX_IOSD-UNIVERSALK9-M),
Version 15.4(2)S, RELEASE SOFTWARE (fc2)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2014 by Cisco Systems, Inc.
Compiled Wed 26-Mar-14 21:09 by mcpre
Cisco IOS-XE software, Copyright (c) 2005-2014 by cisco Systems, Inc.
All rights reserved. Certain components of Cisco IOS-XE software are
licensed under the GNU General Public License ("GPL") Version 2.0. The
software code licensed under GPL Version 2.0 is free software that comes
with ABSOLUTELY NO WARRANTY. You can redistribute and/or modify such
GPL code under the terms of GPL Version 2.0. For more details, see the
documentation or "License Notice" file accompanying the IOS-XE software,
or the applicable URL provided on the flyer accompanying the IOS-XE
software.
ROM: IOS-XE ROMMON
csr0 uptime is 1 week, 6 days, 21 hours, 49 minutes
Uptime for this control processor is 1 week, 6 days, 21 hours, 49 minutes
System returned to ROM by reload
System image file is "bootflash:packages.conf"
Last reload reason: <NULL>
This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.
A summary of U.S. laws governing Cisco cryptographic products may be found
at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html
If you require further assistance please contact us by sending email to
export at cisco.com.
License Level: limited
License Type: Default. No valid license found.
Next reload license Level: limited
cisco CSR1000V (VXE) processor with 2170596K/6147K bytes of memory.
Processor board ID 9FIHGN2ARRJ
3 Gigabit Ethernet interfaces
32768K bytes of non-volatile configuration memory.
4194304K bytes of physical memory.
8822783K bytes of virtual hard disk at bootflash:.
Configuration register is 0x2102
I am now at the point where I am unable to figure out why the router is
working in this manner. Any additional pointers would be appreciated. If
need be I will try this on our ASRs during a maintenance window but I
prefer to understand what is going on. Obviously it could be a bug unique
to the CSR virtual appliance but I prefer to be safe. Any assistance would
be greatly appreciated.
--
Ryan McHugh
More information about the cisco-nsp
mailing list