[j-nsp] ex4200 routing issue

[Cord MacLeod]

I have a very strange routing issue in progress.  The short of it is, I am advertising 10.10.134.0/24 from gsw1.xxx with BGP, which is an ex4200 6 member switch.  I advertise this to crs1.xxx which is an ex4200 2 member switch and a route reflector.  When one of the crs1.xxx member switches dies, all routing dies as expected.  However when routing re-establishes for some reason 10.10.134.0/24 is no longer reachable even though a route exists for this network in BGP through out the whole network.  crs1.xxx even claims "no route to host" even when showing a route in it's routing table.

When the 2nd member of crs1.xxx comes back online, everything is fixed.  Obviously this is a major issue because I now essentially have no redundancy at my aggregation layer, crs1.xxx.


Here's normal operation:

root at crs1.xxx> show virtual-chassis    

Virtual Chassis ID: 0026.8868.4080
                                          Mastership            Neighbor List  
Member ID  Status   Serial No    Model    priority    Role      ID  Interface
0 (FPC 0)  Prsnt    BM0xxxxxxxxx0 ex4200-24t      200  Master*    1  vcp-0      
1 (FPC 1)  Prsnt    BM0xxxxxxxxx2 ex4200-24t      200  Backup     0  vcp-1      

root at crs1.xxx> show virtual-chassis 

root at crs1.xxx> show route 10.10.134.0 

inet.0: 20 destinations, 21 routes (20 active, 0 holddown, 0 hidden)
Restart Complete
+ = Active Route, - = Last Active, * = Both

10.10.134.0/24     *[BGP/170] 00:20:46, localpref 100, from 10.10.135.47
                      AS path: I
                    > to 10.10.135.58 via ge-1/0/23.0
                      to 10.10.135.62 via ge-0/0/23.0

root at crs1.xxx> show bgp summary 
Groups: 2 Peers: 3 Down peers: 0
Table          Tot Paths  Act Paths Suppressed    History Damp State    Pending
inet.0                 4          4          0          0          0          0
Peer                     AS      InPkt     OutPkt    OutQ   Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped...
10.10.135.13          11XXX         27         29       0       0       10:53 1/1/1/0              0/0/0/0
10.10.135.14          11XXX         26         29       0       0       10:47 1/1/1/0              0/0/0/0
10.10.135.47          11XXX         28         28       0       0       10:56 2/2/2/0              0/0/0/0


root at crs1.xxx> ping 10.10.134.1 
PING 10.10.134.1 (10.10.134.1): 56 data bytes
64 bytes from 10.10.134.1: icmp_seq=0 ttl=64 time=1.324 ms
64 bytes from 10.10.134.1: icmp_seq=1 ttl=64 time=1.570 ms
64 bytes from 10.10.134.1: icmp_seq=2 ttl=64 time=1.285 ms
64 bytes from 10.10.134.1: icmp_seq=3 ttl=64 time=1.297 ms


Here's abnormal operation:

Virtual Chassis ID: 0026.8868.4080
                                          Mastership            Neighbor List  
Member ID  Status   Serial No    Model    priority    Role      ID  Interface
0 (FPC 0)  Prsnt    BM0xxxxxxxxx0 ex4200-24t      200  Master*  
1 (FPC 1)  NotPrsnt BM0xxxxxxxxx2 ex4200-24t

root at crs1.xxx> show bgp summary    
Groups: 2 Peers: 3 Down peers: 0
Table          Tot Paths  Act Paths Suppressed    History Damp State    Pending
inet.0                 4          4          0          0          0          0
Peer                     AS      InPkt     OutPkt    OutQ   Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped...
10.10.135.13          11XXX          3          5       0       0           8 1/1/1/0              0/0/0/0
10.10.135.14          11XXX          2          5       0       0           2 1/1/1/0              0/0/0/0
10.10.135.47          11XXX          4          3       0       0          11 2/2/2/0              0/0/0/0

root at crs1.xxx> show route 10.10.134.0 

inet.0: 14 destinations, 15 routes (14 active, 0 holddown, 0 hidden)
Restart Complete
+ = Active Route, - = Last Active, * = Both

10.10.134.0/24     *[BGP/170] 00:03:35, localpref 100, from 10.10.135.47
                      AS path: I
                    > to 10.10.135.62 via ge-0/0/23.0

{master:0}
root at crs1.xxx> ping 10.10.134.1 
PING 10.10.134.1 (10.10.134.1): 56 data bytes
ping: sendto: No route to host
ping: sendto: No route to host
ping: sendto: No route to host
ping: sendto: No route to host
ping: sendto: No route to host
ping: sendto: No route to host