[c-nsp] Loop/Unreachable problem with C6500/SUP720

[Sebastian Wiesinger]

Hello,

I'm having a strange problem with a Cisco 6500/SUP720 running
12.2(33)SXJ3.

Currently we're testing this router in the lab. We have one OSPF
connection to the outside and iBGP enabled.

As soon as I enable the iBGP i get really strange effects:

I have 10.1.66.0/25 connected to a SVI (Vlan412) and all IPs are
terminated on one server for testing.

Now when I ping some IPs I get a normal answer:


$ ping 10.1.66.51
PING 10.1.66.51 (10.1.66.51) 56(84) bytes of data.
64 bytes from 10.1.66.51: icmp_req=1 ttl=60 time=3.88 ms
64 bytes from 10.1.66.51: icmp_req=2 ttl=60 time=3.91 ms

And now the problem:

$ ping 10.1.66.84
PING 10.1.66.84 (10.1.66.84) 56(84) bytes of data.
>From 10.2.14.9 icmp_seq=1 Time to live exceeded
>From 10.2.14.9 icmp_seq=2 Time to live exceeded
>From 10.2.14.9 icmp_seq=3 Time to live exceeded

A traceroute shows the packet is looping, the lab router is sending it
back to it's OSPF peer (instead of sending it out through Vl412) who
sends it straight back:

 5  10.2.14.10  4.592 ms  4.561 ms  4.575 ms
 6  10.2.14.9  4.996 ms  4.162 ms  4.240 ms
 7  10.2.14.10  4.204 ms  4.135 ms  4.156 ms
 8  10.2.14.9  4.273 ms  4.193 ms  4.106 ms
 9  10.2.14.10  4.015 ms  4.237 ms  4.156 ms
...

Issuing various commands I can see no difference between the two IPs
on the router:

lab-rtr1#show ip route 10.1.66.51                          
Routing entry for 10.1.66.0/25
  Known via "connected", distance 0, metric 0 (connected, via interface)
  Routing Descriptor Blocks:
  * directly connected, via Vlan412
      Route metric is 0, traffic share count is 1

lab-rtr1#show ip route 10.1.66.84
Routing entry for 10.1.66.0/25
  Known via "connected", distance 0, metric 0 (connected, via interface)
  Routing Descriptor Blocks:
  * directly connected, via Vlan412
      Route metric is 0, traffic share count is 1


CEF entries:


lab-rtr1#show ip cef adjacency Vlan412 10.1.66.51 internal 
IPv4 CEF is enabled for distributed and running
VRF base:
 64739 prefixes (64737/2 fwd/non-fwd)
 Default network 0.0.0.0/0
 Table id 0
 Database epoch:        7 (64739 entries at this epoch)

10.1.66.51/32, epoch 7, flags attached, refcount 4, per-destination sharing
  sources: Adj 
  feature space:
   NetFlow: Origin AS 0, Peer AS 0, Mask Bits 25
  subblocks:
   Adj source: IP adj out of Vlan412, addr 10.1.66.51 5136EEC0
    Dependent covered prefix type adjfib cover 10.1.66.0/25
  ifnums:
   Vlan412(180): 10.1.66.51
  path 5110F968, path list 5110C090, share 1/1, type adjacency prefix, for IPv4
  attached to Vlan412, adjacency IP adj out of Vlan412, addr 10.1.66.51 5136EEC0
  output chain: IP adj out of Vlan412, addr 10.1.66.51 5136EEC0

lab-rtr1#show ip cef adjacency Vlan412 10.1.66.84 internal 
IPv4 CEF is enabled for distributed and running
VRF base:
 64739 prefixes (64737/2 fwd/non-fwd)
 Default network 0.0.0.0/0
 Table id 0
 Database epoch:        7 (64739 entries at this epoch)

10.1.66.84/32, epoch 7, flags attached, refcount 4, per-destination sharing
  sources: Adj 
  feature space:
   NetFlow: Origin AS 0, Peer AS 0, Mask Bits 25
  subblocks:
   Adj source: IP adj out of Vlan412, addr 10.1.66.84 5136A6C0
    Dependent covered prefix type adjfib cover 10.1.66.0/25
  ifnums:
   Vlan412(180): 10.1.66.84
  path 51110C70, path list 5110D2F8, share 1/1, type adjacency prefix, for IPv4
  attached to Vlan412, adjacency IP adj out of Vlan412, addr 10.1.66.84 5136A6C0
  output chain: IP adj out of Vlan412, addr 10.1.66.84 5136A6C0

This is totally random. When I shut/no shut the SVI the problem goes
away and comes back soon after but on different IPs.

Clearing the CEF table has the same effect.

When I shut down BGP the problem seems to go away.

Also BGP sessions are flapping from time to time without apparent
reason.

Any idea what can cause this?

Next steps include switching to the other Supervisor installed and/or
downgrade IOS.

Regards

Sebastian


-- 
GPG Key: 0x93A0B9CE (F4F6 B1A3 866B 26E9 450A  9D82 58A2 D94A 93A0 B9CE)
'Are you Death?' ... IT'S THE SCYTHE, ISN'T IT? PEOPLE ALWAYS NOTICE THE SCYTHE.
            -- Terry Pratchett, The Fifth Elephant