[c-nsp] VRRP / MAC Forwarding Problem on Sup2/PFC2

[Geoffrey Pendery]

1.  You mention 12.2(18)SXF15 - I assume you're running native?  With
"ip cef"?  The "memorize the MAC address" you mentioned sounds like
the old style MLS on hybrid...

2.  I've seen the "traceroute doesn't match ip route path" behavior
before, with a CEF bug.  The CEF table had been holding onto an old
default route even though the IP routing table had a new one.  You
might be able to confirm this behavior with a "show ip cef" on 6509#1
(looking to confirm that the next hop leaves via the interface to BB#1
instead of 6509#2), and/or a "clear ip route *" on 6509#1, which
should clear the CEF FIB as well and force it to rebuild the table.

3.  A sniffer on 6509#1 might shed more light on the problem, as
you'll be able to really see the destination MAC addresses and the
ingress/egress ports.  The only other idea that's coming to me which
fits into semi-normal routing behavior is - maybe 6509#2 wants to
route the packet out BB#1 anyway, and is sending an ICMP redirect to
your server after the first packet, telling him to use 6509#1 instead.
 Another idea for checking this - clear the ARPs on your server, then
run some of this traffic, then show the ARPs.  See if he has an ARP
for the backup VRRP address, not just his default gateway.


Hope that helps.  Even if I'm wrong, I'd be curious to know how it turns out.


-Geoff


On Tue, May 19, 2009 at 3:11 AM, Sascha E. Pollok <nsp-list at pollok.net> wrote:
> Hello people,
>
> recently I have discussed a problem here and there and there
> is not proper solution/explanation yet so I thought I'd share
> it with you:
>
>                     Server
>                       |
>                       |
>               +-----3548XL-----+
>    .1q Trunk  |                | .1q Trunk
>               |      MST       |
>               |                |
>   MST Root  6509#1----------6509#2  MST Backup
>  VRRP Backup   |    L2 Trunk    |    VRRP Master
>   on SVI      |       .1q      |      on SVI
>               |                |
>             BB#1              BB#2   Backbone Routers
>
>
> Problem: 6509#1 was VRRP Master before and terminated all the
> IP traffic from the server on its SVI and routed it towards the
> internet (via BB#1).
>
> Now 6509#2 is configured VRRP master. Thus, because of MST,
> the ethernet frames should travel on layer 2 to 6509#2 and
> fall out of the SVI there like this:
>
>  Server -> 3548XL -> 6509#1 -> 6509#2 -> SVI -> BB#2.
>
> What actually happens is the strange part: the traffic
> can still be seen on the SVI on 6509#1. 6509#1 is still
> pushing the traffic from layer 2 to layer 3 on its local SVI
> and the 6509#2 never sees the traffic coming from the server.
>
> Even if I deconfigure VRRP on 6509#1. And: even if I do
> "no ip address" on the SVI on 6509#1 it is still routing
> the traffic (of course only incoming from the server as
> the connected routes are gone when doing "no ip address").
>
> The 6509#2 only sees the traffic in two cases:
>
>  1. I shutdown the SVI on 6509#1. In this case the traffic
>     is immediately switched to 6509#2 and routed to the 6509#2's
>     SVI. When I "no shut" the SVI on 6509#1 we are back to the
>     previous situation. Deleting and re-creating the SVI on
>     6509#1 does not help.
>
>  2. When I change the VRRP Group on 6509#2 to a VRRP Group that
>     NEVER was configured on the SVI on 6509#1. Then the traffic
>     goes as expected via the SVI on 6509#2.
>     When I make 6509#1 the VRRP master in this situation, the
>     L3 Traffic switches to 6509#1. When I make the 6509#2
>     the master again, it does NOT switch back to 6509#2.
>
> This leads me to the following assumption: the 6509#1 somehow
> memorizes the destination MAC addresses that it is supposed to
> "route" to the locally configured SVI even when the ethernet
> frame's destination address is reachable via a remote trunk.
>
> "sh mac-address-table vlan xx" on 6509#1 shows that it has been
> dynamically learned via the trunk towards 6509#2. This looks
> perfect. So there must be something else to let the 6509#1
> send the packet to the SVI.
>
> To make it more complex, I would like to add that a traceroute
> from the server to the Internet looks like this:
>
>   1  6509#2    (yes, thats correct)
>   2  BB#1      (oh that can't be since there is
>                 no link to BB#1 on 6509#2)
> Details about the platform: 6509 with SUP2, MSFC2
> and SFM cards running 12.2(18)SXF15.
>
> Anyone?
>
> Thank you for your thoughts.
> Sascha
> _______________________________________________
> cisco-nsp mailing list  cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>