[c-nsp] some weird routing problems after GSR upgrade

[Rodney Dunn]

On Thu, Aug 21, 2008 at 12:49:57PM +0200, Primoz Jeroncic wrote:
> Hi
> 
> After upgrade of one of our c12008 from GRP-B to PRP-1, and with
> this required IOS upgrade (from service provider 12.0.30.S5 to
> 12.0.33.S1) I'm starting to get some really weird issues with
> routing.
> At the moment my network topology looks like this:
> 
> Upstream-1 -- GSR-1 -- GSR-2 -- GSR-3 -- c7401 -- c7206 -- backup upstream
> 
> Now show explanation. Route engine line card and IOS were upgraded
> on GSR-1.
> One prefix is originated and advertised in BGP from GSR-2.
> Second prefix is originated and advertised from c7206.
> In case if matters, MPLS is enabled from GSR-1 to c7401. C7206 is
> not in MPLS enabled network.
> Both prefixes are announced through same AS, and if everything is fine,
> they both go out through GSR-1 and Upstream-1.
> 
> And now my problem. Config is same as it was before. Before everything
> was working fine, and when link between c7401 and c7206 went down,
> prefix originated on c7206 went out through backup upstream and came
> into my network (to GSR-2) through upstream-1 and GSR-1.

That doesn't make sense because if you break the link between
7401 and 7206 then GSR1 doesn't have reachability to the 7206 to get
the route to send it out since the 7206 originated it.

> After upgrade, things doesn't work like this anymore. Until I clear
> route for prefix originated on c7206 on GSR-1 (clear ip route x.x.x.0),
> trafic goes from GSR-2 to GSR-3, then to c7401, then back to GSR-3 and
> GSR-2, and then another loop.

We'd need to see 'sh ip route' to figure out what exact prefix is being
used in the routing table at the time along with 'sh ip cef' for the
ip address you see a looping traceroute for.

> 
> After link goes down, there's no routes for this prefix in GSR-3 and
> c7401, so traffic should go over default route. There is route for
> this prefix on GSR-2, but it's pointing to GSR-1, and on GSR-1 there's
> route for this prefix through Upstream-1. So based on routing tables
> everything looks fine. But traffic still bounces as described above.

You have to verify with 'sh ip cef' since it will print the recursion
for you to the next hops.


> 
> I know easiest solution would be to downgrade GSR-1 to previous IOS
> which was working. But with older IOS, one of Gigabitethernet linecards
> in GSR-1 was crashing constantly every 15 to 20mins. Considering there
> is about 200 BGP peerings on that GE, you can imagine that peerings
> were down pretty much all the time :) With 12.0.33.S1, this crashes
> are history, but this routing problem is something what I didn't have
> before.
> 
> If anyone has problems like this, or if anyone has any idea how to solve
> this, please let me know. I would be extremely greatful for any hint
> on this.
> 
> Thanks in advance, and if I wasn't clear enough, please drop me note, and
> I will try to explain again.
> 
> Have fun,
> Primoz Jeroncic
> Support - IP Connectivity & Routing
> -------------------------------------------------------------------
> Softnet d.o.o.  tel:  +386 1 562 31 40   |
> Borovec 2       fax:  +386 1 562 18 55   |       1 + 1 = 3
> 1236 Trzin      primoz(at)softnet.si     | for larger values of 1
> Slovenija       http://flea.softnet.si/
> -------------------------------------------------------------------
> 
> _______________________________________________
> cisco-nsp mailing list  cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/