[c-nsp] Very Weird Problem with BGP Peer Groups and iBGP

[John Neiberger]

I had something really weird just happen that I think might be a bug,
but it happened at two different sites configured exactly the same way.

Imagine Router A that has an eBGP session to an ISP, and it is
accepting a default route. Router A has four iBGP peers (connected via
frame relay point-to-point subinterfaces) configured in a peer group.
The peer group settings include their remote-as and "next-hop-self". If
you look at the BGP table on one of the iBGP peers you would see that
the next hop for all routes is Router A, specifically the serial
subinterface that connects the router to Router A.

At one point today, the default route was withdrawn temporarily by the
ISP and then readvertised in a couple of minutes. The weird thing is
that I lost IP connectivity from yet another remote site to a couple of
the iBGP peers of Router A. When I checked their BGP tables I saw that
several of the next hops were wrong. Instead of the IP address of the
subinterface that connects the peer to Router A, I was seeing the IP
address of some of the *other* subinterfaces!! Freaking weird. That
should never happen, should it? If next-hop-self is configured, the next
hop should always be the directly connected interface, right? 

This is very very odd. I hope I've explained it in a way that makes
sense. The story has been modified and abbreviated because I'm in a
hurry.

Any thoughts?

Thanks!
John
--