[c-nsp] BGP neighbor not establishing session

[Abello, Vinny]

On Nov 24, 2010, at 9:32 AM, Lobo wrote:

> 
> 
> On 11/23/2010 5:13 PM, Eric Oosting wrote:
>> 
>> On Tue, Nov 23, 2010 at 3:00 PM, Lobo <lobotiger at gmail.com 
>> <mailto:lobotiger at gmail.com>> wrote:
>> 
>> 
>> 
>>    On 11/23/2010 6:14 AM, Mark Tinka wrote:
>> 
>>        Can't think of why this would be an issue.
>>        The longest distance we have today between two iBGP
>>        neighbors is 160ms (and soon, the farthest we'll have will
>>        be about 230ms), and that has no problems at all.
>> 
>>        I'd suspect MTU issues here.
>> 
>>        Cheers,
>> 
>>        Mark.
>> 
>> 
>>    It's looking like it somehow might be related to that.  For a
>>    test, we set the mtu on the toro-router2's interfaces back to 1500
>>    from 9216.  After that was done, the sessions to the vanc routers
>>    both came up!  Strange because we didn't change any of the mtus on
>>    the vanc routers.....they're all still 9216.
>> 
>>    One thing I didn't mention before was that all of the interfaces
>>    on these routers had jumbo frames turned on.  The cloud between
>>    toro and vanc though is mostly limited to 1546 but this has never
>>    posed a problem before with the previous IOS versions.
>> 
>> 
>> The default MSS in older IOS was something in the 500s, so even if you 
>> had a large MTU on the interface and the path would only accommodate 
>> 1500ish you'd still be OK. It could be that your newer IOS either went 
>> to a larger default MSS that could take advantage of the 9216 MTU and 
>> there was no PMTUD, or PMTUD is busted in between so the MTU squeeze 
>> between the two routers isn't detected.
>> 
>> -e
>> 
>> 
>>    Jose
>> 
>>    P.S. I've opened up a TAC case as well to see if they can figure
>>    something out.
>> 
>>    _______________________________________________
>>    cisco-nsp mailing list cisco-nsp at puck.nether.net
>>    <mailto:cisco-nsp at puck.nether.net>
>>    https://puck.nether.net/mailman/listinfo/cisco-nsp
>>    archive at http://puck.nether.net/pipermail/cisco-nsp/
>> 
>> 
> 
> And we have a winner!  I disabled path mtu discovery for both of the 
> remote neighbors and after clearing the sessions they both came up and 
> have been stable since last night.
> 
> You're right that it appears that path mtu discovery must be broken or 
> something in this version of IOS because when I look at the MSS for the 
> neighbors they still show as 9176 (9216 - 40) where it should show 
> something closer to the 1506 mark.
> 
> Thanks for the tips everyone!  I'll let our TAC engineer know about 
> these findings as well.
> 
> Jose

Be sure you do not have "no ip unreachables" on any interfaces in the path or it will break path-mtu-discovery. Also check to be sure you aren't using an mls rate-limiter to completely discard icmp unreachables instead of rate-limiting them, ie setting the rate to 0.

-Vinny