[j-nsp] MX960 JunOS recommendations

Wed Nov 11 14:36:43 EST 2009

With MTUs around 9000 configured on ALL links in the network there should be no problem with BGP,
since as per RFC4271, section 4:

The maximum message size is 4096 octets.  All implementations are required to support this maximum
message size.

So even if MPLS and IP MTUs slightly differ, with sizes around 9000 it doesn't matter from BGP
perspective.

The only thing that comes in my mind, that there are some L2 switches in between and there is
something wrong with MTU on those switches. Worth to check.

Could you paste from the log the Notification message generated when the BGP session is tear down?

Thanks,
Krzysztof

-----Original Message-----
From: Tima Maryin [mailto:tima at transtelecom.net] 
Sent: Wednesday, 11 November, 2009 15:12
To: kszarkowicz at gmail.com
Cc: juniper-nsp at puck.nether.net
Subject: Re: [j-nsp] MX960 JunOS recommendations

Uhm, i see your point here.
We indeed have cisco - cisco - Jun - Jun setup

My cisco interface mtu = ip mtu = mpls mtu =9000
But i reeeealy doubt that bgp keepalive packet size can come close to that mtu.

On Juniper i set interface mtu = cisco mtu +14 and it works fine!
And! As you say, it reports different mpls mtu value:

 > show interfaces xe-1/0/0 | match MTU
   Link-level type: Ethernet, MTU: 9014, LAN-PHY mode, Speed: 10Gbps, Loopback: 
None, Source filtering: Disabled,
     Protocol inet, MTU: 9000
     Protocol mpls, MTU: 8988
     Protocol multiservice, MTU: Unlimited

As far as i understand "default mpls mtu" term (not sure that i _fully_ 
understand it though) it seems, Juniper supposes 3 labels maximum.
I dont see any reasons for device to drop packets which has 1 or 2 labels and 
bigger than mpls mtu, but still ok from interface mtu point ov view.

As per your logic, device should drop all traffic that match such criteria but 
it seems only bgp session keepalives and i didn't see any other problems

But still, i made an experiment on Juniper and cisco which has bgp session 
between them.

cisco:
#sh mpls interfaces g 0/0 detail  | i MTU
         MTU = 9000
#sh ip int g 0/0 | i MTU
   MTU is 9000 bytes
#sh run int g 0/0
Building configuration...

Current configuration : 212 bytes
!
interface GigabitEthernet0/0
  description --- to 7606-2 ---
  mtu 9000
  ip address 10.3.13.2 255.255.255.0
  load-interval 30
  duplex full
  speed 1000
  media-type gbic
  no negotiation auto
  tag-switching ip
end

If i set mtu 9000 under family mpls and commit it, it looks like this:

 > show interfaces xe-1/0/0 | match MTU
   Link-level type: Ethernet, MTU: 9014, LAN-PHY mode, Speed: 10Gbps, Loopback: 
None, Source filtering: Disabled,
     Protocol inet, MTU: 9000
     Protocol mpls, MTU: 9000
       Flags: Is-Primary, User-MTU
     Protocol multiservice, MTU: Unlimited

and problem still persists

please let me know if you have any other ideas :)

p.s. Its the same effect if i set tag-sw mtu 8988 on cisco and leave it 
'default' (=8988) on juniper

Krzysztof Szarkowicz wrote:
> Let me guess.
> 
> Your network is multivendor network (JNPR and CSCO) and some transit devices are CSCO?
> 
> CSCO and JNPR uses different algorithm to calculate default MPLS MTU (if MPLS MTU is not
explicitely
> configured) which results in 4 byte difference between CSCO side and JNPR side of the same link
for
> MPLS MTU (the IP MTU is equal on both ends, so no problem with OSPF).
> 
> If on JNPR side your MPLS MTU is say 1500 and on the CSCO side the MPLS MTU is 1504, when the CSCO
> device send an BGP update packet towards JNPR device with size 1502, this packet is dropped by
JNPR
> device (as it is to big), and TCP ACK is not sent back. CSCO is keeping by resending this 1502
long
> packet, and JNPR is constantly dropping. Thus, after hold timer expires, the Notification message
is
> sent.
> 
> I assume that with 9.3.R3.8 you didn't catched the '1502' packet sizes.
> 
> Could you check with some show commands, what is the MPLS MTU on both ends of the link (which is
> terminated on CSCO on one side and JNPR on other side)?
> 
> //Krzysztof
> 
> -----Original Message-----
> From: Tima Maryin [mailto:tima at transtelecom.net] 
> Sent: Wednesday, 11 November, 2009 9:57
> To: kszarkowicz at gmail.com
> Cc: juniper-nsp at puck.nether.net
> Subject: Re: [j-nsp] MX960 JunOS recommendations
> 
> What did you mean by "inappropriately configured" ?
> 
> There are the same mtu settings everywhere and traffic passes quite well.
> And ospf session goes up without problems.
> 
> And how comes that "inappropriately configured IP and MPLS MTU" work well on 
> 9.3R3.8 ?
> 
> 
> Krzysztof Szarkowicz wrote:
>> It is not a nasty bug, but problem of inappropriately configured IP and MPLS MTUs on transit
> nodes.
>> //Krzysztof
>>
>> -----Original Message-----
>> From: juniper-nsp-bounces at puck.nether.net [mailto:juniper-nsp-bounces at puck.nether.net] On Behalf
> Of
>> Tima Maryin
>> Sent: Wednesday, 11 November, 2009 8:28
>> To: juniper-nsp at puck.nether.net
>> Subject: Re: [j-nsp] MX960 JunOS recommendations
>>
>> 9.3R4.4 has a nasty bug which occures in setup when you have bgp session over 
>> chain of few routers/links with ospf/ldp
>>
>> bgp session occasionally goes down with notification timeout. Even when there is 
>> no traffic at all and no physical errors
>>
>> rollback to 9.3r3 helps though
>>
>>
>> JTAC still not confirmed it, but it easlily can be reprodused in lab