[c-nsp] MPLS MTU / Jumbo frames etc.

[Brandon Applegate]

On Wed, 22 Jul 2009, Brandon Applegate wrote:

> I know this has been covered, at least in part on this list before, and I 
> have read those posts.  However, I'm still trying to wrap my head around what 
> is happening internally (or rather on the wire) in the various scenarios.
>
> Scenario #3
> ===========
> 10 gig interface (ES20 CXL based) - mtu changed to 9216
> MPLS turned on - mpls mtu == interface mtu by default (does not show up in 
> config)
>
> One label packet, with 9216 size (linux ping -M do -s 9188) goes through, 
> with no icmp frag needed.
>
> So I'm confused on what's happening in one scenario vs. another.  It seems 
> that in scenario 1, the 'outer' MTU is 'signalling' down and kicking off a 
> icmp frag needed.
>
> Scenario 2 goes through because we are telling the router it's allowed to 
> send a 'baby-giant' (i hate that term).
>
> Scenario 3 really gets me though.  Why doesnt it complain and tell me icmp 
> frag to 9212 or something ?  Isnt the frame 9220 when it's all said and done 
> ?  Is the router fragmenting this in software at the 'mpls level' and just 
> not telling me ?  Should I set mtu down to 9212 or something to make sure 
> that the router NEVER frags frames ?
>
> I guess a fireaxe solution would be for us to simply define 'jumbo frames' in 
> our network as 9000 bytes, period.  But I'd like to actually understand why 
> this behaviour seems to change as I slide the MTU around.  I want to make 
> sure that our $$$ isnt being wasted by killing the CPU with fragmentation (if 
> thats whats happening, again scenario 3 is really puzzling me).
>

I think I figured (part of) this out.  Packets to the router != packets 
through the router.  Trying to ping something on the far side with packet 
size of 9188/9216 gets me the expected icmp frag @ 9212.  I still think 
I'm going to proclaim that jumbo == 9000 to make it easier for server / 
storage guys to remember anyway :)

--
Brandon Applegate - CCIE 10273
PGP Key fingerprint:
7407 DC86 AA7B A57F 62D1 A715 3C63 66A1 181E 6996
"SH1-0151.  This is the serial number, of our orbital gun."