[c-nsp] converting cmts from pure ip routing to mpls pe - uBR7246VXR

Aaron aaron1 at gvtc.com
Tue Apr 30 09:17:54 EDT 2013

Mike/JF/et al, 

Thanks so much for the feedback regarding this....as a few of y'all
mentioned, it seemed that it was an MTU issue.

I was able to see a problem in the lab.  using (2) pc's , one windows XP and
one centos Linux, I was able to see that the Linux machine would not be able
to surf the internet after moving the lab cmts from my legacy 10 gig
switched network to me new mpls asr9k 10 gig network.....

it seems that default MTU 1514 (9k) was the problem, during the browsing
problem from the Linux machine I was running wireshark sniffer and seeing a
lot of icmp type 3 code 4 destination unreachable/fragmentation needed.  

I changed it from 1514 on asr9k to 1518 and then the Linux web browsing
problem goes away and I see no more icmp fragmentation needed messages.
(well, actually I had to tell ospf to ignore mtu since I only change
physical interface mtu on one side)

I then changed it to our more standard jumbo frame setting in our network to
9216 and is still good. (I then did this on both ends of link cmts (vanilla
ios 9202 and asr9k ios xr 9216 and then removed ospf ignore mtu)

I then proceeded to throw operational cmts during maintenance window and it
went great!  We've been running good for 5 days now.

A question is why didn't this present a problem with a cmts connected to an
me3600x ?  I didn't have to do any mtu changes on that one and it worked
fine.  I left me3600 interface at 1500 and I've heard of no customer
complaints on that cmts


-----Original Message-----
From: Jean-Francois.Dube at videotron.com
[mailto:Jean-Francois.Dube at videotron.com] 
Sent: Monday, April 01, 2013 5:20 PM
To: aaron1 at gvtc.com
Cc: cisco-nsp at puck.nether.net
Subject: Re: [c-nsp] converting cmts from pure ip routing to mpls pe -

Hi Aaron,

If you were already using MTU above 1508 for your CMTS to ME3600 links than
you would not need to change anything.

The issue with CMTS to ASR9K only exist if you have configured the very same
MTU on both sides.

You need to check that your "IOS-XR" MTU is equal to your "IOS" MTU + 14

(You need two 4-bytes labels for MPLS VPN so if you are using Ethernet your
"IOS" MTU should be 1508 at least)



De :	aaron1 at gvtc.com
A :	Jean-Francois Dube <Jean-Francois.Dube at videotron.com>,
Cc :	cisco-nsp at puck.nether.net
Date :	2013-03-29 15:21
Objet :	Re: [c-nsp] converting cmts from pure ip routing to mpls pe -

Thanks JF,  is there a reason why this would be required for CMTS to asr9k,
but not required for CMTS to me3600x ?

My CMTS PE to me3600 p is running fine, I didn't make any Mtu changes there.

----- Original Message -----
From: Jean-Francois Dube <Jean-Francois.Dube at videotron.com>
To: cisco-nsp at puck.nether.net
Sent: Fri, 29 Mar 2013 09:18:33 -0400 (EDT)
Subject: Re: [c-nsp] converting cmts from pure ip routing to mpls pe -
uBR7246VXR Hi Aaron, It sounds like you may be having MTU issue.
At least that is my experience when you can ping and only browse "some"
Your CMTS is running IOS and your ASR9K is running IOS-XR.
In IOS-XR you need to account for the L2 header of 14 bytes so the "default"
MTU is 1514.
If you are running MPLS you'll need to increase the MTU even higher to
account for the extra headers/labels.
That means your CMTS interfaces should be using something like 1516 and your
ASR9K would be 1530.
Jean-François Dubé
Technicien, Opérations Réseau IP
Ingénierie Exploitation des Réseaux
cisco-nsp-bounces at puck.nether.net a écrit sur 2013-03-28 15:24:42 :
> De : "Aaron" <aaron1 at gvtc.com>
> A : <cisco-nsp at puck.nether.net>,
> Date : 2013-03-28 15:31
> Objet : [c-nsp] converting cmts from pure ip routing to mpls pe -
> Envoyé par : cisco-nsp-bounces at puck.nether.net
> I have (5) cmts's (uBR7246VXR) ..4 operational and 1 in lab for testing.
> We have a new mpls network comprised of asr901's, me3600's and asr9k's 
> functioning as p's and pe's.
> I wanted to move my cmts's off my traditional routed/switched network 
> to
> new mpls network. I wanted to have cmts's function as pe's so as to 
> potentially take advantage of the mpls LxVPN's
> I successfully converted one of my cmts's to pe and it's running 
> nicely, uplinked into p box (me3600). What I did was basically convert 
> wan
> to mpls, remove igp and replace with core mpls network igp process, 
> and
> bring up the expected mp-ibgp and vrf stuff, and then convert all 
> those traditional routing interfaces and services (ntp, logging, aaa 
> and
> to be vrf based..works.
> Now for the second cmts that I wanted to convert to pe, I've tried 
> twice
> and have seen similar strange behavior. wan uplink utilization drops 
> to about 50% of what was previously seen before change..cpu 
> utilization
> from 30-40% utilization to about 0-10%....given those observations on 
> the first attempt last week, I left it that way, thinking not too much 
> of it
> it was 2:30 a.m. in the morning and was thinking that low utilization 
> at that hour is conceivable. later I got woken up with a phone call 
> from one
> my front-line noc network analysts at 7:15 a.m. saying that we had
> subs calling in saying that they could not get to most internet web 
> pages but only some were reachable.. (I think the web pages they could 
> get to
> our local company web site hosted on-net, and some of our local Akamai
> other cached pages)..strangely I could ping and trace to and from 
> those subnets on that cmts to and from internet route server (looking 
> glass)
> locations.. I didn't know what to make of this..i couldn't find a
> so was forced to hurry up and throw the cmts back to old 
> switched/routed network.
> ..i tried again a few nights ago and saw similar drop in wan 
> utilization
> cpu load..not knowing what to make of it, and concerned that subs 
> would
> unable to get to web sites that following morning, I moved it back. I
> have a test modem off of this cmts to test with but will need to get 
> one
> there if I try again.
> .I have a tac case open, and I am going to try to reproduce this in 
> the
> cmts. (but all previous tests on the lab cmts show good results.and as 
> I mentioned, the other cmts is running fine in mpls net)
> Difference between the one that worked and the one that doesn't is one 
> is uplinked into me3600 (working one) and the one that didn't work is
> into asr9k
> Interestingly, the module in the asr9k that I uplink that second cmts
> crashed a couple weeks ago..it took a double ecc error and ios xr 
> showed
> forced reset on that module..strange.. tac ios xr team said that it's 
> probably an isolated (transient) error and shouldn't happen again, but 
> if
> does, they will RMA that 2/20 module in that asr9k. ..several
> are still working on that asr9k linecard and so I didn't think that 
> this second cmts being mpls uplinked through there would be an 
> issue..but I
> to mention it since I'm seeing weirdness..
> Any thoughts or input would be appreciated.
> Aaron
> _______________________________________________
> cisco-nsp mailing list cisco-nsp at puck.nether.net 
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
cisco-nsp mailing list cisco-nsp at puck.nether.net
archive at http://puck.nether.net/pipermail/cisco-nsp/

More information about the cisco-nsp mailing list