[c-nsp] converting cmts from pure ip routing to mpls pe - uBR7246VXR

[Aaron]

I have (5) cmts's (uBR7246VXR) ..4 operational and 1 in lab for testing.

 

We have a new mpls network comprised of asr901's, me3600's and asr9k's
functioning as p's and pe's.

 

I wanted to move my cmts's off my traditional routed/switched network to my
new mpls network.  I wanted to have cmts's function as pe's so as to
potentially take advantage of the mpls LxVPN's

 

I successfully converted one of my cmts's to pe and it's running nicely,
uplinked into p box (me3600).  What I did was basically convert wan uplink
to mpls, remove igp and replace with core mpls network igp process, and then
bring up the expected mp-ibgp and vrf stuff, and then convert all those
traditional routing interfaces and services (ntp, logging, aaa and tacacs)
to be vrf based..works.

 

Now for the second cmts that I wanted to convert to pe, I've tried twice now
and have seen similar strange behavior. wan uplink utilization drops to
about 50% of what was previously seen before change..cpu utilization drops
from 30-40% utilization to about 0-10%....given those observations on the
first attempt last week, I left it that way, thinking not too much of it as
it was 2:30 a.m. in the morning and was thinking that low utilization at
that hour is conceivable. later I got woken up with a phone call from one of
my front-line noc network analysts at 7:15 a.m. saying that we had several
subs calling in saying that they could not get to most internet web pages
but only some were reachable.. (I think the web pages they could get to were
our local company web site hosted on-net, and some of our local Akamai and
other cached pages)..strangely I could ping and trace to and from those
subnets on that cmts to and from internet route server (looking glass) test
locations.. I didn't know what to make of this..i couldn't find a problem,
so was forced to hurry up and throw the cmts back to old switched/routed
network.

 

..i tried again a few nights ago and saw similar drop in wan utilization and
cpu load..not knowing what to make of it, and concerned that subs would be
unable to get to web sites that following morning, I moved it back.  I don't
have a test modem off of this cmts to test with but will need to get one out
there if I try again.

 

.I have a tac case open, and I am going to try to reproduce this in the test
cmts. (but all previous tests on the lab cmts show good results.and as I
mentioned, the other cmts is running fine in mpls net)

 

Difference between the one that worked and the one that doesn't is one is
uplinked into me3600 (working one) and the one that didn't work is uplinked
into asr9k

 

Interestingly, the module in the asr9k that I uplink that second cmts into,
crashed a couple weeks ago..it took a double ecc error and ios xr showed a
forced reset on that module..strange.. tac ios xr team said that it's
probably an isolated (transient) error and shouldn't happen again, but if it
does, they will RMA that 2/20 module in that asr9k.   ..several connections
are still working on that asr9k linecard and so I didn't think that this
second cmts being mpls uplinked through there would be an issue..but I had
to mention it since I'm seeing weirdness..

 

Any thoughts or input would be appreciated.

 

Aaron