[c-nsp] VIP/PA question
oid at saudi.net.sa
Wed Oct 6 02:07:13 EDT 2004
No thought or pointers here. I just wanted to say that we went through a
similar problem. The VIP CPU utils would hit the roof and it would start
The throughput of the router was less than 200Mbps total, it had 4 VIPs,
a a combination of VIP2s and VIP4s, ATM, FX and Gigabit PAs.
It started when we upgraded from 12.1(22)E1 to 12.3-5b, to fix some
counter problems. Although in our test environment nothing went wrong,
one of the VIPs (VIP4-50) starting crashing. So we decided to go back to
12.1(22)E1 until we could figure out why. About 5 days later two VIP CPU
utilizations would reach 98-100% and start dropping packets, this
happened about 4 times within one day, lasting about 5 minutes each
time, then it would go back to forwarding packets. We played around with
dCEF and CEF, removing it from some of the cards, placing the load on
the RSP. This seemed to calm it down a bit, but on occasion some VIPs
would still hit 100%.
We replaced several VIPs think that it might be a hardware failure, we
even replaced the entire chassis and all the interfaces/VIPs, without
any success (using the cold standby.)
Then we decided to reduce the load on the device and we split the
traffic between two 7500s.
Since then we had not had any such incidents, but we still don't know
what went wrong.
Here are the current utils:
VIP2-40 CPU utilization for five seconds: 55%/54%; one minute: 54%; five
VIP2-50 CPU utilization for five seconds: 2%/2%; one minute: 2%; five
VIP4-80 CPU utilization for five seconds: 41%/37%; one minute: 41%; five
RSP4 CPU utilization for five seconds: 30%/28%; one minute: 28%; five
VIP4-50 CPU utilization for five seconds: 27%/27%; one minute: 27%; five
VIP2-50 CPU utilization for five seconds: 25%/25%; one minute: 24%; five
VIP4-80 CPU utilization for five seconds: 0%/0%; one minute: 0%; five
RSP4 CPU utilization for five seconds: 2%/0%; one minute: 2%; five
We went through an extremely frustrating time. And were very clueless as
to what was happening. Nothing similar existed on mailing list or the
web in general. We didn't have TAC support, and were accustomed to
solving it our selves.
I personally detest this box. And very dissatisfied with the quality of
Sorry I couldn't be of any help. And I feel your pain :-)
-Osama I. Al-Dosary
Pete Templin wrote:
> Oleksandr Pantus wrote:
>> Take a look at:
> Already familiar with this, and I don't think it applies here. The
> outgoing interfaces are not congested (outbound rates of 40-100kbps on
> a T1), yet the router appears to be dropping most or all traffic
> headed out those interfaces once the "magic trigger" is pulled.
> I'm starting to suspect the PA-MC-2T3. We had an episode where EVERY
> T1/frac T1 dropped line protocol (PPP or HDLC) simultaneously. A week
> later, we had the scenario mentioned above, and it's recurred twice
> since then. It's always been the same slot/PA that's been affected,
> and the companion PA-FE-TX doesn't appear to be afflicted at all.
> Any other thoughts?
> Thanks for the help!
> cisco-nsp mailing list cisco-nsp at puck.nether.net
> archive at http://puck.nether.net/pipermail/cisco-nsp/
More information about the cisco-nsp