[c-nsp] VIP/PA question

[Pete Templin]

I've got a 7507/RSP4 running 12.0(27)S1 with four VIP2-50s in it.  Three 
VIPs have a PA-FE-TX and a PA-MC-2T3 in them, one has only a PA-FE-TX. 
We've been running into some problems lately where one of the PA-MC-2T3 
cards begins showing loads of drops, and packets can't seem to enter 
this router and make it out the T1s (but locally-generated pings can). 
VIP CPU on some subset of the VIPs goes to 100%, and it seems the only 
fix is to remove and reseat the VIP.  We've swapped the VIP with no 
success.  Cisco TAC hasn't come up with a solution yet, and I'm starting 
to lose my hair (OK, that was already happening naturally).

Any thoughts on what to check, what to watch for, etc.?  This PA-MC-2T3 
handles roughly 60-80 fractional T1s on one T3 port, and 28 full-rate 
T1s on the other T3 port.  We have a similar layout on each PA-MC-2T3, 
but the second card only has about 25-35 fractional T1s (about 13 of the 
28 T1s have channel-groups on them) but a full load of 28 full-rate T1s 
on the other T3 port.  The third PA-MC-2T3 has about 20-30 fractional 
T1s but no full-rate T1s (yet).  This box moves a total of about 24-28 
Mbps, so I'd have a hard time imagining that any portion of it is 
overloaded.  We've got a few "mistakes" on it that we're going to 
rectify, such as two of the PA-FE are using subinterfaces, which isn't 
CEF switched in this version.  There are five VRFs along with the global 
table.  Each VRF uses only connected and static routes, and never 
actually becomes MPLS encapsulated (2-10 T1s and/or frac T1s and one 
subinterface; a default route points to a node in the subnet of the 
subinterface).

Any pointers?