[c-nsp] Two DS3s Bounced at the Same Time

[John Bittenbender]

On Sat, 19 Mar 2005 17:06:51 -0700, John Neiberger <jneiberger at gmail.com> wrote:
> We have an OC-12 coming into our main facility, and are using four
> DS3s on that OC12 at the moment. Two of the DS3s are channelized and
> mostly carry voice circuits. Two of the DS3s are not channelized and
> carry data. The two data DS3s terminate on two PA-T3+ port adapters
> that reside on the same VIP2-50.
> 
> On Friday, in the middle of the day, both DS3s went down at precisely
> the same second, but they came back up within a minute. This still
> caused some rather nasty network issues because of the visibility of
> the traffic on those links. The problem is that we can't find a reason
> for both of them to go down.
> 
> This could have been a problem on the VIP or the port adapters but
> there were no error messages in the logs other than the link down
> messages. The OC12 did not bounce because our two channelized DS3s
> stayed up. Qwest has verified that they saw our two data DS3s go down
> and they thought they some some errors in one of the relevant central
> offices. However, after intrusive testing last night they were not
> able to find a problem.

*snip*

Well, seeing as you have an OC12 into your facility and you had 2 T3's
drop and not two others, you could perhaps trace back your cables and
see if the two that dropped share a card on the MUX and the two that
didn't share a different card. If they do share a card, perhaps the
MUX card had an issue.

As a side note, did you check you router logs to see if you got RDI or
RAI on your controller? If you did your end was up and something down
the line was down.

John B