[c-nsp] 7200 Line protocol bouncing

[FF]

Michael,

I have seen this sort of problem, and if its the exact same one, you
have a bear of a time ahead of you. I'm writing this because I've been
waiting more than a dozen years to share the story. :) The reason you
are having problems tracking this down is because you are (believe it
or not) having an analog problem, not a digital one. You'll see what I
mean.

The window you are doing the extended test for (1+hr) is not enough,
or sufficient. You have to do the test during a window when you'd
normally catch one of these outages. THEN it will show up in a BER
test. I totally understand why you won't want to do a 24+ hour test,
but eventually you'll end up having to do that if you can't find
another cause.

I'd stop trying to screw with your known working configs.

What the root cause of my problem so many years ago on a pair of T3s
going across IXCs was a faulty mux. Moreover, it was a single card in
a mux. It was a TRANSMIT (as in a single, unidirectional problem,
errors show up in one direction but not the other) card. It would
freak out every few hours, for a few seconds, and would otherwise show
up (and test) totally normal. Eventually, we isolated the problem to a
single mux and the technician in the CO *heard* the transmit card
buzzing in its slot while at the same time seeing bit errors. Not
buzzing, no errors. It was an OC48 card, but the point is the same.

So this is what we did.

We confirmed that the problem shows up (eventually) during a long,
long BER test.

Then we split the span in half and ran each half separately.
(Successively to keep cutting the span lengths in half). If you can
get two pairs of test equipment in mid span, this will go faster.
Keep going, until you can nail down which mux it is limited to. (our
span had 9 muxes in it for two identical T3 paths).

Since you have a combination of T1 and T3 circuits, its likely they
diverge, so this testing should really occur over the common elements
(especially if they happen at the same time) so that may limit/reduce
the amount of testing you have to eliminate elements.


Hopefully this helps, but your problem sounds almost exactly like what
we were seeing.

Best,

FF