[j-nsp] Debugging SONET (OC-3) errors

[Kevin Oberman]

> Date: Mon, 4 Aug 2008 13:54:27 -0500
> From: Chris Adams <cmadams at hiwaay.net>
> Sender: juniper-nsp-bounces at puck.nether.net
> 
> I have an OC-3 (POS running PPP) between two POPs (an M10i at each end)
> getting errors, and this is the first time I've had to debug errors on a
> SONET link.
> 
> I am seeing BIP-B3 and REI-P incrementing in spurts on the Z end (e.g.
> 99 BIP-B3 and 45 REI-P, both showing 2 seconds), as well as ES-P and
> ES-PFE matching the seconds (2 in since I cleared the counters).  No
> other error counters are incrementing (no section or line errors at
> all).  I have seen some errors at the A end as well, but not as many or
> as often.
> 
> When I get a spurt of errors, both ends will show an interface flap (at
> least most of the time; I'm not sure this is 100% of the time or not).
> 
> Reading various docs, this means there is a problem somewhere between
> telco devices with the transmission from the Z end to the A end.  Is
> that correct?
> 
> I opened a telco ticket, but they pretty much blew it off.  I know
> little about SONET so I am having trouble telling them what to check.
> I'd like to avoid intrusive testing if possible (since this is a major
> circuit for us).
> 
> Part of the problem is that this circuit goes through 7 (yes 7!)
> different carriers, and I am not sure that the telco that owns the whole
> thing is looking at anything other than the loop on the Z end (the only
> piece they actually own).
> 
> Any suggestions?

I don't envy tracking something like this through 7 carriers!

In any case, BIP-B3 errors are data errors and REI errors are "Remote
Error Indicators". Since you see PATH errors on both ends of the
circuit, it is somewhere in the middle. The errors are in bursts which
would indicate electrical noise at some OEO location (probably an ADM,
though both amplifiers and regens can do this) or a clocking problem.

Since REIs are reaching your equipment, some piece of gear at one of the
carriers is reporting errors. It's just tracking it down that can be
daunting. In may experience, the standard answer the first time you ask
a carrier about line errors is that they have seen nothing. The error is
probably at a hand-off from one carrier to another. Try to confirm which
carrier(s) are seeing any non-path errors. This will at least help
localize the issue.

Good luck!
-- 
R. Kevin Oberman, Network Engineer
Energy Sciences Network (ESnet)
Ernest O. Lawrence Berkeley National Laboratory (Berkeley Lab)
E-mail: oberman at es.net			Phone: +1 510 486-8634
Key fingerprint:059B 2DDF 031C 9BA3 14A4  EADA 927D EBB3 987B 3751
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 224 bytes
Desc: not available
URL: <https://puck.nether.net/pipermail/juniper-nsp/attachments/20080805/ec111cf7/attachment.bin>