[cisco-voip] Clicking/popping noise on inbound side of PRI

[Carter, Bill]

Avaya!!!

-----Original Message-----
From: Mike Armstrong [mailto:mfa at lal.ufl.edu] 
Sent: Friday, September 19, 2003 3:56 PM
To: cisco-voip at puck.nether.net
Subject: [cisco-voip] Clicking/popping noise on inbound side of PRI


Warning: I've gone through 4 Cisco engineers at TAC, 3 Verizon
technicians, 2 VWICs, and 2 SmartJacks, with no resolution to this
problem yet, so this may not be easy.  I'm looking for out-of-the-box
thinking, unencumbered by reason and conventional troubleshooting
procedures.  Have a few beers before working on this.

I've got a 3640/NM-HDV/VWIC-1MFT-T1 connected to an Adtran HTU-R
(Verizon calls it a "SmrtJack") which is the T1 (DS1) interface to an
HDSL circuit between us and the Verizon CO.  There is an intermittent
loud clicking/popping noise -- like a Geiger counter -- on the inbound
side only. It was first reported in early August around the time we
upgraded the router
to IOS version 12.2(15)T.   Reverting to the previous version -- 
12.2(13)T -- did not help.

Cisco has replaced the VWIC, even though they didn't think it was the
problem.  Verizon replaced the HTU-R even though they didn't think it
was the problem.  Both Cisco and Verizon claimed it was my unshielded
patch cable between the VWIC and the HTU-R, so I replaced it with a
certified, shielded cable (10 feet long).  None of these replacements
had any effect. In all this time, looking at the diagnostic data
recorded by the HTU-R, I have seen only a few errors of any kind
associated with the Telco circuit, and most of them were apparently
caused by us screwing around with the circuit.  During the same time, I
have recorded 6.7M Line Code Violations and 800K Path Code Violations
(hereafter referred to as "errors") on the VWIC.

We all believe that layer 1 timing is the issue.  I monitored an open
line for a couple of hours and confirmed that the noise was correlated
with T1 controller errors.  I've got "sho controller t1 1/0" data going
back to August 8th, with only a few gaps, in a database keyed to the
15-minute-collection-interval time, and from all this data can tell you:

o We see error rates from 0 to 109,333 per 15-minute interval. o We can
go for hours with 0 errors, then jump to tens of thousands in the next
15-minute interval. o Once they start, the errors tend to continue for a
while, but it's not guaranteed. o There are more errors Saturday,
Sunday, and Monday than Tue-Fri. o There is a broad peak from 2PM-10PM.
o There is no evidence of electrical noise (all our equipment is plugged
into UPS, Telco equipment is line powered). o There is no evidence of
EMF noise in the area, measured crudely with a broadband AM receiver. o
I can't correlate error-free or error-intensive periods with much of
anything.

We had the Telco run loopbacks and BERT tests.  Cisco has done the same.
They did it together for an hour and a half last week.  I've done the
same. None of us has yet been able to produce similar errors in any test
environment.

Relevant router configs are:

controller T1 1/0
 framing esf
 linecode b8zs
 cablelength short 133
 pri-group timeslots 1-20,24
 no yellow generation
 no yellow detection

...

interface Serial1/0:23
 no ip address
 no logging event link-status
 isdn switch-type primary-ni
 isdn incoming-voice voice
 (encapsulation HDLC by default)

Any AHA!s out there?

Mike Armstrong
University of Florida CREC
Lake Alfred, FL
863.956.5891

_______________________________________________
cisco-voip mailing list
cisco-voip at puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-voip