[cisco-voip] no audio for 10 seconds

Fri Nov 4 10:58:00 EDT 2011

No real updates on this issue yet.

We're trying some reorganization and 'restacking' of our resources to see if we can isolate the problem over the weekend.  The real brain scratcher is that CUCM reports packet loss while our switches (nasty 4500's which are almost 10 years old) report no loss.

-----Original Message-----
From: cisco-voip-bounces at puck.nether.net [mailto:cisco-voip-bounces at puck.nether.net] On Behalf Of Wellnitz, Erick A.
Sent: Thursday, November 03, 2011 9:04 AM
To: Wes Sisk
Cc: cisco-voip at puck.nether.net
Subject: Re: [cisco-voip] no audio for 10 seconds

Wes,

Based on your excellent advice we've discovered major pacet loss and hundreds of SIP timeouts.

It happens with phones on both local subscribers so we're thinking:

Bad port or cable somewhere
Inadequate QoS
Something strange with firmware is a distant third

On Nov 2, 2011, at 3:52 PM, "Wes Sisk" <wsisk at cisco.com<mailto:wsisk at cisco.com>> wrote:

That is certainly a candidate.

With 34 seconds duration you should have
34 seconds * 20 packets/second = 680 packets

With only 168 packets something is questionable.  That means you're missing 512 packets.  at 20 pps that's ~25 seconds where the call was connected but no packets were being received.

For "10 seconds of silence" you'd expect to be missing ~10 seconds * 20packets/second = ~200 packets.

/wes

On Nov 2, 2011, at 4:05 PM, Wellnitz, Erick A. wrote:

The following looks to be what I'd be looking for to indicate this 'delay' if I'm not mistaken.

origdevicename               duration               orignumberpacketsreceived       datetimeconnect             datetimedisconnect                 destnumberpacketssent              datetimeorigination
=============== ======== ========================= =============== ================== ===================== ===================
SEPB8621F6D6648            34                           168                                                         1315258847                         1315258881                          NULL                                                     1315258830

Connect and disconnect add up to the duration of 34 but there is a difference of 17 between origination and connection.

From: Wes Sisk [mailto:wsisk at cisco.com]
Sent: Wednesday, November 02, 2011 1:25 PM
To: Wellnitz, Erick A.
Cc: <mailto:cisco-voip at puck.nether.net> cisco-voip at puck.nether.net<mailto:cisco-voip at puck.nether.net>
Subject: Re: [cisco-voip] no audio for 10 seconds

good observation on the PVDM3's it could be an IOS DSP problem.  Do phones from other locations call through that gateway?

how about this:

# find 9971 model
admin:run sql select first 10 * from typemodel where name like '%9971%'
enum name       tkclass moniker          showindevicedefaults defaultdtmfcapability tkrisclass tksubclass
==== ========== ======= ================ ==================== ===================== ========== ==========
493  Cisco 9971 1       MODEL_Cisco_9971 t                    10                    1          1
# find 10x 9971 phones
admin:run sql select first 10 pkid,name,description from device where tkmodel=493
pkid                                 name            description
==================================== =============== ===========
bfc01793-c71a-4207-ba41-ef7c81d966d7 SEP44E4D9449BB4 Auto 86102

# find CDR's based on devicename from last query admin:run sql car select origdevicename,duration,orignumberpacketsreceived,datetimeconnect,datetimedisconnect,destnumberpacketssent,datetimeorigination from tbl_billing_data where origdevicename in ("SEP44E4D9449BB4","SEP44E4D9449BB5")
origdevicename duration orignumberpacketsreceived datetimeconnect datetimedisconnect destnumberpacketssent datetimeorigination ============== ======== ========================= =============== ================== ===================== ===================

import the final output into excel.  compare packetssent and packetsreceived. compare packet counts to duration. compare duration to datetimedisconnect-datetimeconnect.

/wes

On Nov 2, 2011, at 12:25 PM, Wellnitz, Erick A. wrote:

Only reported on the 9971.  No consistency to the issue.

We've tried a selective firmware update to 9-2-2 without success.
We have checked for possible delays in call setup with MGCP, Q.931 and SIP with no clues or even momentary delays in the setup.
We have exhausted tweak options for the QSIG integration (timers, etc.) We have attempted to recreate the issue without success even with a phone known to experience the issue.

Unfortunately we don't have the tools available to determine if we're experiencing troubles with the RTP stream initialization, especially seeing as the issue is intermittent.

The one thing which stands out to me is that we have two other locations with the 9971 and they are not experiencing these issues.  Only difference here is the PBX integration with QSIG.  This site is also the only one with PVDM3.

To: Wellnitz, Erick A.
Cc: <mailto:cisco-voip at puck.nether.net> cisco-voip at puck.nether.net<mailto:cisco-voip at puck.nether.net>
Subject: Re: [cisco-voip] no audio for 10 seconds

Hi Erick,

Based on history you're a pretty enterprising.  Any insight on what you've checked so far?

Is it only reported on 9971's? Is there any other consistency in the reports thus far?

One of my favorite approaches to get a bit of insight on this type of issue is CMR data.  Look at CMR's for "simple" calls - direct call, connection, duration hangup; no mute/transfer/hold/park/etc.  If you're using fixed packet rate audio codec (not ilbc) then the packets sent/received should be linearly proportional to call duration barring a few special situations.

Assuming 20packets per second are the number of packets received appropriate for the duration of the call?

If the packet counts are correct then it's an audio cut through issue on the gateway or audio rendering issue inside the phone.
gateway concerns: call going into connected state, any progress indicators exchanged (affect audio cut through timing)

If the packet counts are off then there is a network issue.

/wes

On Nov 2, 2011, at 11:46 AM, Wellnitz, Erick A. wrote:

All, especially you really smart guys at Cisco...

We're at a loss for where to continue looking for a potential cause on a very intermittent issue.

Mixed 9971 and 7975 environment with shared lines.  Inbound calls come into our old (and temporary) Siemens PBX.  Cross a QSIG trunk group to Cisco 3945 with PVDM3-192

Caller on our Cisco side places a call.  They hear ringback.  Call connects without audio for about 10 seconds.

We couldn't find anything similar to this in bug toolkit.

9971 firmware: 9-2-1
CUCM Ver: 8.0.3.21900

SR 619601919

We have a  laundry list of issues with the 9971 model but this is currently the most urgent.

ERICK A. WELLNITZ
Network Engineer
Katten Muchin Rosenman LLP
525 W. Monroe Street / Chicago, IL 60661-3693 p / (312) 577-8041 erick.wellnitz at kattenlaw.com<mailto:erick.wellnitz at kattenlaw.com> / www.kattenlaw.com<http://www.kattenlaw.com/>

===========================================================

CIRCULAR 230 DISCLOSURE: Pursuant to Regulations Governing Practice Before the Internal Revenue

Service, any tax advice contained herein is not intended or written to be used and cannot be used

by a taxpayer for the purpose of avoiding tax penalties that may be imposed on the taxpayer.

===========================================================

CONFIDENTIALITY NOTICE:

This electronic mail message and any attached files contain information intended for the exclusive

use of the individual or entity to whom it is addressed and may contain information that is

proprietary, privileged, confidential and/or exempt from disclosure under applicable law.  If you

are not the intended recipient, you are hereby notified that any viewing, copying, disclosure or

distribution of this information may be subject to legal restriction or sanction.  Please notify

the sender, by electronic mail or telephone, of any unintended recipients and delete the original

message without making any copies.

===========================================================

NOTIFICATION:  Katten Muchin Rosenman LLP is an Illinois limited liability partnership that has

elected to be governed by the Illinois Uniform Partnership Act (1997).

===========================================================

_______________________________________________
cisco-voip mailing list
<mailto:cisco-voip at puck.nether.net>cisco-voip at puck.nether.net<mailto:cisco-voip at puck.nether.net>
<https://puck.nether.net/mailman/listinfo/cisco-voip>https://puck.nether.net/mailman/listinfo/cisco-voip

_______________________________________________
cisco-voip mailing list
cisco-voip at puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-voip