[cisco-voip] Delay issues with CUCM862a Can someone point me in the right direction ??

Fri Apr 13 08:20:17 EDT 2012

What process was spiking the CPU?  The symptoms you describe sound 
similar to CSCty36110.

-Adam

------------------------------------------------------------------------
*From:* Gregory Wenzel <gwenzel at conres.com>
*Sent:* Fri, Apr 13, 2012 7:28:08 AM
*To:* 'Wes Sisk' <wsisk at cisco.com>
*CC:* "cisco-voip at puck.nether.net" <cisco-voip at puck.nether.net>
*Subject:* Re: [cisco-voip] Delay issues with CUCM862a Can someone point 
me in the right direction ??

> Other complaints started coming in that worried me.
>
> Besides the delayed audio the directory services menu options 4 and 5 
> intermittantly showed an error then it went away and showed up.
>
> Directory searches intermittently take 8 seconds to appear.
>
> I started monitoring cpu and I am seeing it spike to 96% and stay near 
> 80-90% then dropping down to 70% then I saw a LowMemoryAlarm message...
>
> I stopped troubleshooting the audio delay maybe this is a function of 
> the cpu spike.
>
> I did use the recommended ova file from the wiki site when I built the 
> vm in v8.5 but when I upgraded to 8.6.2a I didn't add the extra memory.
>
> I am going to recommend to my client to add more memory to the vm.
>
> Greg
>
> *From:*Wes Sisk [mailto:wsisk at cisco.com]
> *Sent:* Thursday, April 12, 2012 11:42 AM
> *To:* Gregory Wenzel
> *Cc:* cisco-voip at puck.nether.net
> *Subject:* Re: [cisco-voip] Delay issues with CUCM862a Can someone 
> point me in the right direction ??
>
> StartMediaTransmission can only be sent when the peer device responds 
> with the UDP/RTP IP and port where it wants to receive audio. 
>  Investigate the signaling with the remote device. Either the remote 
> device is slow to respond, the response is delayed back to ccm, or ccm 
> is slow to propagate that information back to this phone device.
>
> Regards,
>
> Wes
>
> On Apr 12, 2012, at 11:25 AM, Gregory Wenzel wrote:
>
>
>
> Im using a tool called CCM Translator X that Paul Geralt wrote a few 
> years back.
>
> I am seeing a delay in signaling for the network that is in bldg 1.
>
> I see the call state for normal openreceivechannel then its 
> stationopenreceivechannelAckID and openreceivechannelAck are within a 
> few milliseconds of each other
>
> The normal time shows
>
> OpenReceiveChannel at time 10.26.28.362
>
> Then
>
> startMediaTransmission at time 10.26.28.364
>
> Then
>
> Both stationopenreceivechannelAckID and openreceivechannelAck at time 
> 10.26.28.367
>
> I see a delay in the startMediaTransmision for this particular user 
> has a gap between the openreceivechannel and the 
> startMediaTransmission and its also missing the 
> stationopenreceivechannelAckID and it seems there is a big gap in the 
> startMedisTransmission
>
> OpenReceiveChannel at time 10.26.48.523
>
> Then
>
>  Instead of startMediaTransmission starting I see that 
> OpenReceiveChannelAck at 10.26.48.560
>
> Then startMediaTransmisttion starts much later at 10.26.48.617
>
> Greg
>
> *From:*Wes Sisk [mailto:wsisk at cisco.com]
> *Sent:*Thursday, April 12, 2012 10:30 AM
> *To:*Gregory Wenzel
> *Cc:*cisco-voip at puck.nether.net <mailto:cisco-voip at puck.nether.net>
> *Subject:*Re: [cisco-voip] Delay issues with CUCM862a Can someone 
> point me in the right direction ??
>
> heuristics say:
>
> upgrade is at least OS if not server hardware replacement.  did 
> speed/duplex get set correctly? SCCP uses TCP which will retransmit 
> lost packets but lost packets will manifest to user as delays.
>
> Otherwise, for delay answering a bit more clarification would help:
>
> * how are they answering calls? handset/headset/speaker/answer 
> softkey/3rd party headset lifter/bluetooth headset/...
>
> * does the phone promptly enter the offhook state? look at the lcd, 
> does it look connected?
>
> * does the delay happen in audio cut through? i.e. phone is clearly 
> offhook but audio does not cut through for 5-10 sec?
>
> * what model phones and what phone load?
>
> ccm sdi and sdl traces can be used to investigate some of this. find 
> offhook or answer from phone, then the callstate=5 connected,  and 
> then the subsequent openreceivechannel/startmediatransmission sent to 
> phone. if these are "timely" in the ccm sdi traces then the culprit is 
> either:
>
> 1. delays in the network delivering these messages
>
> 2. delay in the far end device beginning to transmit or receive audio
>
> =>for this check the ccm sdi traces for signaling with the far end 
> device. any sign of delays in signaling?
>
> after clearing everything possible in the SDI/SDL traces the next step 
> would be a packet capture at the endpoint device to investigate 
> signaling, tcp retransmits, and RTP stream statistics there.
>
> For the phones there are a few known issues. From the TAC Hot Issues 
> feed for Endpoints onCisco.com <http://Cisco.com>:
>
>
> Delay in starting RTP from 6941, Fixed CSCtc96381
> RTP sequence number gaps result in one way audio on 7960, Fixed CSCtn69362
> Minimize RTP Delay from HW side when answering call on RT-Lite, 
> Open CSCtd57345
> delay in audio cut through on 894x, Open CSCtu08287
> 6945 sccp: SW polling time improvement for RTL Clipping issue, 
> Fixed CSCtq80427
> Intermittent one way audio on 6901 phones, Fixed CSCts39379
>
> ( there is one for rtp seq number looping that caused lots of problems 
> that i cannot find at the moment, i'll keep looking but this should 
> get you started)
>
> /wes
>
> On Apr 11, 2012, at 6:24 PM, Gregory Wenzel wrote:
>
> I've just started troubleshooting these issues for a client whom I 
> upgraded from ccm4.x to cucm8.6.2a.
> They claim the problem did not exist when ccm4.x was in production.
>
> Can someone tell me which traces are best to tblshoot kind of issue, 
> kick me in the right direction before I start RTMT on cucm and debug 
> on the gateways.
>
> The problem is not system wide and only occur in this one particular 
> building.
>
> *Complaints*
>
> Delay in answering calls in Bldg 1
>
> Delay in transferring call in Bldg 1
>
> Problems started immediately after the upgrade but they are telling me 
> now 4 months later
>
> IP Comm - Web interface is slow Sometimes when making a call you do 
> not hear the ring tone
>
> *Variables*
>
> Switches used  "EXTREME" brand non-Cisco and I do not have access to 
> these switches either
>
> No network changes occurred since the upgrade that the customer wants 
> to admit to.
>
> Has its own subscriber in bldg 1
>
> Every other phone in the 2K phone sized network all register to the 
> same subscriber do not have this issue
>
> It uses a 2811 gateway running 12.4.24T that is on the other end of a 
> 1 gig fiber link across the street and I have no access to the devices 
> that connect the two building together.
>
> What major differences between signaling with version ccm4 and 
> cucm862a would cause this?
>
> I'm looking through the bug finder now. I would love to tell the 
> client it's their Extreme switches but why then would it work on the 
> old windows version.
>
> Maybe it didn't work and the problem just go more pronounced after the 
> upgrade.
>
> I'm just looking for someone to point me in the right direction...:)
>
> TIA
>
>
>
>
>
> Greg Wenzel
>
>
> This message w/attachments (message) is solely for the use of the 
> intended recipient(s) and may contain information that is privileged, 
> confidential or proprietary. If you are not an intended recipient, 
> please notify the sender, and then please delete and destroy all 
> copies and attachments, and be advised that any review or 
> dissemination of, or the taking of any action in reliance on, the 
> information contained in or attached to this message is prohibited. 
> Unless specifically indicated, this message is not an offer to sell or 
> a solicitation of any products.      
> _______________________________________________
> cisco-voip mailing list
> cisco-voip at puck.nether.net <mailto:cisco-voip at puck.nether.net>
> https://puck.nether.net/mailman/listinfo/cisco-voip
>
>
>
> _______________________________________________
> cisco-voip mailing list
> cisco-voip at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-voip

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/cisco-voip/attachments/20120413/472d49b7/attachment.html>