[cisco-voip] UCS C series audio issues

[Daniel Pagan]

I've worked an issue very similar to this on a UCS C series server. Some background... the UCS server hosted two CUCM nodes and a CUC subscriber. The issue was first reported as conference audio sounding "garbled". Troubleshooting the issue for a few minutes revealed that we could recreate the issue on both CUCM nodes through use of CFB and MOH resources, and on Unity Connection with standard vm playback via TUI but recordings relayed via SMTP were fine. Additional troubleshooting confirmed network issues were not the cause. Resolving the issue came down to two action items:


1-      Update the NIC drivers through root access in ESXi.

We found the NIC drivers installed during the build were incompatible with the version of ESXi running on the UCS server.

To find your current driver version via ESXi CLI, use the output from esxcfg-nics -l combined with ethtool -i:
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1027206



We used the tool below to determine the exact driver version required based on server model, NIC adapter model, etc.

http://www.cisco.com/web/techdoc/ucs/interoperability/matrix/matrix.html


2-      Update vm tools through vSphere

The vm tools weren't up-to-date in this customer's environment. I highly doubt this was the root cause but it doesn't hurt to bring a customer's environment to meeting best practices :)

We also updated the CIMC and BIOS during our maintenance window for good measure. The problem was reproducible nearly 100% of the time prior to updating NIC drivers and was immediately resolved post maintenance window. I'm not sure if this is the same issue you're facing but the interoperability tool might be worth looking into.

Hope this helps.

- Dan

From: cisco-voip [mailto:cisco-voip-bounces at puck.nether.net] On Behalf Of Erick Bergquist
Sent: Thursday, February 20, 2014 4:01 PM
To: Brian Meade (brmeade)
Cc: voip puck
Subject: Re: [cisco-voip] UCS C series audio issues

In Wireshark the RTP stream delta is between 16ms and 24ms, not a constant 20ms with capture off switchport plugged into UCS.  This is with CUCM updated to 8.5.1 SU6 which has the bug fix for CSCtz29142 from release notes.

No VM Snapshots enabled either.

On Wed, Feb 12, 2014 at 1:50 PM, Erick Bergquist <erickbee at gmail.com<mailto:erickbee at gmail.com>> wrote:
Thanks guys, I will check into those items.

One system having this is effected by the bug id (8.5.1.11900-21, SU1) but the other was on 8.6.2a and we updated to 8.6.2a SU3 and same issue and the bug info states it should be fixed in 8.6.2a SU3 (23900).

On Wed, Feb 12, 2014 at 1:39 PM, Brian Meade (brmeade) <brmeade at cisco.com<mailto:brmeade at cisco.com>> wrote:
What full CUCM version are you running?

You're probably hitting CSCtz29142-VMware CUCM Media resources introduces jitter causing bad voice quality.

If you get a packet capture from CUCM for one of the calls using a software resource.  check the Delta values for the RTP stream in Wireshark.  If you're hitting this bug, delta will swap back and forth between 16ms and 24ms.  It should be a constant delta of 20ms in a working scenario.

Thanks,
Brian Meade

From: cisco-voip [mailto:cisco-voip-bounces at puck.nether.net<mailto:cisco-voip-bounces at puck.nether.net>] On Behalf Of Erick Bergquist
Sent: Wednesday, February 12, 2014 2:19 PM
To: voip puck
Subject: [cisco-voip] UCS C series audio issues

Has anyone experienced choppy/garbled/broken-up audio for MoH, conference calls or MTP audio from UC Servers on UCS C series?  All software based (IPVMS).

The network side is fine, QoS and no errors on the switch ports. The CPU load is minimal and CM performance minimal also, so no CPU pegging.

Thanks,
Erick



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/cisco-voip/attachments/20140220/fe0040fc/attachment.html>