<div dir="ltr">I had a case where the guy had his CUCM server switch ports SPANed to some dedicated capture gear. He was able to get me captures from all his CUCM servers up to a week back. That was a pretty nice thing to have. Unfortunately, you're limited on the phone side in that you'll need to set it up for quite a few phones to increase your chances of catching it.</div>
<div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Jun 5, 2014 at 2:29 PM, Lelio Fulgenzi <span dir="ltr"><<a href="mailto:lelio@uoguelph.ca" target="_blank">lelio@uoguelph.ca</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div><div style="font-family:verdana,helvetica,sans-serif;font-size:10pt;color:#000000">Just to add, we too are experiencing the same issue. Although I don't think we've had complaints of dropped calls.<br><br>We first saw the issue when upgrading from v4.1 to v7.1 around two years ago, but because we didn't get complaints we had to prioritize. We're now seeing many more dropped phones. I'm not sure if the alerts were there in v4.1 and we just didn't have the alerts set up to come to us or not. It was a long time ago.<br>
<br>We opened a TAC case, but unfortunately, we have not been able to work on it due to other pressing issues. I have to go back to our team to co-ordinate getting this going again. I'd like to resolve this. There should be no reason to have such a large number of de-registration events.<br>
<br>We are primarily 7940/60 7912 phones with a number of 7911s, 7942, 7962 interspersed.<br><br>I too am of the thought that it's not CUCM version dependent, but more so either switch configuration (loop detection) or firewall issues.<br>
<br>The case owner said the only real way to figure things out is to get a packet capture from both the callmanager and the phones when it happens. <br><br>Unfortunately, it's difficult to do so, since how do you guess where the phones are going to un-register from? We might have to do some sort of multi-vlan capture, but that is gonna be tough too.<br>
<br><br><br><div><span name="x"></span>---<br>Lelio Fulgenzi, B.A.<br>Senior Analyst, Network Infrastructure<br>Computing and Communications Services (CCS)<br>University of Guelph<br><br><a href="tel:519%E2%80%90824%E2%80%904120%20Ext%2056354" value="+15198244120" target="_blank">519‐824‐4120 Ext 56354</a><br>
<a href="mailto:lelio@uoguelph.ca" target="_blank">lelio@uoguelph.ca</a><br><a href="http://www.uoguelph.ca/ccs" target="_blank">www.uoguelph.ca/ccs</a><br>Room 037, Animal Science and Nutrition Building<br>Guelph, Ontario, N1G 2W1<span name="x"></span><br>
</div><br><hr><div style="color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt"><b>From: </b>"Tashi Mar" <<a href="mailto:tmar@align.com" target="_blank">tmar@align.com</a>><br>
<b>To: </b>"Terry Oakley" <<a href="mailto:Terry.Oakley@rdc.ab.ca" target="_blank">Terry.Oakley@rdc.ab.ca</a>>, "Dennis Heim" <<a href="mailto:Dennis.Heim@wwt.com" target="_blank">Dennis.Heim@wwt.com</a>>, <a href="mailto:cisco-voip@puck.nether.net" target="_blank">cisco-voip@puck.nether.net</a><br>
<b>Sent: </b>Thursday, June 5, 2014 2:10:17 PM<div><div class="h5"><br><b>Subject: </b>Re: [cisco-voip] Intermittent phone resets<br><br>
<div>
<p class="MsoNormal"><span style="color:#1f497d">Hi all, appreciate the feedback. There are some SCCP phones (along with SIP) that reset too; TAC could not pinpoint any phone firmware or CUCM bugs either.</span></p>
<p class="MsoNormal"><span style="color:#1f497d"> </span></p>
<p class="MsoNormal"><span style="color:#1f497d">Will find out the switch OS, but environment is all stacked 3750s at the core and access layers in both locations. Any chance you recall the bug ID?</span></p>
<p class="MsoNormal"><span style="color:#1f497d"> </span></p>
<p class="MsoNormal"><span style="color:#1f497d">Definitely will check into:</span></p>
<p><span style="color:#1f497d"><span>-<span style="font:7.0pt "Times New Roman"">
</span></span></span><span style="color:#1f497d">Switch bug related to spanning tree and excessive Topology Change Notifications (TCN). They actually replaced some access switches, but that doesn’t mean the configuration was modified.</span><span style="color:#1f497d"></span></p>
<p><span style="color:#1f497d"><span>-<span style="font:7.0pt "Times New Roman"">
</span></span></span><span style="color:#1f497d">Circular buffer packet captures</span></p>
<p class="MsoNormal"><span style="color:#1f497d"> </span></p>
<p class="MsoNormal"><span style="color:#1f497d">We did verify that the ASAs are not doing any SIP or SCCP inspection; voice traffic on this network does not pass through firewalls anyway.</span></p>
<p class="MsoNormal"><span style="color:#1f497d"> </span></p>
<p class="MsoNormal"><span style="color:#1f497d">Thanks!</span></p>
<p class="MsoNormal"><span style="color:#1f497d"> </span></p>
<div>
<p class="MsoNormal" style="margin-right:0in;margin-bottom:2.0pt;margin-left:0in">
<span style="font-size:12.0pt;color:#595959">Tashi Mar</span><span style="font-size:12.0pt;color:#7f7f7f"> |
</span><span style="font-size:9.0pt;color:#7f7f7f">IP Telephony Engineer</span><span style="font-size:12.0pt;color:#7f7f7f"></span></p>
</div>
<p class="MsoNormal"><span style="color:#1f497d"> </span></p>
<div>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Terry Oakley [mailto:<a href="mailto:Terry.Oakley@rdc.ab.ca" target="_blank">Terry.Oakley@rdc.ab.ca</a>]
<br>
<b>Sent:</b> Thursday, June 05, 2014 2:02 PM<br>
<b>To:</b> Heim, Dennis; Mar, Tashi; <a href="mailto:cisco-voip@puck.nether.net" target="_blank">cisco-voip@puck.nether.net</a><br>
<b>Subject:</b> RE: Intermittent phone resets</span></p>
</div>
</div>
<p class="MsoNormal"> </p>
<p class="MsoNormal"><span style="color:#1f497d" lang="EN-CA">What version of switches and OS were you using?</span></p>
<p class="MsoNormal"><span style="color:#1f497d" lang="EN-CA"> </span></p>
<p class="MsoNormal"><span style="color:#1f497d" lang="EN-CA">Thanks</span></p>
<p class="MsoNormal"><span style="color:#1f497d" lang="EN-CA"> </span></p>
<p class="MsoNormal"><span style="color:#1f497d" lang="EN-CA">Terry</span></p>
<p class="MsoNormal"><span style="color:#1f497d" lang="EN-CA"> </span></p>
<div>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> cisco-voip [<a href="mailto:cisco-voip-bounces@puck.nether.net" target="_blank">mailto:cisco-voip-bounces@puck.nether.net</a>]
<b>On Behalf Of </b>Heim, Dennis<br>
<b>Sent:</b> June-05-14 11:50 AM<br>
<b>To:</b> Mar, Tashi; <a href="mailto:cisco-voip@puck.nether.net" target="_blank">cisco-voip@puck.nether.net</a><br>
<b>Subject:</b> Re: [cisco-voip] Intermittent phone resets</span></p>
</div>
</div>
<p class="MsoNormal"><span lang="EN-CA"> </span></p>
<p class="MsoNormal"><span style="color:#1f497d">I had this awhile back, and it ended up being a bug on the switches related to spanning tree and excessive Topology Change Notifications (TCN).</span></p>
<p class="MsoNormal"><span style="color:#1f497d"> </span></p>
<div>
<p class="MsoNormal" style="text-autospace:none"><b><span style="font-size:10.0pt;color:black">Dennis Heim | Collaboration Solutions Architect</span></b><span style="color:black"></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:10.0pt;color:black">World Wide Technology, Inc. | <a href="tel:%2B1%20314-212-1814" value="+13142121814" target="_blank">+1 314-212-1814</a></span></p>
<p class="MsoNormal" style="text-autospace:none"><a href="https://twitter.com/CollabSensei" target="_blank"><span style="text-decoration:none"><img src="cid:image001.png@01CF80C7.E0808500" alt="twitter" border="0" height="25" width="124"></span></a><span style="color:black"></span></p>
<p class="MsoNormal" style="text-autospace:none"><a><span style="color:#1f497d;text-decoration:none"><img src="cid:image002.png@01CF80C7.E0808500" alt="chat" border="0" height="28" width="95"></span></a><a href="tel:+13142121814" target="_blank"><span style="color:#1f497d;text-decoration:none"><img src="cid:image003.png@01CF80C7.E0808500" alt="Phone" border="0" height="28" width="95"></span></a><a><span style="color:#1f497d;text-decoration:none"><img src="cid:image004.png@01CF80C7.E0808500" alt="video" border="0" height="28" width="95"></span></a><span style="color:#1f497d"></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="color:black"> </span></p>
</div>
<p class="MsoNormal"><span style="color:#1f497d"> </span></p>
<div>
<div style="border:none;border-top:solid #e1e1e1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> cisco-voip [<a href="mailto:cisco-voip-bounces@puck.nether.net" target="_blank">mailto:cisco-voip-bounces@puck.nether.net</a>]
<b>On Behalf Of </b>Mar, Tashi<br>
<b>Sent:</b> Thursday, June 05, 2014 1:43 PM<br>
<b>To:</b> <a href="mailto:cisco-voip@puck.nether.net" target="_blank">cisco-voip@puck.nether.net</a><br>
<b>Subject:</b> [cisco-voip] Intermittent phone resets</p>
</div>
</div>
<p class="MsoNormal"> </p>
<p class="MsoNormal">Hi, troubleshooting an issue where phones intermittently reset, either on hook or off hook (drops the call in progress). Logs and TAC claim it’s a network issue, but has anyone seen similar problems on Call Manager 9.1? Basically, we
want to rule out Call Manager as the culprit.</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">Majority of phones using SIP, models 8891 and 9951, experiencing issues similar to those outlined here:
<a href="https://supportforums.cisco.com/discussion/10846666/ip-phones-randomly-rebooting" target="_blank">
https://supportforums.cisco.com/discussion/10846666/ip-phones-randomly-rebooting</a></p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">Steps already taken:</p>
<p class="MsoNormal">- upgrade to latest firmware</p>
<p class="MsoNormal">- verify no POE errors on switch</p>
<p class="MsoNormal">- cpu, memory on UCS servers ok</p>
<p class="MsoNormal">- point phones to backup SUB, no change. Phones randomly reset when homed to Pub or Sub (both in separate locations).</p>
<p class="MsoNormal">- Wireshark to see KeepAlives dropping</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">1. Phone logs show:</p>
<p class="MsoNormal"><span style="font-size:10.0pt">SEP0DD3B4E90000.cnf.xml.sgn (HTTP)</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt"> 8:05:20p TCP connection timed out</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt"> 8:07:22p Falling back to different CUCM</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt"> 11:16:50p TCP connection timed out</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt"> 11:18:51p Falling back to different CUCM</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt"> 6:52:38p TCP connection timed out</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt"> 6:54:39p Falling back to different CUCM</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt"> 9:55:04a TCP connection timed out</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt"> 9:57:05a Falling back to different CUCM</span></p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">2. Call Manager logs show:</p>
<p class="MsoNormal">- phone unregisters from Primary CUCM due to:</p>
<p class="MsoNormal">[Reason=6] ConnectivityError - Network communication between the device and Unified CM has been interrupted.</p>
<p class="MsoNormal">- phones fallback to secondary CUCM and then rehome to Primary CUCM:</p>
<p class="MsoNormal">Reason code = 28, FallbackInitiated - The device has initiated a fallback and will automatically re-register to a higher-priority Unified CM. No action is necessary.</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">3. Call Manager syslogs display EndPointUnregistered:</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal"><span style="font-size:10.0pt">Apr 01 14:34:18 %UC_CALLMANAGER-3-EndPointUnregistered: %[DeviceName=SEP0DD3B4E90000][IPAddress=10.1.10.211][Protocol=SIP][DeviceType=540][Description=User][Reason=6][IPAddrAttributes=0][LastSignalReceived=SIPConnControlInd][CallState=5286-call_initiated1][AppID=Cisco
CallManager][ClusterID=NAregion][NodeID=CCMSub]: An endpoint has unregistered</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt"> </span></p>
<p class="MsoNormal"><span style="font-size:10.0pt">Apr 01 14:36:22 %UC_-3-LastOutOfServiceInformation: %[DeviceName=SEP0DD3B4E90000][DeviceIPv4Address=<a href="http://10.1.10.211/24%5D%5BIPv4DefaultGateway=192.168.51.1%5D%5BDeviceIPv6Address=::%5D%5BIPv6DefaultGateway=::%5D%5BModelNumber=CP-8961%5D%5BNeighborIPv4Address=%5D%5BNeighborIPv6Address=%5D%5BNeighborDeviceID=%5D%5BUNKNOWN_PARAMNAME:NeighborIPortID=%5D%5BDHCPv4Status=1%5D%5BDHCPv6Status=3%5D%5BTFTPCfgStatus=1%5D%5BDNSStatusUnifiedCM1=4%5D%5BDNSStatusUnifiedCM2=4%5D%5BDNSStatusUnifiedCM3=0%5D%5BUNKNOWN_PARAMNAME:DNSv6StatusUnifiedCM1=0%5D%5BUNKNOWN_PARAMNAME:DNSv6StatusUnifiedCM2=0%5D%5BUNKNOWN_PARAMNAME:DNSv6StatusUnifiedCM3=0%5D%5BVoiceVLAN=50%5D%5BUnifiedCMIPAddress=10.1.10.5%5D%5BLocalPort=50382%5D%5BTimeStamp=1396377375%5D%5BReasonForOutOfService=18%5D%5BLastProtocolEventSent=Sent:INVITE" target="_blank">10.1.10.211/24][IPv4DefaultGateway=192.168.51.1][DeviceIPv6Address=::][IPv6DefaultGateway=::][ModelNumber=CP-8961][NeighborIPv4Address=][NeighborIPv6Address=][NeighborDeviceID=][UNKNOWN_PARAMNAME:NeighborIPortID=][DHCPv4Status=1][DHCPv6Status=3][TFTPCfgStatus=1][DNSStatusUnifiedCM1=4][DNSStatusUnifiedCM2=4][DNSStatusUnifiedCM3=0][UNKNOWN_PARAMNAME:DNSv6StatusUnifiedCM1=0][UNKNOWN_PARAMNAME:DNSv6StatusUnifiedCM2=0][UNKNOWN_PARAMNAME:DNSv6StatusUnifiedCM3=0][VoiceVLAN=50][UnifiedCMIPAddress=10.1.10.5][LocalPort=50382][TimeStamp=1396377375][ReasonForOutOfService=18][LastProtocolEventSent=Sent:INVITE</a>
<a>sip:915138521010@10.1.10.2;user=phone</a> SIP/2.0 Cseq:101 INVITE CallId:b4e9b08c-a3ad023c-1fec1fa7-3b9727b1@10.1][LastProtocolEventReceived=Rcvd:SIP/2.0 202 Accepted Cseq:104 REFER <a href="mailto:CallId%3Ab4e9b08c-a3ad026a-30d892f1-02c748ba@10.1.10.211" target="_blank">CallId:b4e9b08c-a3ad026a-30d892f1-02c748ba@10.1.10.211</a>][AppID=Cisco
CallManager][ClusterID=NAregion][NodeID=CCMSub]: Information related to the last out-of-service event
</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt"> </span></p>
<p class="MsoNormal"><span style="font-size:10.0pt">Apr 01 14:36:24 %UC_CALLMANAGER-6-EndPointRegistered: %[DeviceName=SEP0DD3B4E90000][IPAddress=10.1.10.211][Protocol=SIP][DeviceType=540][PerfMonObjType=2][Description=User][UserID=User1][LoadID=sip8961.9-4-1-9][AssociatedDNs=5286,5285,5242,5237,5262,5217,5284,5294,5282,#86,][MACAddress=0DD3B4E90000][IPAddrAttributes=0][ActiveLoadId=sip8961.9-4-1-9.loads][InactiveLoadId=sip8961.9-3-2SR1-1.loads][AppID=Cisco
CallManager][ClusterID=NAregion][NodeID=CCMSub]: Endpoint registered</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt"> </span></p>
<p class="MsoNormal"><span style="font-size:10.0pt">Apr 01 14:36:24 %UC_CALLMANAGER-6-DeviceDnInformation: %[DeviceName=SEP0DD3B4E90000][DeviceType=540][StationDesc=User][StationDn=5286,5285,5242,5237,5262,5217,5284,5294,5282,#86][AppID=Cisco CallManager][ClusterID=NAregion][NodeID=CCMSub]:
List of directory numbers (DN) associated with this device</span></p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">4. Switchport configuration:</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">interface GigabitEthernet3/0/x</p>
<p class="MsoNormal">switchport access vlan 10</p>
<p class="MsoNormal">switchport mode access</p>
<p class="MsoNormal">switchport voice vlan 11</p>
<p class="MsoNormal">switchport port-security maximum 3</p>
<p class="MsoNormal">switchport port-security</p>
<p class="MsoNormal">switchport port-security aging time 2</p>
<p class="MsoNormal">switchport port-security violation restrict</p>
<p class="MsoNormal">switchport port-security aging type inactivity</p>
<p class="MsoNormal">load-interval 30</p>
<p class="MsoNormal">spanning-tree portfast</p>
<p class="MsoNormal">spanning-tree bpduguard enable</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal">Thanks, Tashi</p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Times New Roman","serif""> </span></p>
<div class="MsoNormal" style="text-align:center" align="center"><span style="font-size:12.0pt;font-family:"Times New Roman","serif"">
<hr align="center" size="2" width="100%">
</span></div>
<p class="MsoNormal"><span style="font-size:7.5pt;font-family:"Arial","sans-serif";color:gray"><br>
The information contained in this message is confidential and is intended only for the use of the individual or entity named above. It may contain proprietary or legally privileged information. Mistransmission shall not constitute a waiver of any rights or
privileges. If you are not the designated recipient of this message, you are hereby notified that any use, dissemination, distribution or reproduction of this message is strictly prohibited. If you have received this message in error, please immediately notify
the sender.<br>
<br>
Although this e-mail and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that they are virus-free. Align Communications
Inc. does not accept, and specifically disclaims, any liability or obligation for any loss or damage arising in any way from the use of this e-mail or any attachment. Thank You.</span><span style="font-size:12.0pt;font-family:"Times New Roman","serif""></span></p>
</div>
<br>
<hr>
<font color="Gray" face="Arial" size="1"><br>
The information contained in this message is confidential and is intended only for the use of the individual or entity named above. It may contain proprietary or legally privileged information. Mistransmission shall not constitute a waiver of any rights or
privileges. If you are not the designated recipient of this message, you are hereby notified that any use, dissemination, distribution or reproduction of this message is strictly prohibited. If you have received this message in error, please immediately notify
the sender.<br>
<br>
Although this e-mail and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that they are virus-free. Align Communications
Inc. does not accept, and specifically disclaims, any liability or obligation for any loss or damage arising in any way from the use of this e-mail or any attachment. Thank You.<br>
</font>
<br></div></div><div class="">_______________________________________________<br>cisco-voip mailing list<br><a href="mailto:cisco-voip@puck.nether.net" target="_blank">cisco-voip@puck.nether.net</a><br><a href="https://puck.nether.net/mailman/listinfo/cisco-voip" target="_blank">https://puck.nether.net/mailman/listinfo/cisco-voip</a><br>
</div></div><br></div></div><br>_______________________________________________<br>
cisco-voip mailing list<br>
<a href="mailto:cisco-voip@puck.nether.net">cisco-voip@puck.nether.net</a><br>
<a href="https://puck.nether.net/mailman/listinfo/cisco-voip" target="_blank">https://puck.nether.net/mailman/listinfo/cisco-voip</a><br>
<br></blockquote></div><br></div>