<HTML xmlns="http://www.w3.org/TR/REC-html40" xmlns:v = "urn:schemas-microsoft-com:vml" xmlns:o = "urn:schemas-microsoft-com:office:office" xmlns:w = "urn:schemas-microsoft-com:office:word" xmlns:x = "urn:schemas-microsoft-com:office:excel" xmlns:p = "urn:schemas-microsoft-com:office:powerpoint" xmlns:a = "urn:schemas-microsoft-com:office:access" xmlns:dt = "uuid:C2F41010-65B3-11d1-A29F-00AA00C14882" xmlns:s = "uuid:BDC6E3F0-6DA3-11d1-A2A3-00AA00C14882" xmlns:rs = "urn:schemas-microsoft-com:rowset" xmlns:z = "#RowsetSchema" xmlns:b = "urn:schemas-microsoft-com:office:publisher" xmlns:ss = "urn:schemas-microsoft-com:office:spreadsheet" xmlns:c = "urn:schemas-microsoft-com:office:component:spreadsheet" xmlns:oa = "urn:schemas-microsoft-com:office:activation" xmlns:html = "http://www.w3.org/TR/REC-html40" xmlns:q = "http://schemas.xmlsoap.org/soap/envelope/" XMLNS:D = "DAV:" xmlns:x2 = "http://schemas.microsoft.com/office/excel/2003/xml" xmlns:ois = "http://schemas.microsoft.com/sharepoint/soap/ois/" xmlns:dir = "http://schemas.microsoft.com/sharepoint/soap/directory/" xmlns:ds = "http://www.w3.org/2000/09/xmldsig#" xmlns:dsp = "http://schemas.microsoft.com/sharepoint/dsp" xmlns:udc = "http://schemas.microsoft.com/data/udc" xmlns:xsd = "http://www.w3.org/2001/XMLSchema" xmlns:sp = "http://schemas.microsoft.com/sharepoint/" xmlns:sps = "http://schemas.microsoft.com/sharepoint/soap/" xmlns:xsi = "http://www.w3.org/2001/XMLSchema-instance" xmlns:udcxf = "http://schemas.microsoft.com/data/udc/xmlfile" xmlns:wf = "http://schemas.microsoft.com/sharepoint/soap/workflow/" xmlns:mver = "http://schemas.openxmlformats.org/markup-compatibility/2006" xmlns:m = "http://schemas.microsoft.com/office/2004/12/omml" xmlns:mrels = "http://schemas.openxmlformats.org/package/2006/relationships" xmlns:ex12t = "http://schemas.microsoft.com/exchange/services/2006/types" xmlns:ex12m = "http://schemas.microsoft.com/exchange/services/2006/messages"><HEAD><TITLE>CAD losing connection to CRS Server</TITLE>
<META http-equiv=Content-Type content="text/html; charset=utf-8">
<META content="MSHTML 6.00.2900.3243" name=GENERATOR>
<STYLE>
<!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
..MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page Section1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.Section1
{page:Section1;}
-->
</STYLE>
</HEAD>
<BODY lang=EN-US vLink=purple link=blue>
<DIV>We are having a similar problem. I see the following errors from RTMT when the CSR agents disconnect:</DIV>
<DIV> </DIV>
<DIV>Call Manager heartbeat rate below 24 beats per minute.<BR>Current heartbeat rate is 4 beats per minute.</DIV>
<DIV> </DIV>
<DIV>TFTP Server heartbeat rate below 24 beats per minute.<BR>Current heartbeat rate is 9 beats per minute.</DIV>
<DIV> </DIV>
<DIV>From the CAD Log:</DIV>
<DIV> </DIV>
<DIV>
<P class=MsoPlainText style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 8pt; mso-bidi-font-size: 10.0pt; mso-fareast-font-family: 'MS Mincho'"><FONT face="Courier New">16:29:17 02/15/2008 FATAL<SPAN style="mso-spacerun: yes"> </SPAN>FCICD0014 Return: sockRC=<15>, <failed to receive message header from icd server>.<o:p></o:p></FONT></SPAN></P>
<P class=MsoPlainText style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 8pt; mso-bidi-font-size: 10.0pt; mso-fareast-font-family: 'MS Mincho'"><FONT face="Courier New">16:29:17 02/15/2008 FATAL<SPAN style="mso-spacerun: yes"> </SPAN>FCICD0011 ERROR: <16:error in receiving socket message from icd server>.<o:p></o:p></FONT></SPAN></P>
<P class=MsoPlainText style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 8pt; mso-bidi-font-size: 10.0pt; mso-fareast-font-family: 'MS Mincho'"><FONT face="Courier New">16:29:17 02/15/2008 FATAL<SPAN style="mso-spacerun: yes"> </SPAN>PhoneDev PD0151 HeartBeatFailure: ICD offline<o:p></o:p></FONT></SPAN></P>
<P class=MsoPlainText style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 8pt; mso-bidi-font-size: 10.0pt; mso-fareast-font-family: 'MS Mincho'"><FONT face="Courier New">16:29:17 02/15/2008 MINOR<SPAN style="mso-spacerun: yes"> </SPAN>FastCall AW1896 Unable to get RASCAL Global ID for agent state change.<o:p></o:p></FONT></SPAN></P>
<P class=MsoPlainText style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 8pt; mso-bidi-font-size: 10.0pt; mso-fareast-font-family: 'MS Mincho'"><FONT face="Courier New">16:29:17 02/15/2008 MINOR<SPAN style="mso-spacerun: yes"> </SPAN>FastCall AW1933 Unable to write agent state change to RASCAL: GID=0 Error=RASCAL Global ID error<o:p></o:p></FONT></SPAN></P>
<P class=MsoPlainText style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 8pt; mso-bidi-font-size: 10.0pt; mso-fareast-font-family: 'MS Mincho'"><FONT face="Courier New">16:29:17 02/15/2008 MINOR<SPAN style="mso-spacerun: yes"> </SPAN>FastCall AW1952 Unable to write agent state change to RASCAL: GID=0 Error=RASCAL Global ID error<o:p></o:p></FONT></SPAN></P>
<P class=MsoPlainText style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 8pt; mso-bidi-font-size: 10.0pt; mso-fareast-font-family: 'MS Mincho'"><FONT face="Courier New">16:29:17 02/15/2008 INFO<SPAN style="mso-spacerun: yes"> </SPAN>DA0010<SPAN style="mso-spacerun: yes"> </SPAN>Failed to bind to LDAP server. Can't contact LDAP server<o:p></o:p></FONT></SPAN></P>
<P class=MsoPlainText style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 8pt; mso-bidi-font-size: 10.0pt; mso-fareast-font-family: 'MS Mincho'"><FONT face="Courier New">16:29:17 02/15/2008 INFO<SPAN style="mso-spacerun: yes"> </SPAN>DA0010<SPAN style="mso-spacerun: yes"> </SPAN>Failed to bind to LDAP server. Can't contact LDAP server<o:p></o:p></FONT></SPAN></P>
<P class=MsoPlainText style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 8pt; mso-bidi-font-size: 10.0pt; mso-fareast-font-family: 'MS Mincho'"><FONT face="Courier New">16:29:17 02/15/2008 INFO<SPAN style="mso-spacerun: yes"> </SPAN>DAClient<SPAN style="mso-spacerun: yes"> </SPAN>Could not get service FastCall Rascal Server<o:p></o:p></FONT></SPAN></P>
<P class=MsoPlainText style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 8pt; mso-bidi-font-size: 10.0pt; mso-fareast-font-family: 'MS Mincho'"><FONT face="Courier New">16:29:17 02/15/2008 INFO<SPAN style="mso-spacerun: yes"> </SPAN>DA0010<SPAN style="mso-spacerun: yes"> </SPAN>Failed to bind to LDAP server. Can't contact LDAP server<o:p></o:p></FONT></SPAN></P>
<P class=MsoPlainText style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 8pt; mso-bidi-font-size: 10.0pt; mso-fareast-font-family: 'MS Mincho'"><FONT face="Courier New">16:29:17 02/15/2008 INFO<SPAN style="mso-spacerun: yes"> </SPAN>DAClient<SPAN style="mso-spacerun: yes"> </SPAN>Could not get service Chat Server<o:p></o:p></FONT></SPAN></P>
<P class=MsoPlainText style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 8pt; mso-bidi-font-size: 10.0pt; mso-fareast-font-family: 'MS Mincho'"><FONT face="Courier New">16:29:46 02/15/2008 INFO<SPAN style="mso-spacerun: yes"> </SPAN>Enterprise_Data<SPAN style="mso-spacerun: yes"> </SPAN>EA0202 Shutting down<o:p></o:p></FONT></SPAN></P></DIV>
<DIV> </DIV>
<DIV>The CRS server is in the same subnet as this CCM Subscriber server. The Publisher is in a separate subnet across a MAN. We have a sniffer set up on the sub/CRS subnet to try to capture the errors. Couple of questions:</DIV>
<DIV> </DIV>
<DIV>Which CCM server is generating the heartbeats? Is it the Sub to the Pub with the Pub acknowledging or the other way around?</DIV>
<DIV> </DIV>
<DIV>Who is the CRS server exchanging heartbeats with?</DIV>
<DIV> </DIV>
<DIV>Thanks for any advice!</DIV>
<DIV> </DIV>
<DIV>Steve <BR><BR>>>> "Jason Aarons (US)" <jason.aarons@us.didata.com> 2/18/2008 1:21 PM >>><BR></DIV>
<DIV class=Section1>
<P class=MsoNormal><SPAN style="FONT-SIZE: 11pt; COLOR: #1f497d; FONT-FAMILY: 'Calibri','sans-serif'">Any IOS IPS ? I had a similar issue and turning off the IPS resolved it, customer was going to open a TAC case about the IPS.<o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style="FONT-SIZE: 11pt; COLOR: #1f497d; FONT-FAMILY: 'Calibri','sans-serif'"><o:p> </o:p></SPAN></P>
<P class=MsoNormal><SPAN style="FONT-SIZE: 11pt; COLOR: #1f497d; FONT-FAMILY: 'Calibri','sans-serif'">We did extensive troubleshooting with TAC as the problem was intermittent, the IPS turn off was after 2-3 weeks of not being able to resolve the problem, lucky guess on our part.<o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN style="FONT-SIZE: 11pt; COLOR: #1f497d; FONT-FAMILY: 'Calibri','sans-serif'"><o:p> </o:p></SPAN></P>
<DIV>
<DIV style="BORDER-RIGHT: medium none; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; PADDING-LEFT: 0in; PADDING-BOTTOM: 0in; BORDER-LEFT: medium none; PADDING-TOP: 3pt; BORDER-BOTTOM: medium none">
<P class=MsoNormal><B><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: 'Tahoma','sans-serif'">From:</SPAN></B><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: 'Tahoma','sans-serif'"> cisco-voip-bounces@puck.nether.net [mailto:cisco-voip-bounces@puck.nether.net] <B>On Behalf Of </B>Micah Bennett<BR><B>Sent:</B> Monday, February 18, 2008 10:18 AM<BR><B>To:</B> cisco-voip@puck.nether.net<BR><B>Subject:</B> [cisco-voip] CAD losing connection to CRS Server<o:p></o:p></SPAN></P></DIV></DIV>
<P class=MsoNormal><o:p> </o:p></P>
<P><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: 'Arial','sans-serif'">Hello gang. Hope all is well in your parts of the world.</SPAN><o:p></o:p></P>
<P><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: 'Arial','sans-serif'">We are trying to root cause an issue where various users on our IPCC system (IPCC 4.05 HA) are losing connection for a few seconds between the CAD application and the CRS server.</SPAN><o:p></o:p></P>
<P><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: 'Arial','sans-serif'">The bulk of the reports are coming from our extended office call center. This location is about 100 miles from our primary office where the IPCC cluster is located. The remote office is connected via a multi-meg MPLS WAN connection back to our primary office.</SPAN><o:p></o:p></P>
<P><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: 'Arial','sans-serif'">We have not found a pattern yet, and the disconnection only last for a few seconds with the normal CAD message that the connection to the CRS server was lost and reestablished. I have also noticed that at the same time as the CAD disconnection, the clients Microsoft Communicator IM client will lose connection as well. </SPAN><o:p></o:p></P>
<P><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: 'Arial','sans-serif'">Is there some type of</SPAN> <SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: 'Arial','sans-serif'">“keep alive” signal for the CAD/CRS connection that may be getting delayed.</SPAN><o:p></o:p></P>
<P><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: 'Arial','sans-serif'">Thanks again for your advice. </SPAN> <o:p></o:p></P>
<P><A name=""><B><SPAN style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: 'Arial','sans-serif'">Micah Bennett</SPAN></B></A><o:p></o:p></P>
<P><SPAN style="FONT-SIZE: 10pt; COLOR: navy; FONT-FAMILY: 'Arial','sans-serif'">Telecommunications Admin</SPAN><o:p></o:p></P>
<P><B><SPAN style="FONT-SIZE: 10pt; COLOR: maroon; FONT-FAMILY: 'Arial','sans-serif'">Automated License Systems</SPAN></B><o:p></o:p></P>
<TABLE class=MsoNormalTable cellPadding=0 border=0>
<TBODY>
<TR>
<TD style="PADDING-RIGHT: 0.75pt; PADDING-LEFT: 0.75pt; BACKGROUND: white; PADDING-BOTTOM: 0.75pt; PADDING-TOP: 0.75pt">
<P class=MsoNormal><SPAN style="COLOR: black"><BR><BR><BR><BR><BR><BR>The information contained in this email and any attachments is confidential and may be subject to copyright or other intellectual property protection. If you are not the intended recipient, you are not authorized to use or disclose this information, and we request that you notify us by reply mail or telephone and delete the original message from your mail system.</SPAN><o:p></o:p></P></TD></TR></TBODY></TABLE>
<P class=MsoNormal><o:p> </o:p></P></DIV>
<P>
<HR SIZE=1>
<P></P>
<P><STRONG>Disclaimer: This e-mail communication and any attachments may contain confidential and privileged information and is for use by the designated addressee(s) named above only. If you are not the intended addressee, you are hereby notified that you have received this communication in error and that any use or reproduction of this email or its contents is strictly prohibited and may be unlawful. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. </STRONG></P>
<P>************************************<br>
This email may contain privileged and/or confidential information that is intended solely for the use of the addressee. If you are not the intended recipient or entity, you are strictly prohibited from disclosing, copying, distributing or using any of the information contained in the transmission. If you received this communication in error, please contact the sender immediately and destroy the material in its entirety, whether electronic or hard copy. This communication may contain nonpublic personal information about consumers subject to the restrictions of the Gramm-Leach-Bliley Act and the Sarbanes-Oxley Act. You may not directly or indirectly reuse or disclose such information for any purpose other than to provide the services for which you are receiving the information.<br>
There are risks associated with the use of electronic transmission. The sender of this information does not control the method of transmittal or service providers and assumes no duty or obligation for the security, receipt, or third party interception of this transmission.<br>
************************************<br>
</P></BODY></HTML>