<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div class="">
<blockquote type="cite" class="">
<div class="">#1 0x044a9935 in raise () from /lib/tls/libc.so.6</div>
<div class="">#2 0x044ab399<font color="#ff2600" class=""> in abort () f</font>rom /lib/tls/libc.so.6</div>
<div class="">#3 0x0842e457 in preabort () at ProcessCMProcMon.cpp:80</div>
<div class="">#4 0x0842fe7c in CMProcMon::verifySdlRouterServices () at ProcessCMProcMon.cpp:720</div>
</blockquote>
</div>
<div class=""><br class="">
</div>
<div class="">The ccm process is killing itself because it isn’t getting enough resources.</div>
<br class="">
<blockquote type="cite" class="">Nov 29 17:26:12 CMBL-03-01 local7 2 : 1: CMBL-03-01.localdomain: Nov 29 2018 19:26:12.340 UTC : %UC_CALLMANAGER-2-CallManagerFailure: %[HostName=CMBL-03-01][IPAddress=192.168.183.3][Reason=4][<font color="#ff2600" class="">Text=CCM
Intentional Abort:</font> SignalName: SIPSetupInd, DestPID: SIPD[1:100:67:7]][AppID=Cisco CallManager][ClusterID=StandAloneCluster][NodeID=CMBL-03-01]: Indicates an internal failure in Unified CM</blockquote>
<div class=""><br class="">
</div>
<div class="">So much good info in the syslog.</div>
<div class="">Here’s a super-useful tidbit.</div>
<div class=""><br class="">
</div>
<div class="">
<blockquote type="cite" class="">
<div class="">Nov 28 03:59:23 CMBL-03-01 local7 2 : 1543: CMBL-03-01.localdomain: Nov 28 2018 05:59:23.840 UTC : %UC_RTMT-2-RTMT_ALERT: %[<font color="#ff2600" class="">AlertName=CallProcessingNodeCpuPegging][AlertDetail= Processor load over configured threshold
for configured duration of time .</font> Configured high threshold is 90 % tomcat (2 percent) uses most of the CPU. </div>
<div class=""> Processor_Info: </div>
<div class=""> </div>
<div class=""> For processor instance 1: %CPU= 99, %User= 2, %System= 2, %Nice= 0, %Idle= 0,
<font color="#ff2600" class="">%IOWait= 97</font>, %softirq= 0, %irq= 0. </div>
<div class=""> </div>
<div class=""> For processor instance _Total: %CPU= 93, %User= 2, %System= 1, %Nice= 0, %Idle= 7,
<font color="#ff2600" class="">%IOWait= 90,</font> %softirq= 0, %irq= 0. </div>
<div class=""> </div>
<div class=""> For processor instance 0: %CPU= 86, %User= 2, %System= 1, %Nice= 0, %Idle= 14,
<font color="#ff2600" class="">%IOWait= 83</font>, %softirq= 0, %irq= 0. </div>
<div class=""> </div>
<div class=""> For processor instance 3: %CPU= 87, %User= 2, %System= 2, %Nice= 0, %Idle= 13,
<font color="#ff2600" class="">%IOWait= 83,</font> %softirq= 0, %irq= 0. </div>
<div class=""> </div>
<div class=""> For processor instance 2: %CPU= 99, %User= 4, %System= 1, %Nice= 0, %Idle= 0,
<font color="#ff2600" class="">%IOWait= 96</font>, %softirq= 0, %irq= 0. </div>
<div class=""> ][AppID=Cisco AMC Service][ClusterID=][NodeID=CMBL-03-01]: RTMT Alert</div>
</blockquote>
<br class="">
</div>
<div class="">Looking back just a bit further, and there are a TON of these.</div>
<div class=""><br class="">
</div>
<div class="">
<blockquote type="cite" class="">Nov 15 21:22:00 CMBL-03-01 local7 2 : 582: CMBL-03-01.localdomain: Nov 15 2018 23:22:00.256 UTC : %UC_RTMT-2-RTMT_ALERT: %[<font color="#ff2600" class="">AlertName=HardwareFailure</font>][AlertDetail= At Thu Nov 15 21:22:00
BRST 2018 on node 192.168.183.3, the following HardwareFailure events generated: hwStringMatch : Nov 15 21:21:26 CMBL-03-01 daemon 4 Director Agent:
<font color="#ff2600" class="">LSIESG_DiskDrive_Modified 500605B0027C6D50 Command timeout on PD
</font>01(e0xfc/s1) Path 500000e116ac4ce2, CDB: 2a 00 10 98 b9 9d 00 00 08 00 Sev: 3. AppID : Cisco Syslog Agent ClusterID : NodeID : CMBL-03-01 TimeStamp : Thu Nov 15 21:21:26 BRST 2018 hwStringMatch : Nov 15 21:21:26 CMBL-03-01 daemon 4 Director Agent:
LSIESG_AlertIndication 500605B0027C6D50 Command timeout on PD 01(e0xfc/s1) Path 500000e116ac4ce2, CDB: 2a 00 10 98 b9 9d 00 00 08 00 Sev: 3. AppID : Cisco Syslog Agent ClusterID : NodeID : CMBL-03-01 TimeStamp : Thu Nov 15 21:21:27 BRST 2018 hwStringMatch
: Nov 15 21:21:26 CMBL-03-01][AppID=Cisco AMC Service][ClusterID=][NodeID=CMBL-03-01]: RTMT Alert</blockquote>
</div>
<div class=""><br class="">
</div>
<div class="">You’ve lost or are in the middle of losing at least one disk drive. It probably lost them all at the same time on the 13th and the OS marked the entire filesystem readonly. </div>
<div class=""><br class="">
<div class="">-Ryan </div>
<div><br class="">
<blockquote type="cite" class="">
<div class="">On Dec 3, 2018, at 9:28 AM, Nilson Costa <<a href="mailto:nilsonlino@gmail.com" class="">nilsonlino@gmail.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div dir="ltr" class="">
<div class="">Hello All,</div>
<div class=""><br class="">
</div>
<div class="">I´m deploying a new CUCM on a customer that has an old one working just as call routing for a Genesys system for call center.</div>
<div class=""><br class="">
</div>
<div class="">As you can see the picture below, they have some MGCP Gateways connected to this CUCM where the calls come in and via some CTI route points, controlled by Genesys, route the call to to 2 Avaya PBX or to a another CUCM<br class="">
</div>
<div class=""><br class="">
</div>
<div class="">
<div class="">
<div class=""><span id="cid:ii_jp8eli1x1"><image.png></span><br class="">
</div>
</div>
</div>
<div class="">On november 13th they lost access to Tomcat on the Publisher, when we looked at the server several services were restarting including Cisco CallManager, just on the Publisher.</div>
<div class="">We decided to reboot the whole cluster, but after the reboot we are facing some wierd issues that are not that relevant, I think, but there is one which we are really worried</div>
<div class=""><br class="">
</div>
<div class="">The Cisco CallManager process are still restarting ramdomly and generating some coredumps I´m attaching this logs here also I´m attaching the syslogs from the publisher.</div>
<div class=""><br class="">
</div>
<div class="">Can anybody here on the group help me finding out what is triggering the Cisco CallManager restart?</div>
<div class=""><br class="">
</div>
<div class="">-- <br class="">
<div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature">
<div dir="ltr" class="">Nilson Lino da Costa Junior<br class="">
</div>
</div>
</div>
</div>
<span id="cid:f_jp8erze32"><coredump.txt></span><span id="cid:f_jp8erzej3"><publiser-syslog-29-11.txt></span>_______________________________________________<br class="">
cisco-voip mailing list<br class="">
<a href="mailto:cisco-voip@puck.nether.net" class="">cisco-voip@puck.nether.net</a><br class="">
https://puck.nether.net/mailman/listinfo/cisco-voip<br class="">
</div>
</blockquote>
</div>
<br class="">
</div>
</body>
</html>