[cisco-voip] FW: [RTMT-ALERT-CMPUB-Cluster] CoreDumpFileFound

Mon Jun 27 18:47:27 EDT 2011

Sorry..... I took Wes off line.  here is his response:

Read backtraces from the bottom up:
https://supportforums.cisco.com/docs/DOC-14743

I searched on CQBE_Parser::qbeGet because it was the last function call
before calling a system library.

Regards,
Wes

On Mon, Jun 27, 2011 at 3:10 PM, Justin Steinberg <jsteinberg at gmail.com>wrote:

> Wes,
>
> Using Scott's core backtrace as an example, what are the important parts of
> the backtrace that you would use to form a BTK query ?  Would you just try
> various hex values or filenames?
>
> Thanks.
>
> Justin
>
>
> On Mon, Jun 27, 2011 at 11:54 AM, Wes Sisk <wsisk at cisco.com> wrote:
>
>> **
>> It's just the regular bug toolkit:
>>
>>
>> You have the fix for the first but not the second.  Core in CEF is
>> generally non-impacting to users. It is unfortunately noisy to sys admins.
>>
>> Regards,
>> Wes
>>
>>
>>
>> On 6/27/2011 10:21 AM, Scott Voll wrote:
>>
>> Wes--
>>
>>  How do I get to the "backTrace" part of bug toolkit?
>>
>>  I have another one ;-)
>>
>>  Scott
>>
>>  backtrace
>>  ===================================
>>  #0  0x00ce1eb0 in strncpy () from /lib/tls/libc.so.6
>> #1  0x005f8f38 in OK_BOD_strncpy () from /lib/csa/libcsa.so.6
>> #2  0x005f6570 in strncpy () from /lib/csa/libcsa.so.6
>> #3  0x08107a19 in CQBE_Parser::qbeGet (pMsgBuf_=0xae80ef1f,
>> pParsedMsg_=0xae80ef20, pQbeString_=0x180033, pNextOffset_=0xae80edcc) at
>> qbeparser.cpp:480
>> #4  0x0810a182 in CTIQBE::ProviderOpenCompletedEvent::qbeGet
>> (this=0x8107a19, pMsgBuf_=0xae812ef0, pNextOffset_=0xae80edcc) at
>> qbeparser.cpp:3217
>> #5  0x0810e9aa in CQBE_Parser::qbeParse (this=0xae812ed0,
>> pInputMsg_=0xae812ef0, inputMsgLength_=94, pParsedMsg_=0xae80eed0,
>> parsedMsgBufferSize_=16384) at qbeparser.cpp:6481
>> #6  0x080e6878 in CCtiInterface::receiveLoop (this=0x86aed50) at
>> ctireceiver.cpp:444
>> #7  0x080e64d5 in CCtiReceiver::receiveThreadEntry (pParam=0x86be178) at
>> ctireceiver.cpp:176
>> #8  0x003d8bef in ACE_OS_Thread_Adapter::invoke (this=0x86b7588) at
>> OS_Thread_Adapter.cpp:94
>> #9  0x0039909f in ace_thread_adapter (args=0xae80ef20) at
>> Base_Thread_Adapter.cpp:137
>> #10 0x005053cc in start_thread () from /lib/tls/libpthread.so.0
>> #11 0x00d4396e in clone () from /lib/tls/libc.so.6
>>
>>  Ver 7.1.5
>> On Wed, Jan 27, 2010 at 9:14 AM, Voll, Scott <Scott.Voll at wesd.org> wrote:
>>
>>>  Where do I find this “backtrace” in the bug toolkit?
>>>
>>>
>>>
>>> Here is my backtrace:
>>>
>>>
>>>
>>> #0  0x0032ca92 in strncpy () from /lib/tls/libc.so.6
>>>
>>> #1  0x00e17f38 in OK_BOD_strncpy () from /lib/csa/libcsa.so.6
>>>
>>> #2  0x00e15570 in strncpy () from /lib/csa/libcsa.so.6
>>>
>>> #3  0x080b26eb in CQBE_Parser::qbeGet (pMsgBuf_=0x1756f000,
>>> pParsedMsg_=0x1755e5ec, pQbeString_=0x2e312e36, pNextOffset_=0x1755e46c) at
>>> qbeparser.cpp:410
>>>
>>> #4  0x080b4dc6 in CTIQBE::ProviderOpenCompletedEvent::qbeGet
>>> (this=0x80b26eb, pMsgBuf_=0x175625c0, pNextOffset_=0x1755e46c) at
>>> qbeparser.cpp:2847
>>>
>>> #5  0x080b948c in CQBE_Parser::qbeParse (this=0x175625a0,
>>> pInputMsg_=0x175625c0, inputMsgLength_=89, pParsedMsg_=0x1755e5a0,
>>> parsedMsgBufferSize_=16384) at qbeparser.cpp:5542
>>>
>>> #6  0x08095221 in CCtiInterface::receiveLoop (this=0x0) at
>>> ctireceiver.cpp:428
>>>
>>> #7  0x00000000 in ?? ()
>>>
>>>
>>>
>>> Scott
>>>
>>>
>>>
>>> *From:* Wes Sisk [mailto:wsisk at cisco.com]
>>> *Sent:* Wednesday, January 27, 2010 8:46 AM
>>> *To:* Voll, Scott
>>> *Cc:* Cisco VoIP (cisco-voip at puck.nether.net)
>>> *Subject:* Re: [cisco-voip] FW: [RTMT-ALERT-CMPUB-Cluster]
>>> CoreDumpFileFound
>>>
>>>
>>>
>>> yep, it means your cef process generated a core dump.  i.e. it crashed.
>>> this is the equivalent of windows dr.watson and user.dmp.
>>>
>>> First level diagnosis:
>>> from the CLI use 'utils core list' to identify the core file and then
>>> 'utils core analyze' to generate the backtrace from the core file.  you
>>> *should* be able to search for the backtrace in bug toolkit to find the
>>> relevant defect. if btk is not cooperating then open a tac case and provide
>>> your CM version and the backtrace.  That is adequate to identify the defect
>>> for known crashes.
>>>
>>> Second level diagnosis:
>>> If the backtrace does not match a known signature then we have to
>>> diagnose the core.  That requires:
>>> - the actual cef core file (core files may be very large)
>>> - the backtrace from the core file
>>> - cef traces for 1 hour leading up to the failure
>>> - possibly other traces such as cm SDI/SDL for the same timeframe
>>>
>>> /Wes
>>>
>>>
>>> On Wednesday, January 27, 2010 10:41:41 AM, Voll, Scott
>>> <Scott.Voll at wesd.org> <Scott.Voll at wesd.org> wrote:
>>>
>>>  I’m not finding much on a Google search.  Any idea what this means?
>>>
>>>
>>>
>>> I’m guessing a Cisco Extended Function crashed on my first node, but not
>>> quite sure what needs to be done or if this is a tac case.
>>>
>>>
>>>
>>> From a TAC point of view which RTMT logs should I be getting if I need to
>>> open a ticket?
>>>
>>>
>>>
>>> This is the first time I’ve seen this error.
>>>
>>>
>>>
>>> Thanks
>>>
>>>
>>>
>>> Scott
>>>
>>>
>>>
>>> PS. CM 6.1.4-2000.2
>>>
>>>
>>>
>>> *From:* RTMT_Admin at wesd.org [mailto:RTMT_Admin at wesd.org<RTMT_Admin at wesd.org>]
>>>
>>> *Sent:* Wednesday, January 27, 2010 6:06 AM
>>> *To:* Voll, Scott
>>> *Subject:* [RTMT-ALERT-CMPUB-Cluster] CoreDumpFileFound
>>>
>>>
>>>
>>> CoreDumpFileFound TotalCoresFound : 1 CoreDetails : The following lists
>>> up to 6 cores dumped by corresponding applications. Core1 : Cisco Extended
>>> Functions (core.13530.11.cef.1264601139) AppID : Cisco Log Partition
>>> Monitoring Tool ClusterID : NodeID : CM1 . The alarm is generated on Wed Jan
>>> 27 06:05:44 PST 2010.
>>>
>>>
>>>
>>> ------------------------------
>>>
>>>
>>>
>>> _______________________________________________
>>>
>>> cisco-voip mailing list
>>>
>>> cisco-voip at puck.nether.net
>>>
>>> https://puck.nether.net/mailman/listinfo/cisco-voip
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> cisco-voip mailing list
>>> cisco-voip at puck.nether.net
>>> https://puck.nether.net/mailman/listinfo/cisco-voip
>>>
>>>
>>
>> _______________________________________________
>> cisco-voip mailing list
>> cisco-voip at puck.nether.net
>> https://puck.nether.net/mailman/listinfo/cisco-voip
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/cisco-voip/attachments/20110627/fd401c11/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: moz-screenshot-71.png
Type: image/png
Size: 10555 bytes
Desc: not available
URL: <https://puck.nether.net/pipermail/cisco-voip/attachments/20110627/fd401c11/attachment.png>