[cisco-voip] FW: [RTMT-ALERT-CMPUB-Cluster] CoreDumpFileFound

[Voll, Scott]

Wes-

I feel like a dip shmoe for asking.... But how do I get the:

Core files and CEF traces?

Scott

From: Wes Sisk [mailto:wsisk at cisco.com]
Sent: Wednesday, January 27, 2010 11:00 AM
To: Voll, Scott
Cc: Cisco VoIP (cisco-voip at puck.nether.net)
Subject: Re: [cisco-voip] FW: [RTMT-ALERT-CMPUB-Cluster] CoreDumpFileFound

Good news: you're not the only one seeing this on 6.1.4 which is supposed to already contain the fix
bad news: no root cause yet so more investigation is required

best bet is to open a TAC case with the backtrace, version, core file, and cef traces.  it is possible the fix did not get merged properly. but that is still a discussion that needs to come out of a specific tac case.

/Wes

On Wednesday, January 27, 2010 1:42:15 PM, Voll, Scott <Scott.Voll at wesd.org><mailto:Scott.Voll at wesd.org> wrote:

6.1.4.2000.2

From: Wes Sisk [mailto:wsisk at cisco.com]
Sent: Wednesday, January 27, 2010 10:40 AM
To: Voll, Scott
Cc: Cisco VoIP (cisco-voip at puck.nether.net<mailto:cisco-voip at puck.nether.net>)
Subject: Re: [cisco-voip] FW: [RTMT-ALERT-CMPUB-Cluster] CoreDumpFileFound

you didn't include what version you are running ;)

Looks like
CSCso02381    CEF core due to invalid incoming QBE message header

I just updated the RNE to include the include the backtrace, check bug toolkit in 24-48 hours.

Looks like it's fixed in 6.1(3.3206.1), 6.1(4), and 7.0(1).  What version are you running?

/Wes

On Wednesday, January 27, 2010 12:14:32 PM, Voll, Scott <Scott.Voll at wesd.org><mailto:Scott.Voll at wesd.org> wrote:


Where do I find this "backtrace" in the bug toolkit?

Here is my backtrace:

#0  0x0032ca92 in strncpy () from /lib/tls/libc.so.6
#1  0x00e17f38 in OK_BOD_strncpy () from /lib/csa/libcsa.so.6
#2  0x00e15570 in strncpy () from /lib/csa/libcsa.so.6
#3  0x080b26eb in CQBE_Parser::qbeGet (pMsgBuf_=0x1756f000, pParsedMsg_=0x1755e5ec, pQbeString_=0x2e312e36, pNextOffset_=0x1755e46c) at qbeparser.cpp:410
#4  0x080b4dc6 in CTIQBE::ProviderOpenCompletedEvent::qbeGet (this=0x80b26eb, pMsgBuf_=0x175625c0, pNextOffset_=0x1755e46c) at qbeparser.cpp:2847
#5  0x080b948c in CQBE_Parser::qbeParse (this=0x175625a0, pInputMsg_=0x175625c0, inputMsgLength_=89, pParsedMsg_=0x1755e5a0, parsedMsgBufferSize_=16384) at qbeparser.cpp:5542
#6  0x08095221 in CCtiInterface::receiveLoop (this=0x0) at ctireceiver.cpp:428
#7  0x00000000 in ?? ()

Scott

From: Wes Sisk [mailto:wsisk at cisco.com]
Sent: Wednesday, January 27, 2010 8:46 AM
To: Voll, Scott
Cc: Cisco VoIP (cisco-voip at puck.nether.net<mailto:cisco-voip at puck.nether.net>)
Subject: Re: [cisco-voip] FW: [RTMT-ALERT-CMPUB-Cluster] CoreDumpFileFound

yep, it means your cef process generated a core dump.  i.e. it crashed.  this is the equivalent of windows dr.watson and user.dmp.

First level diagnosis:
from the CLI use 'utils core list' to identify the core file and then 'utils core analyze' to generate the backtrace from the core file.  you *should* be able to search for the backtrace in bug toolkit to find the relevant defect. if btk is not cooperating then open a tac case and provide your CM version and the backtrace.  That is adequate to identify the defect for known crashes.

Second level diagnosis:
If the backtrace does not match a known signature then we have to diagnose the core.  That requires:
- the actual cef core file (core files may be very large)
- the backtrace from the core file
- cef traces for 1 hour leading up to the failure
- possibly other traces such as cm SDI/SDL for the same timeframe

/Wes


On Wednesday, January 27, 2010 10:41:41 AM, Voll, Scott <Scott.Voll at wesd.org><mailto:Scott.Voll at wesd.org> wrote:



I'm not finding much on a Google search.  Any idea what this means?

I'm guessing a Cisco Extended Function crashed on my first node, but not quite sure what needs to be done or if this is a tac case.

>From a TAC point of view which RTMT logs should I be getting if I need to open a ticket?

This is the first time I've seen this error.

Thanks

Scott

PS. CM 6.1.4-2000.2

From: RTMT_Admin at wesd.org<mailto:RTMT_Admin at wesd.org> [mailto:RTMT_Admin at wesd.org]
Sent: Wednesday, January 27, 2010 6:06 AM
To: Voll, Scott
Subject: [RTMT-ALERT-CMPUB-Cluster] CoreDumpFileFound

CoreDumpFileFound TotalCoresFound : 1 CoreDetails : The following lists up to 6 cores dumped by corresponding applications. Core1 : Cisco Extended Functions (core.13530.11.cef.1264601139) AppID : Cisco Log Partition Monitoring Tool ClusterID : NodeID : CM1 . The alarm is generated on Wed Jan 27 06:05:44 PST 2010.










________________________________


















_______________________________________________

cisco-voip mailing list

cisco-voip at puck.nether.net<mailto:cisco-voip at puck.nether.net>

https://puck.nether.net/mailman/listinfo/cisco-voip





-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/cisco-voip/attachments/20100127/f5433181/attachment.html>