[cisco-voip] FW: [RTMT-ALERT-CMPUB-Cluster] CoreDumpFileFound

Wed Jan 27 14:40:06 EST 2010

RTMT check boxes:
trace and log central
collect files
"select ccm services/applications" -> cisco extended functions, cisco 
extended functions report
"select system services/applications" -> cisco corefile recovery tool

/Wes

On Wednesday, January 27, 2010 2:17:24 PM, Voll, Scott 
<Scott.Voll at wesd.org> wrote:
>
> Wes---
>
>  
>
> I feel like a dip shmoe for asking.... But how do I get the:
>
>  
>
> Core files and CEF traces?
>
>  
>
> Scott
>
>  
>
> *From:* Wes Sisk [mailto:wsisk at cisco.com]
> *Sent:* Wednesday, January 27, 2010 11:00 AM
> *To:* Voll, Scott
> *Cc:* Cisco VoIP (cisco-voip at puck.nether.net)
> *Subject:* Re: [cisco-voip] FW: [RTMT-ALERT-CMPUB-Cluster] 
> CoreDumpFileFound
>
>  
>
> Good news: you're not the only one seeing this on 6.1.4 which is 
> supposed to already contain the fix
> bad news: no root cause yet so more investigation is required
>
> best bet is to open a TAC case with the backtrace, version, core file, 
> and cef traces.  it is possible the fix did not get merged properly. 
> but that is still a discussion that needs to come out of a specific 
> tac case.
>
> /Wes
>
> On Wednesday, January 27, 2010 1:42:15 PM, Voll, Scott 
> <Scott.Voll at wesd.org> <mailto:Scott.Voll at wesd.org> wrote:
>
> 6.1.4.2000.2
>
>  
>
> *From:* Wes Sisk [mailto:wsisk at cisco.com]
> *Sent:* Wednesday, January 27, 2010 10:40 AM
> *To:* Voll, Scott
> *Cc:* Cisco VoIP (cisco-voip at puck.nether.net 
> <mailto:cisco-voip at puck.nether.net>)
> *Subject:* Re: [cisco-voip] FW: [RTMT-ALERT-CMPUB-Cluster] 
> CoreDumpFileFound
>
>  
>
> you didn't include what version you are running ;)
>
> Looks like
> CSCso02381    CEF core due to invalid incoming QBE message header
>
> I just updated the RNE to include the include the backtrace, check bug 
> toolkit in 24-48 hours.
>
> Looks like it's fixed in 6.1(3.3206.1), 6.1(4), and 7.0(1).  What 
> version are you running?
>
> /Wes
>
> On Wednesday, January 27, 2010 12:14:32 PM, Voll, Scott 
> <Scott.Voll at wesd.org> <mailto:Scott.Voll at wesd.org> wrote:
>
>
> Where do I find this "backtrace" in the bug toolkit?
>
>  
>
> Here is my backtrace:
>
>  
>
> #0  0x0032ca92 in strncpy () from /lib/tls/libc.so.6
>
> #1  0x00e17f38 in OK_BOD_strncpy () from /lib/csa/libcsa.so.6
>
> #2  0x00e15570 in strncpy () from /lib/csa/libcsa.so.6
>
> #3  0x080b26eb in CQBE_Parser::qbeGet (pMsgBuf_=0x1756f000, 
> pParsedMsg_=0x1755e5ec, pQbeString_=0x2e312e36, 
> pNextOffset_=0x1755e46c) at qbeparser.cpp:410
>
> #4  0x080b4dc6 in CTIQBE::ProviderOpenCompletedEvent::qbeGet 
> (this=0x80b26eb, pMsgBuf_=0x175625c0, pNextOffset_=0x1755e46c) at 
> qbeparser.cpp:2847
>
> #5  0x080b948c in CQBE_Parser::qbeParse (this=0x175625a0, 
> pInputMsg_=0x175625c0, inputMsgLength_=89, pParsedMsg_=0x1755e5a0, 
> parsedMsgBufferSize_=16384) at qbeparser.cpp:5542
>
> #6  0x08095221 in CCtiInterface::receiveLoop (this=0x0) at 
> ctireceiver.cpp:428
>
> #7  0x00000000 in ?? ()
>
>  
>
> Scott
>
>  
>
> *From:* Wes Sisk [mailto:wsisk at cisco.com]
> *Sent:* Wednesday, January 27, 2010 8:46 AM
> *To:* Voll, Scott
> *Cc:* Cisco VoIP (cisco-voip at puck.nether.net 
> <mailto:cisco-voip at puck.nether.net>)
> *Subject:* Re: [cisco-voip] FW: [RTMT-ALERT-CMPUB-Cluster] 
> CoreDumpFileFound
>
>  
>
> yep, it means your cef process generated a core dump.  i.e. it 
> crashed.  this is the equivalent of windows dr.watson and user.dmp.
>
> First level diagnosis:
> from the CLI use 'utils core list' to identify the core file and then 
> 'utils core analyze' to generate the backtrace from the core file.  
> you *should* be able to search for the backtrace in bug toolkit to 
> find the relevant defect. if btk is not cooperating then open a tac 
> case and provide your CM version and the backtrace.  That is adequate 
> to identify the defect for known crashes.
>
> Second level diagnosis:
> If the backtrace does not match a known signature then we have to 
> diagnose the core.  That requires:
> - the actual cef core file (core files may be very large)
> - the backtrace from the core file
> - cef traces for 1 hour leading up to the failure
> - possibly other traces such as cm SDI/SDL for the same timeframe
>
> /Wes
>
>
> On Wednesday, January 27, 2010 10:41:41 AM, Voll, Scott 
> <Scott.Voll at wesd.org> <mailto:Scott.Voll at wesd.org> wrote:
>
>
>
> I'm not finding much on a Google search.  Any idea what this means?
>
>  
>
> I'm guessing a Cisco Extended Function crashed on my first node, but 
> not quite sure what needs to be done or if this is a tac case.
>
>  
>
> From a TAC point of view which RTMT logs should I be getting if I need 
> to open a ticket?
>
>  
>
> This is the first time I've seen this error.
>
>  
>
> Thanks
>
>  
>
> Scott
>
>  
>
> PS. CM 6.1.4-2000.2
>
>  
>
> *From:* RTMT_Admin at wesd.org <mailto:RTMT_Admin at wesd.org> 
> [mailto:RTMT_Admin at wesd.org]
> *Sent:* Wednesday, January 27, 2010 6:06 AM
> *To:* Voll, Scott
> *Subject:* [RTMT-ALERT-CMPUB-Cluster] CoreDumpFileFound
>
>  
>
> CoreDumpFileFound TotalCoresFound : 1 CoreDetails : The following 
> lists up to 6 cores dumped by corresponding applications. Core1 : 
> Cisco Extended Functions (core.13530.11.cef.1264601139) AppID : Cisco 
> Log Partition Monitoring Tool ClusterID : NodeID : CM1 . The alarm is 
> generated on Wed Jan 27 06:05:44 PST 2010.
>
>  
>  
>  
> ------------------------------------------------------------------------
>
>
>   
>  
>  
>   
>  
>  
>   
>  
> _______________________________________________
> cisco-voip mailing list
> cisco-voip at puck.nether.net <mailto:cisco-voip at puck.nether.net>
> https://puck.nether.net/mailman/listinfo/cisco-voip
>   
>
>  
>
>  
>
>  
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/cisco-voip/attachments/20100127/3128bf44/attachment.html>