Sorry..... I took Wes off line.  here is his response:<div><br></div><div><span class="Apple-style-span" style="border-collapse: collapse; font-family: arial, sans-serif; font-size: 13px; ">Read backtraces from the bottom up:<br>
<a href="https://supportforums.cisco.com/docs/DOC-14743" target="_blank" style="color: rgb(17, 65, 112); ">https://supportforums.cisco.com/docs/DOC-14743</a><br><br>I searched on CQBE_Parser::qbeGet because it was the last function call before calling a system library.<br>
<br>Regards,<br><font color="#888888">Wes</font></span><br><br><div class="gmail_quote">On Mon, Jun 27, 2011 at 3:10 PM, Justin Steinberg <span dir="ltr"><<a href="mailto:jsteinberg@gmail.com">jsteinberg@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">Wes,<br><br>Using Scott's core backtrace as an example, what are the important parts of the backtrace that you would use to form a BTK query ?  Would you just try various hex values or filenames?<br>
<br>Thanks.<br><font color="#888888"><br>
Justin</font><div><div></div><div class="h5"><br><br><div class="gmail_quote">On Mon, Jun 27, 2011 at 11:54 AM, Wes Sisk <span dir="ltr"><<a href="mailto:wsisk@cisco.com" target="_blank">wsisk@cisco.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<u></u>

  
    
  
  <div text="#000000" bgcolor="#ffffff">
    It's just the regular bug toolkit:<br>
    <img src="cid:part1.06070602.06050503@cisco.com" alt=""><br>
    <br>
    You have the fix for the first but not the second.  Core in CEF is
    generally non-impacting to users. It is unfortunately noisy to sys
    admins.<br>
    <br>
    Regards,<br><font color="#888888">
    Wes</font><div><div></div><div><br>
    <br>
    <br>
    On 6/27/2011 10:21 AM, Scott Voll wrote:
    <blockquote type="cite">Wes--
      <div><br>
      </div>
      <div>How do I get to the "backTrace" part of bug toolkit?</div>
      <div><br>
      </div>
      <div>I have another one ;-)</div>
      <div><br>
      </div>
      <div>Scott</div>
      <div><br>
        <div> backtrace</div>
        <div> ===================================</div>
        <div> #0  0x00ce1eb0 in strncpy () from /lib/tls/libc.so.6</div>
        <div>#1  0x005f8f38 in OK_BOD_strncpy () from
          /lib/csa/libcsa.so.6</div>
        <div>#2  0x005f6570 in strncpy () from /lib/csa/libcsa.so.6</div>
        <div>#3  0x08107a19 in CQBE_Parser::qbeGet (pMsgBuf_=0xae80ef1f,
          pParsedMsg_=0xae80ef20, pQbeString_=0x180033,
          pNextOffset_=0xae80edcc) at qbeparser.cpp:480</div>
        <div>#4  0x0810a182 in
          CTIQBE::ProviderOpenCompletedEvent::qbeGet (this=0x8107a19,
          pMsgBuf_=0xae812ef0, pNextOffset_=0xae80edcc) at
          qbeparser.cpp:3217</div>
        <div>#5  0x0810e9aa in CQBE_Parser::qbeParse (this=0xae812ed0,
          pInputMsg_=0xae812ef0, inputMsgLength_=94,
          pParsedMsg_=0xae80eed0, parsedMsgBufferSize_=16384) at
          qbeparser.cpp:6481</div>
        <div>#6  0x080e6878 in CCtiInterface::receiveLoop
          (this=0x86aed50) at ctireceiver.cpp:444</div>
        <div>#7  0x080e64d5 in CCtiReceiver::receiveThreadEntry
          (pParam=0x86be178) at ctireceiver.cpp:176</div>
        <div>#8  0x003d8bef in ACE_OS_Thread_Adapter::invoke
          (this=0x86b7588) at OS_Thread_Adapter.cpp:94</div>
        <div>#9  0x0039909f in ace_thread_adapter (args=0xae80ef20) at
          Base_Thread_Adapter.cpp:137</div>
        <div>#10 0x005053cc in start_thread () from
          /lib/tls/libpthread.so.0</div>
        <div>#11 0x00d4396e in clone () from /lib/tls/libc.so.6</div>
        <div><br>
        </div>
        <div>Ver 7.1.5</div>
        <div class="gmail_quote">On Wed, Jan 27, 2010 at 9:14 AM, Voll,
          Scott <span dir="ltr"><<a href="mailto:Scott.Voll@wesd.org" target="_blank">Scott.Voll@wesd.org</a>></span>
          wrote:<br>
          <blockquote class="gmail_quote" style="margin:0pt 0pt 0pt 0.8ex;border-left:1px solid rgb(204, 204, 204);padding-left:1ex">
            <div bgcolor="white" link="blue" vlink="purple" lang="EN-US">
              <div>
                <p class="MsoNormal"><span style="color:rgb(31, 73, 125)">Where do I find this “backtrace”
                    in the bug toolkit?</span></p>
                <p class="MsoNormal"><span style="color:rgb(31, 73, 125)"> </span></p>
                <p class="MsoNormal"><span style="color:rgb(31, 73, 125)">Here is my backtrace:</span></p>
                <p class="MsoNormal"><span style="color:rgb(31, 73, 125)"> </span></p>
                <p class="MsoNormal"><span style="color:rgb(31, 73, 125)">#0  0x0032ca92 in strncpy ()
                    from /lib/tls/libc.so.6</span></p>
                <p class="MsoNormal"><span style="color:rgb(31, 73, 125)">#1  0x00e17f38 in OK_BOD_strncpy
                    () from /lib/csa/libcsa.so.6</span></p>
                <p class="MsoNormal"><span style="color:rgb(31, 73, 125)">#2  0x00e15570 in strncpy ()
                    from /lib/csa/libcsa.so.6</span></p>
                <p class="MsoNormal"><span style="color:rgb(31, 73, 125)">#3  0x080b26eb in
                    CQBE_Parser::qbeGet (pMsgBuf_=0x1756f000,
                    pParsedMsg_=0x1755e5ec,
                    pQbeString_=0x2e312e36, pNextOffset_=0x1755e46c) at
                    qbeparser.cpp:410</span></p>
                <p class="MsoNormal"><span style="color:rgb(31, 73, 125)">#4  0x080b4dc6 in
                    CTIQBE::ProviderOpenCompletedEvent::qbeGet
                    (this=0x80b26eb, pMsgBuf_=0x175625c0,
                    pNextOffset_=0x1755e46c) at
                    qbeparser.cpp:2847</span></p>
                <p class="MsoNormal"><span style="color:rgb(31, 73, 125)">#5  0x080b948c in
                    CQBE_Parser::qbeParse (this=0x175625a0,
                    pInputMsg_=0x175625c0,
                    inputMsgLength_=89, pParsedMsg_=0x1755e5a0,
                    parsedMsgBufferSize_=16384) at
                    qbeparser.cpp:5542</span></p>
                <p class="MsoNormal"><span style="color:rgb(31, 73, 125)">#<a href="tel:6%C2%A0%200x08095221" value="+16008095221" target="_blank">6  0x08095221</a>
                    in
                    CCtiInterface::receiveLoop (this=0x0) at
                    ctireceiver.cpp:428</span></p>
                <p class="MsoNormal"><span style="color:rgb(31, 73, 125)">#7  0x00000000 in ?? ()</span></p>
                <p class="MsoNormal"><span style="color:rgb(31, 73, 125)"> </span></p>
                <p class="MsoNormal"><span style="color:rgb(31, 73, 125)">Scott</span></p>
                <p class="MsoNormal"><span style="color:rgb(31, 73, 125)"> </span></p>
                <div>
                  <div style="border-right:medium none;border-width:1pt medium medium;border-style:solid none none;border-color:rgb(181, 196, 223) -moz-use-text-color -moz-use-text-color;padding:3pt 0in 0in">
                    <p class="MsoNormal"><b><span style="font-size:10pt;color:windowtext">From:</span></b><span style="font-size:10pt;color:windowtext"> Wes
                        Sisk [mailto:<a href="mailto:wsisk@cisco.com" target="_blank">wsisk@cisco.com</a>]
                        <br>
                        <b>Sent:</b> Wednesday, January 27, 2010 8:46 AM<br>
                        <b>To:</b> Voll, Scott<br>
                        <b>Cc:</b> Cisco VoIP (<a href="mailto:cisco-voip@puck.nether.net" target="_blank">cisco-voip@puck.nether.net</a>)<br>
                        <b>Subject:</b> Re: [cisco-voip] FW:
                        [RTMT-ALERT-CMPUB-Cluster]
                        CoreDumpFileFound</span></p>
                  </div>
                </div>
                <div>
                  <div>
                    <p class="MsoNormal"> </p>
                    <p class="MsoNormal">yep, it means your cef process
                      generated a core dump. 
                      i.e. it crashed.  this is the equivalent of
                      windows dr.watson and
                      user.dmp.<br>
                      <br>
                      First level diagnosis:<br>
                      from the CLI use 'utils core list' to identify the
                      core file and then 'utils
                      core analyze' to generate the backtrace from the
                      core file.  you *should*
                      be able to search for the backtrace in bug toolkit
                      to find the relevant defect.
                      if btk is not cooperating then open a tac case and
                      provide your CM version and
                      the backtrace.  That is adequate to identify the
                      defect for known crashes.<br>
                      <br>
                      Second level diagnosis:<br>
                      If the backtrace does not match a known signature
                      then we have to diagnose the
                      core.  That requires:<br>
                      - the actual cef core file (core files may be very
                      large)<br>
                      - the backtrace from the core file<br>
                      - cef traces for 1 hour leading up to the failure<br>
                      - possibly other traces such as cm SDI/SDL for the
                      same timeframe<br>
                      <br>
                      /Wes<br>
                      <br>
                      <br>
                      On Wednesday, January 27, 2010 10:41:41 AM, Voll,
                      Scott <a href="mailto:Scott.Voll@wesd.org" target="_blank"><Scott.Voll@wesd.org></a>
                      wrote:<br>
                      <br>
                    </p>
                    <p class="MsoNormal"><span style="color:rgb(31, 73, 125)">I’m not finding much on a Google
                        search.  Any idea what this means?</span></p>
                    <p class="MsoNormal"><span style="color:rgb(31, 73, 125)"> </span></p>
                    <p class="MsoNormal"><span style="color:rgb(31, 73, 125)">I’m guessing a Cisco Extended
                        Function crashed on my first node, but not quite
                        sure what needs to be done or
                        if this is a tac case.</span></p>
                    <p class="MsoNormal"><span style="color:rgb(31, 73, 125)"> </span></p>
                    <p class="MsoNormal"><span style="color:rgb(31, 73, 125)">From a TAC point of view which
                        RTMT logs should I be getting if I need to open
                        a ticket?</span></p>
                    <p class="MsoNormal"><span style="color:rgb(31, 73, 125)"> </span></p>
                    <p class="MsoNormal"><span style="color:rgb(31, 73, 125)">This is the first time I’ve seen
                        this error.</span></p>
                    <p class="MsoNormal"><span style="color:rgb(31, 73, 125)"> </span></p>
                    <p class="MsoNormal"><span style="color:rgb(31, 73, 125)">Thanks</span></p>
                    <p class="MsoNormal"><span style="color:rgb(31, 73, 125)"> </span></p>
                    <p class="MsoNormal"><span style="color:rgb(31, 73, 125)">Scott</span></p>
                    <p class="MsoNormal"><span style="color:rgb(31, 73, 125)"> </span></p>
                    <p class="MsoNormal"><span style="color:rgb(31, 73, 125)">PS. CM 6.1.4-2000.2</span></p>
                    <p class="MsoNormal"><span style="color:rgb(31, 73, 125)"> </span></p>
                    <div style="border-right:medium none;border-width:1pt medium medium;border-style:solid none none;padding:3pt 0in 0in;border-color:-moz-use-text-color">
                      <p class="MsoNormal"><b><span style="font-size:10pt">From:</span></b><span style="font-size:10pt"> <a href="mailto:RTMT_Admin@wesd.org" target="_blank">RTMT_Admin@wesd.org</a> [<a href="mailto:RTMT_Admin@wesd.org" target="_blank">mailto:RTMT_Admin@wesd.org</a>]
                          <br>
                          <b>Sent:</b> Wednesday, January 27, 2010 6:06
                          AM<br>
                          <b>To:</b> Voll, Scott<br>
                          <b>Subject:</b> [RTMT-ALERT-CMPUB-Cluster]
                          CoreDumpFileFound</span></p>
                    </div>
                    <p class="MsoNormal"> </p>
                    <p class="MsoNormal"><span style="font-size:12pt;font-family:"Times New Roman","serif"">CoreDumpFileFound
TotalCoresFound
                        : 1 CoreDetails : The following lists up to 6
                        cores dumped by
                        corresponding applications. Core1 : Cisco
                        Extended Functions
                        (core.13530.11.cef.1264601139) AppID : Cisco Log
                        Partition Monitoring Tool
                        ClusterID : NodeID : CM1 . The alarm is
                        generated on Wed Jan 27 06:05:44 PST
                        2010. </span></p>
                    <pre> </pre>
                    <pre style="text-align:center"><hr align="center" size="4" width="90%">

</pre>
                    <pre> </pre>
                    <pre>_______________________________________________</pre>
                    <pre>cisco-voip mailing list</pre>
                    <pre><a href="mailto:cisco-voip@puck.nether.net" target="_blank">cisco-voip@puck.nether.net</a></pre>
                    <pre><a href="https://puck.nether.net/mailman/listinfo/cisco-voip" target="_blank">https://puck.nether.net/mailman/listinfo/cisco-voip</a></pre>
                    <pre>  </pre>
                    <p class="MsoNormal"><span style="font-size:12pt;font-family:"Times New Roman","serif""> </span></p>
                  </div>
                </div>
              </div>
            </div>
            <br>
            _______________________________________________<br>
            cisco-voip mailing list<br>
            <a href="mailto:cisco-voip@puck.nether.net" target="_blank">cisco-voip@puck.nether.net</a><br>
            <a href="https://puck.nether.net/mailman/listinfo/cisco-voip" target="_blank">https://puck.nether.net/mailman/listinfo/cisco-voip</a><br>
            <br>
          </blockquote>
        </div>
        <br>
      </div>
    </blockquote>
  </div></div></div>

<br>_______________________________________________<br>
cisco-voip mailing list<br>
<a href="mailto:cisco-voip@puck.nether.net" target="_blank">cisco-voip@puck.nether.net</a><br>
<a href="https://puck.nether.net/mailman/listinfo/cisco-voip" target="_blank">https://puck.nether.net/mailman/listinfo/cisco-voip</a><br>
<br></blockquote></div><br>
</div></div></blockquote></div><br></div>