<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META http-equiv=Content-Type content="text/html; charset=us-ascii">
<STYLE>
P
{
margin:0px;
padding:0px
}
body
{
FONT-SIZE: 10pt;
FONT-FAMILY:Tahoma
}
</STYLE>
<META content="MSHTML 6.00.2900.3020" name=GENERATOR></HEAD>
<BODY>
<DIV dir=ltr align=left><SPAN class=068225216-25012007>Anthony, </SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=068225216-25012007></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=068225216-25012007>Whilst you wait for the
peak period have a look at the below URL's, they may assist if it does point to
re-assembly </SPAN></DIV>
<DIV dir=ltr align=left> </DIV>
<DIV dir=ltr align=left><A
href="http://www.cisco.com/en/US/tech/tk801/tk703/technologies_tech_note09186a0080094c4f.shtml">http://www.cisco.com/en/US/tech/tk801/tk703/technologies_tech_note09186a0080094c4f.shtml</A></DIV>
<DIV dir=ltr align=left><A
href="http://www.cisco.com/warp/public/105/pmtud_ipfrag.html">http://www.cisco.com/warp/public/105/pmtud_ipfrag.html</A></DIV>
<DIV dir=ltr align=left> </DIV>
<DIV dir=ltr align=left><SPAN class=068225216-25012007>regards</SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=068225216-25012007>paul..</SPAN></DIV>
<DIV dir=ltr align=left> </DIV><BR>
<BLOCKQUOTE
style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #000000 2px solid; MARGIN-RIGHT: 0px">
<DIV class=OutlookMessageHeader lang=en-us dir=ltr align=left>
<HR tabIndex=-1>
<B>From:</B> cisco-bba-bounces@puck.nether.net
[mailto:cisco-bba-bounces@puck.nether.net] <B>On Behalf Of </B>Anthony
Law<BR><B>Sent:</B> Thursday, January 25, 2007 2:25 PM<BR><B>To:</B>
cisco-bba@puck.nether.net<BR><B>Subject:</B> Re: [cisco-bba] need help on
troubleshooting high cpu on 7206NPE300 LNS<BR><BR></DIV>
<DIV></DIV>Hi,<BR> <BR>Thanks for all of your input again. Since this is
just the start of the day, our traffic is low at this
time &<BR> <BR>sh proc cpu is showing<BR> <BR>CPU
utilization for five seconds: 55%/37%; one minute: 55%; five minutes:
56%<BR> 5 484808196
103563445 4681 0.49%
0.64% 0.86% 0 Pool Manager <BR> 37
11481426841072956389 1070 17.50% 17.17%
18.04% 0 IP Input <BR><BR>Below is how >sh ip traffic looks
like<BR> <BR>sh ip traffic<BR>IP statistics:<BR> Rcvd:
674456349 total, 3035990691 local
destination<BR> 9258 format
errors, 3285179 checksum errors, 6694426 bad hop
count<BR> 2 unknown protocol,
159176 not a gateway<BR> 0
security failures, 57 bad options, 293393 with options<BR> Opts: 0
end, 148 nop, 615 basic security, 0 loose source
route<BR> 0 timestamp, 0
extended security, 148 record
route<BR> 0 stream ID, 0
strict source route, 292573 alert, 0 cipso, 0
ump<BR> 0 other<BR>
Frags: 3012940604 reassembled, 3424934 timeouts, 118523 couldn't
reassemble<BR> 2998380890
fragmented, 3205560 couldn't fragment<BR> Bcast: 5550941 received, 3022
sent<BR> Mcast: 0 received, 0 sent<BR> Sent: 302118429
generated, 3616922117 forwarded<BR> Drop: 6396472 encapsulation
failed, 163 unresolved, 0 no
adjacency<BR> 4485 no route, 0
unicast RPF, 4426667 forced drop<BR> Drop: 0 packets with source
IP address zero<BR>ICMP statistics:<BR> Rcvd: 10 format errors, 120
checksum errors, 469 redirects, 11499
unreachable<BR> 3762935 echo, 2838
echo reply, 0 mask requests, 0 mask replies, 5
quench<BR> 0 parameter, 65
timestamp, 1 info request, 225
other<BR> 1 irdp solicitations, 5
irdp advertisements<BR> Sent: 246725 redirects, 3280755 unreachable,
3853 echo, 3762867 echo reply<BR> 0
mask requests, 0 mask replies, 0 quench, 65
timestamp<BR> 1 info reply, 5222083
time exceeded, 3 parameter
problem<BR> 0 irdp solicitations, 0
irdp advertisements<BR>UDP statistics:<BR> Rcvd: 3031423679 total, 53
checksum errors, 5498341 no port<BR> Sent: 289151419 total, 0 forwarded
broadcasts<BR>TCP statistics:<BR> Rcvd: 785273 total, 1727 checksum
errors, 2886 no port<BR> Sent: 450601 total<BR>Probe
statistics:<BR> Rcvd: 0 address requests, 0 address
replies<BR> 0 proxy name requests, 0
where-is requests, 0 other<BR> Sent: 0 address requests, 0 address
replies (0 proxy)<BR> 0 proxy name
replies, 0 where-is replies<BR>BGP statistics:<BR> Rcvd: 0 total, 0
opens, 0 notifications, 0
updates<BR> 0 keepalives, 0
route-refresh, 0 unrecognized<BR> Sent: 0 total, 0 opens, 0
notifications, 0 updates<BR> 0
keepalives, 0 route-refresh<BR>EGP statistics:<BR> Rcvd: 0 total, 0
format errors, 0 checksum errors, 0 no listener<BR> Sent: 0
total<BR>IGRP statistics:<BR> Rcvd: 0 total, 0 checksum errors<BR>
Sent: 0 total<BR>OSPF statistics:<BR> Rcvd: 0 total, 0 checksum
errors<BR> 0 hello, 0 database desc,
0 link state req<BR> 0 link state
updates, 0 link state acks<BR> Sent: 0 total<BR>IP-IGRP2
statistics:<BR> Rcvd: 0 total<BR> Sent: 0 total<BR>PIMv2
statistics: Sent/Received<BR> Total: 0/0, 0 checksum errors, 0 format
errors<BR> Registers: 0/0, Register Stops: 0/0, Hellos:
0/0<BR> Join/Prunes: 0/0, Asserts: 0/0, grafts: 0/0<BR>
Bootstraps: 0/0, Candidate_RP_Advertisements: 0/0<BR> State-Refresh:
0/0<BR>IGMP statistics: Sent/Received<BR> Total: 0/0, Format errors:
0/0, Checksum errors: 0/0<BR> Host Queries: 0/0, Host Reports: 0/0, Host
Leaves: 0/0 <BR> DVMRP: 0/0, PIM: 0/0<BR>ARP statistics:<BR> Rcvd:
15597477 requests, 294820 replies, 0 reverse, 0 other<BR> Sent: 4637290
requests, 27974487 replies (1776972 proxy), 0 reverse<BR><BR>><FONT
face=Arial>Are still users connected which received a framed-compression
attribute before you made the change?</FONT><BR>
<DIV dir=ltr align=left><SPAN
class=EC_600440108-25012007></SPAN> </DIV>After making changes to our
radius. I have reset all tunnels therefore bumped off everyone from their vpdn
sess & I have verified<BR> that they are not receiving "compression"
anymore<BR> <BR>I'll post some more stats during the peak
period.<BR> <BR>Thanks.<BR> <BR>Anthony<BR><BR>
<BLOCKQUOTE
style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #008080 2px solid; MARGIN-RIGHT: 0px">
<HR id=EC_stopSpelling>
Subject: RE: [cisco-bba] need help on troubleshooting high cpu on 7206NPE300
LNS<BR>Date: Thu, 25 Jan 2007 10:13:20 +0100<BR>From:
oboehmer@cisco.com<BR>To: ariev@vayner.net; antnada@hotmail.com;
cisco-bba@puck.nether.net<BR><BR>
<META content="Microsoft SafeHTML" name=Generator>
<DIV dir=ltr align=left><SPAN class=EC_600440108-25012007><FONT
face=Arial>Arie,</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=EC_600440108-25012007><FONT
face=Arial></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=EC_600440108-25012007><FONT
face=Arial>encapsulating/decapsulating L2TP packets should not happen in IP
Input process, this is done in the interrupt path</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=EC_600440108-25012007><FONT
face=Arial></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=EC_600440108-25012007><FONT
face=Arial>Anthony: S</FONT></SPAN><SPAN
class=EC_600440108-25012007><FONT face=Arial>omething is preventing your
interfaces from interrupt-switching the traffic. Another possibility is
packet re-assembly (which would be shown in "show ip traffic", as Paul just
suggested). Do a "clear counter" and then check "show int stat" which
interface(s) send the majority of pkts in the process path. Are still users
connected which received a framed-compression attribute before you made the
change?</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=EC_600440108-25012007><FONT
face=Arial></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=EC_600440108-25012007>
<FONT face=Arial>oli</FONT></SPAN></DIV><BR>
<BLOCKQUOTE
style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #000000 2px solid; MARGIN-RIGHT: 0px">
<DIV class=EC_OutlookMessageHeader lang=en-us dir=ltr align=left>
<HR>
<FONT face=Tahoma><B>From:</B> cisco-bba-bounces@puck.nether.net
[mailto:cisco-bba-bounces@puck.nether.net] <B>On Behalf Of </B>Arie
Vayner<BR><B>Sent:</B> Thursday, January 25, 2007 8:38 AM<BR><B>To:</B>
Anthony Law; cisco-bba@puck.nether.net<BR><B>Subject:</B> Re: [cisco-bba]
need help on troubleshooting high cpu on 7206NPE300
LNS<BR></FONT><BR></DIV>
<DIV></DIV><BR><BR>
<DIV><SPAN class=EC_gmail_quote>On 1/25/07, <B
class=EC_gmail_sendername>Arie Vayner</B> <<A
href="mailto:ariev@vayner.net">ariev@vayner.net</A>> wrote:</SPAN>
<BLOCKQUOTE class=EC_gmail_quote
style="PADDING-LEFT: 1ex; BORDER-LEFT: rgb(204,204,204) 1px solid">Anthony,<BR><BR>The
high CPU on IP Input is normal, as this is where the L2TP work is being
done.<BR>Also note that you have a high rate of CPU being used in
Interrupts (91%/44% means that 44% is used for Interrupts). Interrupts
on Cisco routers are usually linked directly to a high rate of traffic
(on centralized CPU devices). <BR>I would assume you box is very close
to its limit of how much traffic it can handle. Could you please send
some of the "show interface" outputs (for the FastEthernet/GigE/ATM
ports you might have). This would allow us to get a better estimation.
<BR><BR>You need to take into account that this is a centralized CPU
platform, and all traffic is handled by the CPU. This means that the
scale factor is not only a question of how many sessions you have
concurrently, but also how much traffic (mostly in PPS and not BPS) they
transmit. <BR><BR>Thanks<BR><SPAN class=EC_sg>Arie</SPAN>
<DIV><SPAN class=EC_e id=EC_q_110583013859d0e2_2><BR><BR>
<DIV><SPAN class=EC_gmail_quote>On 1/25/07, <B
class=EC_gmail_sendername>Anthony Law</B> <<A
href="mailto:antnada@hotmail.com"> antnada@hotmail.com</A>>
wrote:</SPAN>
<BLOCKQUOTE class=EC_gmail_quote
style="PADDING-LEFT: 1ex; BORDER-LEFT: rgb(204,204,204) 1px solid">
<DIV><BR>Dear all<BR> <BR>Thank you for all of your input. I
configured vpdn ip udp ignore checksum<BR>& I have corrected
a mis-config on our radius server (passing compression attribute
to cisco) now that the L2TP data daemon is running normal, but I am
still facing high cpu on Pool Manager & IP Input<BR>anymore
suggestions?<BR> <BR> <BR>CPU utilization for five seconds:
91%/44%; one minute: 91%; five minutes: 86%<BR> PID
Runtime(ms) Invoked
uSecs 5Sec 1Min 5Min TTY Process
<BR>
1
4
175 22
0.00% 0.00% 0.00% 0 Chunk
Manager <BR>
2 487964
5014024 97
0.00% 0.00% 0.00% 0 Load
Meter <BR>
3 1606476
870141 1846 0.00%
0.00% 0.00% 0 CEF
Scanner <BR>
4 22428792
3318958 6757 0.00%
0.06% 0.05% 0 Check
heaps <BR> 5
481842360 102963163 4679
9.05% 9.70% 7.90% 0 Pool
Manager <BR> 37
11275060121049358292 1074 36.02%
35.07% 32.40% 0 IP Input <BR><BR>Thank
You<BR> <BR>Anthony<BR><BR><BR><BR>
<HR>
<BR>> Date: Wed, 24 Jan 2007 02:37:10 +0200<BR>> From: <A
href="mailto:nitzan.tzelniker@gmail.com">nitzan.tzelniker@gmail.com</A><BR>>
To: <A
href="mailto:antnada@hotmail.com">antnada@hotmail.com</A><BR>>
Subject: Re: [cisco-bba] need help on troubleshooting high cpu on 7206
NPE300 LNS<BR>> CC: <A
href="mailto:cisco-bba@puck.nether.net">cisco-bba@puck.nether.net</A><BR>>
<BR>> You can try<BR>> <BR>> vpdn ip udp ignore
checksum<BR>> <BR>> Nitzan<BR>> <BR>> On 1/24/07, Anthony
Law <<A href="mailto:antnada@hotmail.com">
antnada@hotmail.com</A>> wrote:<BR>> > Dear all,<BR>>
><BR>> > We have a 7206 w/NPE300 running as a LNS terminating
pppoe sessions from our<BR>> > telco. We are concurrently
running around 360 pppoe sessions. <BR>> ><BR>> >
Recently. I noticed that our 7206 is having extremely high cpu, at
times<BR>> > going to 100%, please see below<BR>>
><BR>> > CPU utilization for five seconds: 99%/42%; one
minute: 99%; five minutes: <BR>> > 99%<BR>> > PID
Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process<BR>> > 1 0
75 0 0.00% 0.00% 0.00% 0 Chunk Manager<BR>> ><BR>>
><BR>> > 5 472509060 101324023 4663 7.65% 8.80% 8.84% 0 Pool
Manager<BR>> ><BR>> > 37 10810547881019294234 1060 22.79%
25.16% 25.51% 0 IP Input<BR>> ><BR>> ><BR>> > 101
705044020 800103660 881 18.89% 21.35% 19.34% 0 L2TP data<BR>> >
daemon <BR>> > 102 53153196 10197928 5212 2.19% 0.46% 0.45% 0
L2TP mgmt<BR>> > daemon<BR>> ><BR>> ><BR>> >
It seemed that Pool Manager + IP Input + L2TP data daemon together
is<BR>> > causing this issue. I was searching for documents
regarding this on google <BR>> > and came to this mailing list.
I am wondering if you guys can help me out by<BR>> > identifying
the mis-configuration that I have on my end as it is my<BR>> >
understanding that a 7206 should at least take close 1000 pppoe
sessions. <BR>> > Thank You in advance for your input.<BR>>
><BR>> ><BR>> > hostname LNS<BR>> > !<BR>>
> boot system slot1:c7200-is-mz.122-32.bin<BR>> > boot system
slot1:c7200-is-mz.120-3.T3 <BR>> > aaa new-model<BR>> >
aaa authentication login default local<BR>> > aaa authentication
login no_rad line<BR>> > aaa authentication ppp default group
radius local<BR>> > aaa authentication ppp vpdn group radius
<BR>> > aaa authorization network default group radius<BR>>
> aaa authorization configuration default group radius<BR>> >
aaa accounting delay-start<BR>> > aaa accounting exec default
start-stop group radius <BR>> > aaa accounting network default
start-stop group radius<BR>> > enable secret 5
XXXXXXXXXXXXXXXXXXXXXXXXXXX<BR>> > !<BR>> > clock timezone
EST -5<BR>> > clock summer-time EDT recurring <BR>> > ip
subnet-zero<BR>> > no ip source-route<BR>> > ip
cef<BR>> > !<BR>> > !<BR>> > ip name-server
XXXXXX<BR>> > ip name-server XXXXXX<BR>> > ip name-server
XXXXXX<BR>> > ! <BR>> > vpdn enable<BR>> > !<BR>>
> vpdn-group XXXXXXXX<BR>> > accept-dialin<BR>> >
protocol l2tp<BR>> > virtual-template 1<BR>> >
terminate-from hostname XXXXXX<BR>> > local name XXXXXXX
<BR>> > lcp renegotiation always<BR>> > !<BR>> >
interface FastEthernet0/0<BR>> > ip address X.X.X.X <A
href="http://255.255.255.192/"
target=_blank>255.255.255.192</A><BR>> > no ip
mroute-cache<BR>> > duplex full<BR>> > !<BR>> >
interface FastEthernet1/0<BR>> > no ip address<BR>> > no
ip mroute-cache<BR>> > duplex full<BR>> > ! <BR>> >
interface FastEthernet1/0.401<BR>> > description
!!XXXXXXXXXXXXXXXXXXXXXXXX!!<BR>> > encapsulation dot1Q
401<BR>> > ip address 10.70.X.X <A
href="http://255.255.255.252/"
target=_blank>255.255.255.252</A><BR>> > no ip
mroute-cache<BR>> > !<BR>> > interface
FastEthernet2/0<BR>> > description !!Internet Feed!!<BR>>
> ip address Y.Y.Y.Y <A href="http://255.255.255.252/"
target=_blank>255.255.255.252</A><BR>> > no ip
mroute-cache<BR>> > duplex full<BR>> > !<BR>> >
interface Virtual-Template1<BR>> > mtu 1492<BR>> > ip
unnumbered FastEthernet2/0<BR>> > peer default ip address pool
internet1 internet2 <BR>> > ppp authentication pap vpdn<BR>>
> !<BR>> > ip local pool internet1 A.A.A.A B.B.B.B<BR>>
> ip local pool internet2 C.C.C.C D.D.D.D<BR>> > ip
classless<BR>> > ip route <A href="http://0.0.0.0/"
target=_blank>0.0.0.0</A> <A href="http://0.0.0.0/"
target=_blank>0.0.0.0</A> Y.Y.Y.Y<BR>> > no ip http
server<BR>> > !<BR>> > ip radius source-interface
FastEthernet0/0 <BR>> > radius-server host X.X.X.X auth-port
1645 acct-port 1646<BR>> > radius-server host X.X.X.X auth-port
1645 acct-port 1646<BR>> > radius-server key 7
ZZZZZZZZZZZZZZZ<BR>> ><BR>> > Anthony <BR>>
><BR>> > ________________________________<BR>> > Be one
of the first to try Windows Live Mail.<BR>> >
_______________________________________________<BR>> > cisco-bba
mailing list<BR>> > <A
href="mailto:cisco-bba@puck.nether.net">cisco-bba@puck.nether.net</A><BR>>
> <A href="https://puck.nether.net/mailman/listinfo/cisco-bba"
target=_blank>https://puck.nether.net/mailman/listinfo/cisco-bba</A><BR>>
><BR>> ><BR>> ><BR><BR><BR>
<HR>
Be one of the first to try <A
href="http://ideas.live.com/programpage.aspx?versionId=5d21c51a-b161-4314-9b0e-4911fb2b2e6d"
target=_blank>Windows Live
Mail.</A></DIV><BR>_______________________________________________<BR>cisco-bba
mailing list<BR><A
href="mailto:cisco-bba@puck.nether.net">cisco-bba@puck.nether.net
</A><BR><A href="https://puck.nether.net/mailman/listinfo/cisco-bba"
target=_blank>https://puck.nether.net/mailman/listinfo/cisco-bba</A><BR><BR><BR></BLOCKQUOTE></DIV><BR></SPAN></DIV></BLOCKQUOTE></DIV><BR></BLOCKQUOTE></BLOCKQUOTE><BR>
<HR>
Be one of the first to try <A
href="http://ideas.live.com/programpage.aspx?versionId=5d21c51a-b161-4314-9b0e-4911fb2b2e6d"
target=_new>Windows Live Mail.</A> </BLOCKQUOTE></BODY></HTML>