[c-nsp] RE: Unknown reason for High CPU 3620
Rodney Dunn
rodunn at cisco.com
Fri Nov 5 08:08:36 EST 2004
I agree this is the way to go. I can't remember
the support story for the ATOM/L2tpv3 story on
the LES (low end systems) (ie: 3620).
You should upgrade it to a 72xx and do as Dave
suggested.
Rodney
On Fri, Nov 05, 2004 at 07:43:40AM -0500, Temkin, David wrote:
> L2tpV3 would work here
>
> > l2tpv3 xconnect l2tpv3 xconnect
> > Firewall ---- eth |7206| atm --------- atm |3620| eth ----- ISP
> (replace with 720x)
>
> Basically, it just replaces the bridging piece. It will crossconnect
> two ethernet segments together, identically to bridging, only it's
> optimized to be handled via CEF.
>
> I've seen it work to 100Mbits+ on smaller (NPE-225/300) platforms.
>
> You would enable IP on the ATM (if it isn't already) and you basically
> route the l2tpv3 tunnel from the first 7206 to the second 720x.
>
> -Dave
>
>
> > -----Original Message-----
> > From: cisco-nsp-bounces at puck.nether.net
> > [mailto:cisco-nsp-bounces at puck.nether.net] On Behalf Of Alex Foster
> > Sent: Friday, November 05, 2004 6:27 AM
> > To: Rodney Dunn
> > Cc: cisco-nsp at puck.nether.net
> > Subject: RE: [c-nsp] RE: Unknown reason for High CPU 3620
> >
> >
> > The Reasons for bridging are explained in the diagram below.
> >
> > Bridging Bridging
> > Firewall ---- eth |7206| atm --------- atm |3620| eth ----- ISP
> >
> > So essentially I am using the ATM network to extend the
> > connection from the ISP to our Firewall. In this situation
> > you can see why I cant route.
> > I cant see how L2TP will be of any help here - comments or
> > suggestions are very welcome - Also the ATM network does not
> > support MPLS (fairly old OS on our ATM switches).
> >
> > Thanks Again
> >
> > Alex
> >
> >
> > -----Original Message-----
> > From: Rodney Dunn [mailto:rodunn at cisco.com]
> > Sent: 04 November 2004 15:42
> > To: Alex Foster
> > Cc: cisco-nsp at puck.nether.net
> > Subject: Re: [c-nsp] RE: Unknown reason for High CPU 3620
> >
> > On Wed, Nov 03, 2004 at 11:46:09PM -0000, Alex Foster wrote:
> > > Hi
> > >
> > > The LANE configuration is used for management only -
> > occassional snmp
> > > queries etc; so the bulk of traffic is being bridged
> > between e1/0 and
> > atm
> > > 0/0.2. Are you saying therefore that bridging is far more intensive
> > than
> > > routing/switching at interrupt level? Why is that (sorry for the
> > question -
> > > just curious).
> >
> > Yes that's correct. Old code and was never designed for high
> > speed switching.
> >
> > >
> > > The load on this connection will soon increase to a constant 40 - 60
> > Mbps
> > > (again bridging - cant route unfortunately). Obviously the
> > 3620 wont
> > handle
> > > this - I have a 7206 (NPE-400) available, would this box suffice for
> > the
> > > traffic load I am estimating - I can upgrade the 7206 to a G1
> > processor if
> > > needs be.
> >
> > It would have to be tested in the lab to know for sure. It's
> > all about processor speed and feature excecution on a per
> > packet basis.
> > The old transparent bridging code isn't a great performer
> > even though it's still under interrupt level. It's a totally
> > different code path than raw packet switching.
> >
> > Someone else mentioned other types of L2 bridging (ATOM/L2TPv3).
> > Those are new technologies designed to carry L2 frames over a
> > L3 transport.
> >
> > It's hard to say what your best solution would be without
> > understanding the overall environement you have. ie: Why are
> > you bridging in the first place?
> >
> >
> > >
> > > Thanks for your time
> > >
> > > Alex
> > >
> > > -----Original Message-----
> > > From: Rodney Dunn [mailto:rodunn at cisco.com]
> > > Sent: 03 November 2004 17:40
> > > To: Alex Foster
> > > Cc: Rodney Dunn; cisco-nsp at puck.nether.net
> > > Subject: Re: [c-nsp] RE: Unknown reason for High CPU 3620
> > >
> > >
> > > Bridging with LANE at that rate. My guess is that would
> > probably be
> > > normal on a 3620.
> > >
> > > Your option would be to get a faster processor or stop bridging.
> > >
> > > To be 100% sure it would need to be set up in the lab with a packet
> > > generator under a controlled test.
> > >
> > > Your sh int stat confirms at least the most of the traffic is being
> > > handled under interrupt level which is good.
> > >
> > >
> > > Rodney
> > >
> > > On Wed, Nov 03, 2004 at 05:19:54PM -0000, Alex Foster wrote:
> > > > Thanks Rodney - find config attached - the collisions were
> > historical -
> > > > the interface is running full-duplex - here is the
> > current interface
> > > > stats - the input errors are being investigated - but
> > shouldn't be
> > > > responsible for the high CPU. Have also included sh vers
> > and sh int
> > > > stat.
> > > >
> > > > #sh int e1/0
> > > > Ethernet1/0 is up, line protocol is up
> > > > Hardware is AmdP2, address is 0005.3245.e070 (bia
> > 0005.3245.e070)
> > > > Description:
> > > > MTU 1500 bytes, BW 10000 Kbit, DLY 1000 usec,
> > > > reliability 255/255, txload 39/255, rxload 39/255
> > > > Encapsulation ARPA, loopback not set
> > > > Keepalive set (10 sec)
> > > > ARP type: ARPA, ARP Timeout 04:00:00
> > > > Last input 00:00:00, output 00:00:00, output hang never
> > > > Last clearing of "show interface" counters 03:03:41
> > > > Queueing strategy: fifo
> > > > Output queue 0/40, 0 drops; input queue 1/75, 790 drops
> > > > 5 minute input rate 1531000 bits/sec, 1935 packets/sec
> > > > 5 minute output rate 1532000 bits/sec, 2170 packets/sec
> > > > 21533985 packets input, 2108844947 bytes, 0 no buffer
> > > > Received 62660 broadcasts, 0 runts, 0 giants, 0 throttles
> > > > 130659 input errors, 1608 CRC, 1272 frame, 0 overrun, 129051
> > > > ignored
> > > > 0 input packets with dribble condition detected
> > > > 23817320 packets output, 2045363543 bytes, 0 underruns(0/0/0)
> > > > 0 output errors, 0 collisions, 0 interface resets
> > > > 0 babbles, 0 late collision, 0 deferred
> > > > 0 lost carrier, 0 no carrier
> > > > 0 output buffer failures, 0 output buffers swapped out
> > > >
> > > > #sh vers
> > > > Cisco Internetwork Operating System Software IOS (tm)
> > 3600 Software
> > > > (C3620-IS-M), Version 12.2(3), RELEASE
> > SOFTWARE
> > > > (fc1)
> > > > Copyright (c) 1986-2001 by cisco Systems, Inc.
> > > > Compiled Wed 18-Jul-01 13:45 by pwade Image text-base:
> > 0x600089A8,
> > > > data-base: 0x6116A000
> > > >
> > > > ROM: System Bootstrap, Version 11.1(20)AA2, EARLY
> > DEPLOYMENT RELEASE
> > > > SOFTWARE (fc1)
> > > >
> > > > xxxxxxxx uptime is 1 year, 29 weeks, 5 days, 1 hour, 48 minutes
> > > > System returned to ROM by power-on System image file is
> > > > "flash:c3620-is-mz.122-3.bin"
> > > >
> > > > cisco 3620 (R4700) processor (revision 0x81) with
> > 41984K/7168K bytes
> > of
> > > > memory.
> > > > Processor board ID 25121827
> > > > R4700 CPU at 80Mhz, Implementation 33, Rev 1.0 Bridging software.
> > > > X.25 software, Version 3.0.0.
> > > > SuperLAT software (copyright 1990 by Meridian Technology Corp).
> > > > 4 Ethernet/IEEE 802.3 interface(s)
> > > > 1 ATM network interface(s)
> > > > DRAM configuration is 32 bits wide with parity disabled.
> > > > 29K bytes of non-volatile configuration memory.
> > > > 16384K bytes of processor board System flash (Read/Write)
> > > >
> > > > Configuration register is 0x2102
> > > >
> > > >
> > > > #sh int stat
> > > > ATM0/0
> > > > Switching path Pkts In Chars In Pkts Out
> > Chars Out
> > > > Processor 60671 5098714 161118
> > 15856770
> > > > Route cache 25347153 2543070267 22778769
> > 2540265463
> > > > Total 25407824 2548168981 22939887
> > 2556122233
> > > > Ethernet1/0
> > > > Switching path Pkts In Chars In Pkts Out
> > Chars Out
> > > > Processor 43533 2674126 7176
> > 430560
> > > > Route cache 22903843 2234233191 25355781
> > 2188907917
> > > > Total 22947376 2236907317 25362957
> > 2189338477
> > > > Interface Ethernet1/1 is disabled
> > > >
> > > > Interface Ethernet1/2 is disabled
> > > >
> > > > Interface Ethernet1/3 is disabled
> > > >
> > > > Building configuration...
> > > >
> > > > Current configuration : 1868 bytes
> > > > !
> > > > version 12.2
> > > > service timestamps debug uptime
> > > > service timestamps log uptime
> > > > service password-encryption
> > > > !
> > > > hostname xxxxxxxxxx
> > > > !
> > > > logging buffered 4096 debugging
> > > > enable secret
> > > > !
> > > > ip subnet-zero
> > > > !
> > > > !
> > > > no ip domain-lookup
> > > > !
> > > > call rsvp-sync
> > > > !
> > > > bridge crb
> > > > !
> > > > interface ATM0/0
> > > > no ip address
> > > > atm uni-version 3.1
> > > > no atm ilmi-keepalive
> > > > pvc 0/5 qsaal
> > > > !
> > > > pvc 0/16 ilmi
> > > > !
> > > > !
> > > > interface ATM0/0.1 multipoint
> > > > ip address xxxxxxx
> > > > lane server-atm-address
> > C5.777777777777777777777777.777777777777.77
> > > > lane client ethernet xxxxxxx
> > > > no cdp enable
> > > > !
> > > > interface ATM0/0.2 multipoint
> > > > pvc 0/103
> > > > encapsulation aal5snap
> > > > !
> > > > bridge-group 1
> > > > !
> > > > interface Ethernet1/0
> > > > description
> > > > no ip address
> > > > no ip mroute-cache
> > > > full-duplex
> > > > no cdp enable
> > > > bridge-group 1
> > > > !
> > > > no ip http server
> > > > !
> > > > no cdp run
> > > > !
> > > > bridge 1 protocol ieee
> > > > !
> > > > dial-peer cor custom
> > > > !
> > > > !
> > > >
> > > > -----Original Message-----
> > > > From: Rodney Dunn [mailto:rodunn at cisco.com]
> > > > Sent: 03 November 2004 16:08
> > > > To: Alex Foster
> > > > Cc: cisco-nsp at puck.nether.net
> > > > Subject: Re: [c-nsp] RE: Unknown reason for High CPU 3620
> > > >
> > > > CPU load at interrupt level is almost always packet
> > > > switching/features or alignment/spurious access corrections.
> > > >
> > > > This CPU sounds high for that amount of traffic.
> > > >
> > > > Can you list the full configuration /*taking out any confidential
> > > > stuff*/ so we can see the features enabled?
> > > >
> > > > 2nd, you really need to clean up that ethernet segement
> > with all the
> > > > collisions. Can you convert it to full-duplex? I don't
> > recall if
> > > > you can do it with the 3620 in later code.
> > > >
> > > > Most likely it's not an issue with the ATM side.
> > > >
> > > > Rodney
> > > >
> > > >
> > > > On Wed, Nov 03, 2004 at 03:44:07PM -0000, Alex Foster wrote:
> > > > > Thanks to the guys who responded - I have managed to narrow down
> > the
> > > > > cause to high interrupt utilization and not processor
> > utilization
> > -
> > > > but
> > > > > Im still a little unclear as to what is causing the
> > high interrupt
> > > > util.
> > > > > - is the ATM interface causing this ? Again the traffic load is
> > > > little
> > > > > over 1.5 meg (in/out) - but constant - so I am wondering if the
> > > > > continual SARing is maybe having a negative effect on the CPU ?
> > If so
> > > > -
> > > > > is there a workaround or am I looking at a bigger platform to
> > solve
> > > > the
> > > > > issue - if so any recommendations (I have a 3640 available but
> > don't
> > > > > know if this will provide much of an improvement).
> > > > >
> > > > >
> > > > >
> > > > > Regards
> > > > >
> > > > >
> > > > >
> > > > > Alex
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > -----Original Message-----
> > > > > From: Alex Foster
> > > > > Sent: 03 November 2004 13:37
> > > > > To: 'cisco-nsp at puck.nether.net'
> > > > > Subject: Unknown reason for High CPU 3620
> > > > >
> > > > >
> > > > >
> > > > > I have a 3620 that is currently maxing out its CPU for
> > no apparent
> > > > > reason - the traffic load on it at the moment is about
> > 1 to 2 meg
> > -
> > > > and
> > > > > sh proc cpu doesn't indicate any process that is
> > causing the high
> > CPU
> > > > > util. Most of the traffic is SIP but I wouldn't have
> > thought this
> > > > would
> > > > > have any bearing. There are only two interfaces on the
> > box in use
> > -
> > > > an
> > > > > ATM 155 and a 10bt. The traffic from the 10bt is
> > bridged to a PVC
> > on
> > > > > the ATM interface - nothing complicated.
> > > > >
> > > > >
> > > > >
> > > > > #sh proc cpu
> > > > >
> > > > > CPU utilization for five seconds: 96%/93%; one minute: 90%; five
> > > > > minutes: 91%
> > > > >
> > > > > PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY
> > Process
> > > > >
> > > > > 1 194708 9899756 19 0.00% 0.00% 0.00%
> > 0
> > > > > Load Meter
> > > > >
> > > > > 2 2008 223 9004 1.41% 0.13%
> > 0.03%
> > > > > 66 Virtual Exec
> > > > >
> > > > >
> > > > >
> > > > > #sh mem sum
> > > > >
> > > > > Head Total(b) Used(b) Free(b)
> > Lowest(b)
> > > > > Largest(b)
> > > > >
> > > > > Processor 61D70EA0 12120416 6616992 5503424
> > 5303744
> > > > > 5305428
> > > > >
> > > > > I/O 2900000 7340032 4387068 2952964
> > 2920520
> > > > > 2947260
> > > > >
> > > > >
> > > > >
> > > > > #sh int e1/0
> > > > >
> > > > > Ethernet1/0 is up, line protocol is up
> > > > >
> > > > > Hardware is AmdP2, address is 0005.3245.e070 (bia
> > 0005.3245.e070)
> > > > >
> > > > > Description:
> > > > >
> > > > > MTU 1500 bytes, BW 10000 Kbit, DLY 1000 usec,
> > > > >
> > > > > reliability 255/255, txload 34/255, rxload 37/255
> > > > >
> > > > > Encapsulation ARPA, loopback not set
> > > > >
> > > > > Keepalive set (10 sec)
> > > > >
> > > > > ARP type: ARPA, ARP Timeout 04:00:00
> > > > >
> > > > > Last input 00:00:00, output 00:00:00, output hang never
> > > > >
> > > > > Last clearing of "show interface" counters 1y4w
> > > > >
> > > > > Queueing strategy: fifo
> > > > >
> > > > > Output queue 0/40, 0 drops; input queue 0/75, 103522 drops
> > > > >
> > > > > 5 minute input rate 1470000 bits/sec, 1984 packets/sec
> > > > >
> > > > > 5 minute output rate 1355000 bits/sec, 2166 packets/sec
> > > > >
> > > > > 2150894411 packets input, 1646909945 bytes, 0 no buffer
> > > > >
> > > > > Received 88151770 broadcasts, 0 runts, 0 giants, 0
> > throttles
> > > > >
> > > > > 6397644 input errors, 42166 CRC, 32878 frame, 0 overrun,
> > 6355422
> > > > > ignored
> > > > >
> > > > > 0 input packets with dribble condition detected
> > > > >
> > > > > 2681876495 packets output, 1769855380 bytes, 0
> > > > > underruns(556991/1663306/0)
> > > > >
> > > > > 165 output errors, 2220297 collisions, 50 interface resets
> > > > >
> > > > > 0 babbles, 47 late collision, 1067254 deferred
> > > > >
> > > > > 118 lost carrier, 0 no carrier
> > > > >
> > > > > 0 output buffer failures, 0 output buffers swapped out
> > > > >
> > > > >
> > > > >
> > > > > #sh int atm 0/0.2
> > > > >
> > > > > ATM0/0.2 is up, line protocol is up
> > > > >
> > > > > Hardware is RS8234 ATMOC3
> > > > >
> > > > > MTU 4470 bytes, BW 155000 Kbit, DLY 80 usec,
> > > > >
> > > > > reliability 255/255, txload 2/255, rxload 2/255
> > > > >
> > > > > Encapsulation ATM
> > > > >
> > > > > 3075506957 packets input, 1094442885139 bytes
> > > > >
> > > > > 2362546968 packets output,988686523562 bytes
> > > > >
> > > > > 0 OAM cells input, 0 OAM cells output
> > > > >
> > > > > AAL5 CRC errors : 81
> > > > >
> > > > > AAL5 SAR Timeouts : 0
> > > > >
> > > > > AAL5 Oversized SDUs : 0
> > > > >
> > > > > AAL5 length violation : 26
> > > > >
> > > > > AAL5 CPI Error : 0
> > > > >
> > > > >
> > > > >
> > > > > Thanks
> > > > >
> > > > >
> > > > >
> > > > > Alex
> > > > >
> > > > >
> > > > >
> > > > > The information in this e-mail and any attachments is
> > confidential
> > and
> > > > may be subject to legal professional privilege. It is intended
> > solely
> > > > for the attention and use of the named addressee(s). If
> > you are not
> > the
> > > > intended recipient, or person responsible for delivering this
> > > > information to the intended recipient, please notify the sender
> > > > immediately. Unless you are the intended recipient or his/her
> > > > representative you are prohibited from, and therefore must not,
> > read,
> > > > copy, distribute, use or retain this message or any part
> > of it. The
> > > > views expressed in this e-mail may not represent those of Gamma
> > Telecom.
> > > > >
> > > > > This message has been scanned for viruses by MailController
> > > > > _______________________________________________
> > > > > cisco-nsp mailing list cisco-nsp at puck.nether.net
> > > > > https://puck.nether.net/mailman/listinfo/cisco-nsp
> > > > > archive at http://puck.nether.net/pipermail/cisco-nsp/
> > > >
> > > >
> > > > This message has been scanned for viruses by MailController -
> > > > www.MailController.altohiway.com
> > >
> > _______________________________________________
> > cisco-nsp mailing list cisco-nsp at puck.nether.net
> > https://puck.nether.net/mailman/listinfo/cisco-nsp
> > archive at http://puck.nether.net/pipermail/cisco-nsp/
> >
> > _______________________________________________
> > cisco-nsp mailing list cisco-nsp at puck.nether.net
> > https://puck.nether.net/mailman/listinfo/cisco-nsp
> > archive at http://puck.nether.net/pipermail/cisco-nsp/
> >
>
>
> IMPORTANT: The information contained in this email and/or its attachments is confidential. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments. Any review, use, reproduction, disclosure or dissemination of this message or any attachment by an unintended recipient is strictly prohibited. Neither this message nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument. Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this message or any of its attachments is free of viruses.
More information about the cisco-nsp
mailing list