[c-nsp] Re: Dropped packets to a specific subnet(Not physical)
Kim Onnel
karim.adel at gmail.com
Sun Feb 26 08:24:27 EST 2006
Whats the command similar to #sh int Fas1/1 in CatOS to show the interface
statistics ?
On 2/25/06, Kim Onnel <karim.adel at gmail.com> wrote:
>
> I am suspecting its not layer 1 because its only happening to hosts in
> this subnet (172.31.10.14), there are other devices on the switch as well,
> who do not suffer from this, so i thought it maybe a routing problem, i
> submitted below the interfaces on the routers and tommorow, i will be able
> to put the switch and firewall.
>
>
>
> WAN_ROUTER#sh int Fa2/1
> FastEthernet2/1 is up, line protocol is up
> Hardware is DEC21140A, address is 0007.ec79.5a90 (bia 0007.ec79.5a90)
> Description: *** LINK TO MCE ***
> MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec,
> reliability 255/255, txload 21/255, rxload 14/255
> Encapsulation 802.1Q Virtual LAN, Vlan ID 1., loopback not set
> Keepalive set (10 sec)
> Full-duplex, 100Mb/s, 100BaseTX/FX
> ARP type: ARPA, ARP Timeout 04:00:00
> Last input 00:00:00, output 00:00:00, output hang never
> Last clearing of "show interface" counters 1d22h
> Input queue: 37/75/689/1592204 (size/max/drops/flushes); Total output
> drops: 0
> Queueing strategy: fifo
> Output queue: 0/40 (size/max)
> 30 second input rate 5554000 bits/sec, 2037 packets/sec
> 30 second output rate 8617000 bits/sec, 2590 packets/sec
> 257175694 packets input, 556239078 bytes
> Received 1469130 broadcasts, 0 runts, 92 giants, 368 throttles
> 1229 input errors, 1229 CRC, 0 frame, 0 overrun, 0 ignored
> 0 watchdog
> 0 input packets with dribble condition detected
> 350477676 packets output, 1216085188 bytes, 255 underruns
> 255 output errors, 255 collisions, 0 interface resets
> 0 babbles, 0 late collision, 0 deferred
> 0 lost carrier, 0 no carrier
> 0 output buffer failures, 0 output buffers swapped out
> WAN_ROUTER#
>
> Please note that WAN_ROUTER is an internet PE in an MPLS cloud of an ISP,
> so there's alot behind it, i can see the underruns, errors, collisions,
> throttles, giants :) but does that explain why certain hosts drop packets.
>
> LAN_ROUTER#sh int Fas 2/1.1
> FastEthernet2/1.1 is up, line protocol is up
> Hardware is DEC21140A, address is 0007.ec79.5ade (bia 0007.ec79.5ade)
> Description: *** Connection with WAN_ROUTER ***
> Internet address is 172.31.10.50/30
> MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec,
> reliability 255/255, txload 16/255, rxload 20/255
> Encapsulation ISL Virtual LAN, Color 10.
> ARP type: ARPA, ARP Timeout 04:00:00
>
> LAN_ROUTER#sh int Fas 2/1
> FastEthernet2/1 is up, line protocol is up
> Hardware is DEC21140A, address is 0007.ec79.5ade (bia 0007.ec79.5ade)
> MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec,
> reliability 255/255, txload 16/255, rxload 20/255
> Encapsulation 802.1Q Virtual LAN, Vlan ID 1., loopback not set
> Keepalive set (10 sec)
> Full-duplex, 100Mb/s, 100BaseTX/FX
> ARP type: ARPA, ARP Timeout 04:00:00
> Last input 00:00:00, output 00:00:00, output hang never
> Last clearing of "show interface" counters never
> Queueing strategy: fifo
> Output queue 0/40, 205 drops; input queue 2/75, 24616 drops
> 30 second input rate 7944000 bits/sec, 2490 packets/sec
> 30 second output rate 6412000 bits/sec, 1953 packets/sec
> 2827954517 packets input, 385489239 bytes
> Received 26651167 broadcasts, 0 runts, 46 giants, 16715 throttles
> 25317 input errors, 25316 CRC, 0 frame, 60 overrun, 9 ignored
> 0 watchdog
> 2 input packets with dribble condition detected
> 1509643285 packets output, 1620877593 bytes, 203179 underruns
> 203179 output errors, 0 collisions, 28 interface resets
> 0 babbles, 0 late collision, 0 deferred
> 0 lost carrier, 0 no carrier
> 0 output buffer failures, 0 output buffers swapped out
> LAN_ROUTER#
>
>
> LAN_ROUTER#sh int Fas 1/1
> FastEthernet1/1 is up, line protocol is up
> Hardware is DEC21140A, address is 0007.ec79.5ace (bia 0007.ec79.5ace)
> Internet address is xx.xx.110.251/26
> MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec,
> reliability 255/255, txload 20/255, rxload 12/255
> Encapsulation ARPA, loopback not set
> Keepalive set (10 sec)
> Full-duplex, 100Mb/s, 100BaseTX/FX
> ARP type: ARPA, ARP Timeout 04:00:00
> Last input 00:00:00, output 00:00:00, output hang never
> Last clearing of "show interface" counters never
> Queueing strategy: fifo
> Output queue 0/40, 0 drops; input queue 0/75, 13594112 drops
> 30 second input rate 5014000 bits/sec, 1927 packets/sec
> 30 second output rate 8072000 bits/sec, 2445 packets/sec
> 1992291019 packets input, 2788861825 bytes
> Received 8284521 broadcasts, 0 runts, 0 giants, 1042424 throttles
> 15 input errors, 0 CRC, 0 frame, 2631 overrun, 315098 ignored
> 0 watchdog
> 0 input packets with dribble condition detected
> 17971387 packets output, 1078666893 bytes, 4881184 underruns
> 4881184 output errors, 0 collisions, 1 interface resets
> 0 babbles, 0 late collision, 0 deferred
> 5 lost carrier, 0 no carrier
> 0 output buffer failures, 0 output buffers swapped out
> LAN_ROUTER#
>
> Also note that i got the above snapshots before clearing the counters, i
> have cleared them and waited for 5 minutes, tried again and there were no
> errors, CRC,..
>
> Please check my problem description, as i said it was working without any
> drops, until i changed the subnet, not the physical connectivity, i'll still
> change the cabling tommorow and submit all interfaces counters all the way.
>
> On 2/23/06, Kim Onnel <karim.adel at gmail.com> wrote:
> >
> > Hi,
> >
> > In our Datacenter, there is a couple of servers in the subnet
> > 172.31.10.0, alot of packets to this subnet are dropped, i attached a
> > network diagram, configurations of the devices in the diagram and ping
> > snapshots and routing information, all other servers in different subnets
> > reply normally.
> >
> > Pinging the Firewall outside interface
> >
> > WAN_ROUTER#ping
> > Protocol [ip]:
> > Target IP address: xx.xx.110.252
> > Repeat count [5]: 100
> > Datagram size [100]: 1500
> > Timeout in seconds [2]:
> > Extended commands [n]:
> > Sweep range of sizes [n]:
> > Type escape sequence to abort.
> > Sending 100, 1500-byte ICMP Echos to xx.xx.110.252, timeout is 2
> > seconds:
> > !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
> > !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
> > Success rate is 100 percent (100/100), round-trip min/avg/max = 1/1/4 ms
> > WAN_ROUTER#
> >
> >
> > Pinging DNS server
> >
> > WAN_ROUTER#ping
> > Protocol [ip]:
> > Target IP address: xx.xx.110.197
> > Repeat count [5]: 100
> > Datagram size [100]: 1500
> > Timeout in seconds [2]:
> > Extended commands [n]:
> > Sweep range of sizes [n]:
> > Type escape sequence to abort.
> > Sending 100, 1500-byte ICMP Echos to xx.xx.110.197, timeout is 2
> > seconds:
> > !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
> > !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
> > Success rate is 100 percent (100/100), round-trip min/avg/max = 1/2/4 ms
> > WAN_ROUTER#
> >
> >
> >
> >
> > WAN_ROUTER#ping
> > Protocol [ip]:
> > Target IP address: 172.31.10.25
> > Repeat count [5]: 10
> > Datagram size [100]: 1500
> > Timeout in seconds [2]:
> > Extended commands [n]:
> > Sweep range of sizes [n]:
> > Type escape sequence to abort.
> > Sending 10, 1500-byte ICMP Echos to 172.31.10.25, timeout is 2 seconds:
> > !.!!!!!.!.
> > Success rate is 70 percent (7/10), round-trip min/avg/max = 1/2/4 ms
> >
> >
> >
> > SSH_SERVER:~# tcpdump icmp
> > tcpdump: verbose output suppressed, use -v or -vv for full protocol
> > decode
> > listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
> > 16:01:52.521633 IP 172.31.10.49 > 172.31.10.25: ICMP echo request, id
> > 26363, seq 0, length 1480
> > 16:01:52.536081 IP 172.31.10.25 > 172.31.10.49: ICMP echo reply, id
> > 26363, seq 0, length 1480
> > 16:01:54.522317 IP 172.31.10.49 > 172.31.10.25: ICMP echo request, id
> > 26363, seq 2, length 1480
> > 16:01:54.522422 IP 172.31.10.25 > 172.31.10.49: ICMP echo reply, id
> > 26363, seq 2, length 1480
> > 16:01:54.524919 IP 172.31.10.49 > 172.31.10.25: ICMP echo request, id
> > 26363, seq 3, length 1480
> > 16:01:54.525017 IP 172.31.10.25 > 172.31.10.49: ICMP echo reply, id
> > 26363, seq 3, length 1480
> > 16:01:54.527720 IP 172.31.10.49 > 172.31.10.25: ICMP echo request, id
> > 26363, seq 4, length 1480
> > 16:01:54.527819 IP 172.31.10.25 > 172.31.10.49: ICMP echo reply, id
> > 26363, seq 4, length 1480
> > 16:01:54.530296 IP 172.31.10.49 > 172.31.10.25: ICMP echo request, id
> > 26363, seq 5, length 1480
> > 16:01:54.530387 IP 172.31.10.25 > 172.31.10.49: ICMP echo reply, id
> > 26363, seq 5, length 1480
> > 16:01:54.533734 IP 172.31.10.49 > 172.31.10.25: ICMP echo request, id
> > 26363, seq 6, length 1480
> > 16:01:54.533832 IP 172.31.10.25 > 172.31.10.49: ICMP echo reply, id
> > 26363, seq 6, length 1480
> > 16:01:54.536519 IP 172.31.10.49 > 172.31.10.25: ICMP echo request, id
> > 26363, seq 7, length 1480
> > 16:01:56.534605 IP 172.31.10.49 > 172.31.10.25: ICMP echo request, id
> > 26363, seq 8, length 1480
> > 16:01:56.534710 IP 172.31.10.25 > 172.31.10.49: ICMP echo reply, id
> > 26363, seq 8, length 1480
> > 16:01:56.537562 IP 172.31.10.49 > 172.31.10.25: ICMP echo request, id
> > 26363, seq 9, length 1480
> > 16:01:56.537658 IP 172.31.10.25 > 172.31.10.49: ICMP echo reply, id
> > 26363, seq 9, length 1480
> >
> >
> >
> > WAN_ROUTER#sh ip route 172.31.10.25
> > Routing entry for 172.31.10.0/27
> > Known via "static", distance 1, metric 0
> > Redistributing via ospf 99, ospf 101, bgp 65000
> > Advertised by ospf 101 subnets
> > bgp 65000
> > Routing Descriptor Blocks:
> > * 172.31.10.50
> > Route metric is 0, traffic share count is 1
> >
> >
> > WAN_ROUTER#sh run | i ip route 172.31.10
> > ip route 172.31.10.0 255.255.255.224 172.31.10.50
> > ip route 172.31.10.29 255.255.255.255 172.31.15.130
> > ip route 172.31.10.32 255.255.255.240 172.31.10.50
> > WAN_ROUTER#
> >
> >
> > WAN_ROUTER#sh ip route | i 172.31.10.
> >
> > C 172.31.10.48/30 is directly connected, FastEthernet2/1.1
> > S 172.31.10.32/28 [1/0] via 172.31.10.50
> > S 172.31.10.29/32 [1/0] via 172.31.15.130
> > S 172.31.10.0/27 [1/0] via 172.31.10.50
> >
> >
> > WAN_ROUTER#sh ip route 172.31.10.50
> > Routing entry for 172.31.10.48/30
> > Known via "connected", distance 0, metric 0 (connected, via interface)
> > Redistributing via bgp 65000
> > Advertised by bgp 65000
> > Routing Descriptor Blocks:
> > * directly connected, via FastEthernet2/1.1
> > Route metric is 0, traffic share count is 1
> >
> >
> >
> > WAN_ROUTER#sh run int FastEthernet2/1
> > Building configuration...
> >
> > Current configuration : 248 bytes
> > !
> > interface FastEthernet2/1
> > description *** LINK TO LAN ROUTER ***
> > no ip address
> > ip access-group block-worms in
> > ip access-group block-worms out
> > ip route-cache flow
> > load-interval 30
> > duplex full
> > no cdp enable
> > end
> > WAN_ROUTER#
> >
> > WAN_ROUTER#sh run int FastEthernet2/1.1
> > !
> > interface FastEthernet2/1.1
> > description *** Connection with LAN ROUTER ***
> > encapsulation isl 10
> > ip address 172.31.10.49 255.255.255.252
> > ip access-group block-worms in
> > ip access-group block-worms out
> > no ip redirects
> > ip ospf hello-interval 5
> > ip ospf dead-interval 15
> > no cdp enable
> > end
> >
> >
> > Moving to LAN_ROUTER:
> >
> > Pinging the SSH server again:
> >
> > LAN_ROUTER#ping
> > Protocol [ip]:
> > Target IP address: 172.31.10.25
> > Repeat count [5]: 10
> > Datagram size [100]: 1500
> > Timeout in seconds [2]:
> > Extended commands [n]:
> > Sweep range of sizes [n]:
> > Type escape sequence to abort.
> > Sending 10, 1500-byte ICMP Echos to 172.31.10.25, timeout is 2 seconds:
> > !!!!!.!!.!
> > Success rate is 80 percent (8/10), round-trip min/avg/max = 1/2/4 ms
> > LAN_ROUTER#
> >
> > zazu:~# tcpdump icmp
> > tcpdump: verbose output suppressed, use -v or -vv for full protocol
> > decode
> > listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
> > 16:10:16.681544 IP xx.xx.110.251 > 172.31.10.25: ICMP echo request, id
> > 5359, seq 2409, length 1480
> > 16:10:16.695414 IP 172.31.10.25 > xx.xx.110.251: ICMP echo reply, id
> > 5359, seq 2409, length 1480
> > 16:10:16.683176 IP xx.xx.110.251 > 172.31.10.25: ICMP echo request, id
> > 5360, seq 2409, length 1480
> > 16:10:16.683277 IP 172.31.10.25 > xx.xx.110.251: ICMP echo reply, id
> > 5360, seq 2409, length 1480
> > 16:10:16.684854 IP xx.xx.110.251 > 172.31.10.25: ICMP echo request, id
> > 5361, seq 2409, length 1480
> > 16:10:16.684985 IP 172.31.10.25 > xx.xx.110.251: ICMP echo reply, id
> > 5361, seq 2409, length 1480
> > 16:10:16.686815 IP xx.xx.110.251 > 172.31.10.25: ICMP echo request, id
> > 5362, seq 2409, length 1480
> > 16:10:16.686931 IP 172.31.10.25 > xx.xx.110.251: ICMP echo reply, id
> > 5362, seq 2409, length 1480
> > 16:10:16.688918 IP xx.xx.110.251 > 172.31.10.25: ICMP echo request, id
> > 5363, seq 2409, length 1480
> > 16:10:16.689004 IP 172.31.10.25 > xx.xx.110.251: ICMP echo reply, id
> > 5363, seq 2409, length 1480
> > 16:10:16.690580 IP xx.xx.110.251 > 172.31.10.25: ICMP echo request, id
> > 5364, seq 2409, length 1480
> > 16:10:18.689945 IP xx.xx.110.251 > 172.31.10.25: ICMP echo request, id
> > 5365, seq 2409, length 1480
> > 16:10:18.690052 IP 172.31.10.25 > xx.xx.110.251: ICMP echo reply, id
> > 5365, seq 2409, length 1480
> > 16:10:18.691900 IP xx.xx.110.251 > 172.31.10.25: ICMP echo request, id
> > 5366, seq 2409, length 1480
> > 16:10:18.691999 IP 172.31.10.25 > xx.xx.110.251: ICMP echo reply, id
> > 5366, seq 2409, length 1480
> > 16:10:18.693729 IP xx.xx.110.251 > 172.31.10.25: ICMP echo request, id
> > 5367, seq 2409, length 1480
> > 16:10:20.693596 IP xx.xx.110.251 > 172.31.10.25: ICMP echo request, id
> > 5368, seq 2409, length 1480
> > 16:10:20.693704 IP 172.31.10.25 > xx.xx.110.251: ICMP echo reply, id
> > 5368, seq 2409, length 1480
> >
> > LAN_ROUTER#sh ip route 172.31.10.25
> > Routing entry for 172.31.10.0/27
> > Known via "static", distance 1, metric 0
> > Routing Descriptor Blocks:
> > * xx.xx.110.252
> > Route metric is 0, traffic share count is 1
> >
> > LAN_ROUTER#sh ip route xx.xx.110.252
> > Routing entry for xx.xx.110.192/26
> > Known via "connected", distance 0, metric 0 (connected, via interface)
> > Routing Descriptor Blocks:
> > * directly connected, via FastEthernet1/1
> > Route metric is 0, traffic share count is 1
> >
> > LAN_ROUTER#sh ip route | i 172.31.10.
> > Gateway of last resort is 172.31.10.49 to network 0.0.0.0
> >
> > C 172.31.10.48/30 is directly connected, FastEthernet2/1.1
> > S 172.31.10.29/32 is directly connected, FastEthernet2/1.1
> > S 172.31.10.0/27 [1/0] via xx.xx.110.252
> > S* 0.0.0.0/0 [1/0] via 172.31.10.49
> > LAN_ROUTER#
> >
> >
> > LAN_ROUTER#sh run int FastEthernet1/1
> >
> > interface FastEthernet1/1
> > .....Alot of secondary IPs omitted....
> > ip address xx.xx.110.3 255.255.255.192 secondary
> > ip address xx.xx.110.251 255.255.255.192
> > ip access-group block-worms in
> > ip access-group block-worms out
> > ip nat inside
> > rate-limit input access-group rate-limit 100 2048000 5000 5000
> > conform-action transmit exceed-action transmit
> > rate-limit output access-group rate-limit 100 2048000 5000 5000
> > conform-action transmit exceed-action transmit
> > load-interval 30
> > full-duplex
> > no cdp enable
> > end
> >
> >
> > LAN_ROUTER#sh access-lists rate-limit 100
> >
> > Rate-limit access list 100
> > 00B0.D064.8774
> >
> > LAN_ROUTER#sh arp | i .252
> > Internet xx.xx.110.252 0 00d0.b782.3aa3 ARPA
> > FastEthernet1/1
> > LAN_ROUTER#sh arp | i 172.31.10.25
> > LAN_ROUTER#sh arp | i 172.31.10.
> > Internet 172.31.10.49 160 0007.ec79.5a90 ARPA
> > FastEthernet2/1.1
> > Internet 172.31.10.50 - 0007.ec79.5ade ARPA
> > FastEthernet2/1.1
> > Internet 172.31.10.29 160 0007.ec79.5a90 ARPA
> > FastEthernet2/1.1
> > LAN_ROUTER#
> >
> >
> > LAN_ROUTER#sh ip access-lists block-worms
> > Extended IP access list block-worms
> > deny tcp any any eq 5554 (21672 matches)
> > deny tcp any any range 135 139 (141805760 matches)
> > deny udp any any range 135 netbios-ss (247404899 matches)
> > deny tcp any any eq 445 (7593990 matches)
> > deny udp any any eq 1026 (196450466 matches)
> >
> >
> >
> >
> > Moving to PIX:
> >
> > : Saved
> > : Written by enable_15 at 11:03:17.364 UTC Tue Feb 14 2006
> >
> > PIX Version 6.3(1)
> >
> > interface ethernet0 100basetx
> > interface ethernet1 100basetx
> > interface ethernet2 100basetx
> > interface ethernet3 100basetx
> > interface ethernet4 100basetx
> > interface ethernet5 auto shutdown
> > interface gb-ethernet0 1000auto
> > interface gb-ethernet1 1000auto
> >
> > nameif ethernet0 collocated security50
> > nameif ethernet1 NOC security60
> > nameif ethernet2 dmz2 security40
> > nameif ethernet3 DMZ3 security55
> > nameif ethernet5 intf5 security10
> > nameif gb-ethernet0 inside security100
> > nameif gb-ethernet1 outside security0
> >
> > hostname PIX-OUTSIDE
> > domain-name ciscopix.com
> >
> > fixup protocol ftp 21
> > fixup protocol ftp 2009
> > fixup protocol ftp 2100
> > fixup protocol ftp 2200
> > fixup protocol ftp 2201
> > fixup protocol ftp 2202
> > fixup protocol ftp 2203
> > fixup protocol ftp 5000
> > fixup protocol h323 h225 1720
> > fixup protocol h323 ras 1718-1719
> > fixup protocol http 80
> > fixup protocol ils 389
> > fixup protocol rsh 514
> > fixup protocol rtsp 554
> > fixup protocol sip 5060
> > fixup protocol sip udp 5060
> > fixup protocol skinny 2000
> > no fixup protocol smtp 25
> > fixup protocol sqlnet 1521
> > names
> >
> >
> >
> > access-list external permit tcp any host 172.31.10.25 eq ssh
> > access-list external permit udp any host 172.31.10.25 eq 2055
> > access-list external permit udp any host 172.31.10.25 eq syslog
> >
> >
> > mtu collocated 1500
> > mtu NOC 1500
> > mtu dmz2 1500
> > mtu DMZ3 1500
> >
> > mtu intf5 1500
> > mtu inside 1500
> > mtu outside 1500
> >
> >
> > ip address NOC 172.31.10.1 255.255.255.224
> > ip address dmz2 10.0.9.1 255.255.255.0
> > ip address DMZ3 10.0.11.1 255.255.255.0
> >
> > no ip address intf5
> >
> > ip address outside xx.xx.110.252 255.255.255.192
> >
> > ip audit info action alarm
> > ip audit attack action alarm
> >
> > ip local pool ippool1 192.168.1.25-192.168.1.28
> >
> > pdm history enable
> > arp timeout 14400
> > nat (netcentrex) 0 access-list 110
> >
> > static (NOC,outside) 172.31.10.0 172.31.10.0 netmask 255.255.255.224 0 0
> >
> > access-group dmz2 in interface dmz2
> > access-group dmz3 in interface DMZ3
> >
> > access-group external in interface outside
> >
> > route outside 0.0.0.0 0.0.0.0 xx.xx.110.251 1
> >
> > timeout xlate 3:00:00
> > timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 rpc 0:10:00 h225
> > 1:00:00
> > timeout h323 0:05:00 mgcp 0:05:00 sip 0:30:00 sip_media 0:02:00
> > timeout uauth 0:05:00 absolute
> >
> > floodguard enable
> > telnet timeout 5
> > ssh timeout 5
> > console timeout 4
> > terminal width 80
> >
> >
> >
> >
> >
> >
> >
> >
> >
>
More information about the cisco-nsp
mailing list