[c-nsp] [Re: huge amount of weird traffic on poin-to-point ethernet link]

[Stephen Wilcox]

Hi James,
 Primoz says he has already done that... he wants to find the source.

As he is using BGP and has no default tho, he would be best to route 0.0.0.0/0 to null0 which will catch any stray packets not just RFC1918 (altho in the absence of any default, packets should be getting dropped anyway)

Also, routing the private addresses to a sniffer will not reveal the MAC as it would simply show the MAC of the router.


Primoz - why not put an access list on the interface, match these packets and specify log-input which will show you the source interface and MAC. Be careful tho, you said your link ran at 100% before, so keep your null route in place to ensure your logging doesnt hog the cpu.

eg

access-l 100 deny ip any 172.16.0.0 0.0.15.255 log-input
access-l 100 deny ip 172.16.0.0 0.0.15.255 any log-input
access-l 100 permit ip any any

int fa x/y
 ip access-g 100 in


HTH

Steve


On Thu, Nov 09, 2006 at 12:03:37PM -0000, James Worley wrote:
> Primoz
> 
> Sounds like an interesting issue....
> 
> Sounds like you only need to route public addresses. Because of this I
> would just do what you have started to do and route the RFC 1918 range of
> address to the Null interface and move on. Sometimes the time taken in
> investigating the issue is greater than the benefit of finding the cause.
> 
> If you are only dealing with public IP's it is, in mho, best practice to
> blackhole route private addresses.
> 
> Failing that you could set up a packet sniffer, give it a private address
> on the L3 switch and route all the RFC1918 ranges to the sniffers IP. Then
> you can review the MAC addresses and start to track the source back...
> 
> Hope this is useful
> 
> Kindest Regards
> James Worley
> 
> Trident Net - Network Architect
> www.tridentnet.net
> 
> 
> > Hi guys
> >
> > I had some weird traffic appearing on one of our ethernet links and
> > I have no idea how to find out source of this traffic.
> >
> > But first a bit background so you can understand my scheme a bit
> > better.
> >
> > I have main router with no default route (just BGP with full table)
> > and then I have connected ethernet link to another router (actually
> > L3 switch) where few clients are connected (on L3 ports of course).
> > This L3 switch has default route pointing to main router.
> >
> > Ok now my problem... Ethernet utilization on main router went up to
> > 100% and when I was checking traffic with netflow I noticed huge
> > amount of traffic between 172.16.6.222 and 172.16.10.101. There
> > were only less then 30 flows in 30mins, but there were more then
> > 6GB of traffic between those two addresses in this time.
> > and
> > Really weird thing is, that I don't route private prefixes over this
> > link and none of routers (main router and L3 switches) has 172.16.0.0/12
> > prefix (or part of it) in its routing tables. Also there's no private
> > prefixes in any of my routing tables.
> >
> > When I was trying to find port from which this traffic went, I noticed
> > there's only high utilization on link between main router and L3
> > switch, and none of ports to clients had some extreme traffic. Traffic
> > between main router and L3 switch was around 100Mbps, while all clients
> > connected to L3 switch together had less then 50Mbps at that time.
> > So it looked to me, like traffic would be generating and then somehow
> > bouncing between L3 switch and main router. But due to really low number
> > of flows it's even more weird.
> >
> > Does anyone have some idea how to locate what exactly is happening?
> > I don't mean for backwards, but in case if it will happen in future
> > again. For now I routed 172.16.0.0/12 to Null0 on main router and
> > traffic stoped, but I still want to know what or who caused this
> > weird traffic.
> >
> > PS: Just another info... traffic didn't go out on any other ports on
> > main router, so it was really just on link between main router and L3
> > switch.
> >
> >
> > Thanks for help in advance.
> >
> > Have fun,
> > Primoz Jeroncic
> > Support - IP Connectivity & Routing
> > -------------------------------------------------------------------
> > Softnet d.o.o.  tel:  +386 1 562 31 40   |
> > Borovec 2       fax:  +386 1 562 18 55   |       1 + 1 = 3
> > 1236 Trzin      primoz(at)softnet.si     | for larger values of 1
> > Slovenija       http://flea.softnet.si/
> > -------------------------------------------------------------------
> >
> > _______________________________________________
> > cisco-nsp mailing list  cisco-nsp at puck.nether.net
> > https://puck.nether.net/mailman/listinfo/cisco-nsp
> > archive at http://puck.nether.net/pipermail/cisco-nsp/
> >
> 
> _______________________________________________
> cisco-nsp mailing list  cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/

-- 
Stephen J. Wilcox
BSc (Hons).  CCIE #10730
Technical Director, Telecomplete
http://www.telecomplete.co.uk/