[c-nsp] Ethernet debugging...MTU?

[Brad Gould]

Are you pinging with DF set?

Most PA-FE's I've seen only support an MTU of 1500 bytes.

Each device tends to also have a maximum packet size it can handle - 
even after fragmentation.  I've seen some cheap CPE not respond to pings 
larger than about 8k.

Brad

Andy Dills wrote:
> Ok, I have a weird situation that I'd like to get some input on.
> 
> Cliff notes:
> 
> Is the inability to get ping replies with datagrams of larger than 9216 
> bytes across a 100mbps ethernet circuit an indication that the far end is 
> setup with an MTU consistent with jumbo frames? 
> 
> What to do if the far end swears it's set for 1500?
> 
> 
> Background:
> 
> We're in the process of turning up a new fast-e to a company located in 
> Equinix. We are not located in Equinix, but Level3 is, and we have Level3 
> fiber in our datacenter. There are two major segments of this circuit: the 
> long haul to Ashburn and the cross connect in Equinix. The run at Equinix 
> is too long for copper, and Level3 for some reason insisted upon a copper 
> handoff, so Equinix supplied and installed transceivers to enable a fiber 
> x-con that is delivered via copper to the cage.
> 
> In our datacenter, the ethernet circuit is connected directly via a short 
> copper run from Level3's space to a standard FE port on a Cisco router, 
> that has previously been in use and is known to be working. If it matters, 
> for now it's on a PA-2FEISL-TX (but will later be moved to a more current 
> PA when put into production...to my knowledge, even though that PA isn't 
> ideal, it should still work fine at lower bandwidth levels and with proper 
> full-duplex, etc. The previously attached customer had no problems pushing 
>> 30mbps, for example).
> 
> When testing the circuit, using the cisco ping utility, with datagrams of 
> 9216 bytes or less, we have no packet loss. When datagrams larger than 
> 9216 bytes are used, we have 100% packet loss.
> 
> Given the packet size when total failure occurs, my first reaction is that 
> the other company has somehow misconfigured their switch to use jumbo 
> frames, as if the circuit was a gig-e.
> 
> According to the company at the far end, their device doesn't even support 
> jumbo frames, and other people are attached and working fine. They seem to 
> be quite certain the problem isn't on their end. I have a hard time not 
> believing them; checking the MTU is a pretty cut and dry thing, and I'm 
> working with senior level people at the other company, who I'm assuming 
> (perhaps an incorrect assumption) run and manage their international 
> network.
> 
> 
> To narrow down the problem, we have had Level3 setup a laptop in their 
> cage at Equinix, attached to the interface facing our datacenter. When 
> testing to that, I had no packet loss with packets of any size up to the 
> maximum of 18024 bytes. To me this eliminates the long-haul portion from 
> consideration.
> 
> We've also had Equinix double check (at the supervisor level) that the 
> transceivers are 10/100, hardcoded for 100 full-duplex (as is everything 
> else end to end). I also have a hard time believing that Equinix would 
> have any difficulties installing the correct model and properly configured 
> ethernet transceivers; Ashburn is a top notch facility with good people. 
> (I was hoping with my fingers crossed they had accidently installed gig-e 
> transceivers, or that Level3 had accidentally ordered gig-e 
> transceivers...no luck).
> 
> As this has been a lingering issue for some time, I'm currently pushing 
> for technical reps from all of the companies involved to meet up at 
> Equinix, sit in a room, and figure it out. This is a bit difficult for the 
> other company as they don't have any real local staffing. So, I'm hoping 
> to come up with a solution that doesn't involve them getting on an 
> airplane, but it's starting to look like our only avenue of resolution.
> 
> That said, has anybody encountered this before, or has any theories about 
> ways I can debug this short of having the other company (who doesn't have 
> local staff) visit their cage and attach a laptop facing us, to localize 
> the issue to either their switch or the Equinix cross connect?
> 
> Am I likely correct in my theory that something is configured for the 
> jumbo frame MTU and thus response packets aren't being properly 
> fragmented?
> 
> Thanks for any insight.
> 
> Andy
> 
> ---
> Andy Dills
> Xecunet, Inc.
> www.xecu.net
> 301-682-9972
> ---
> _______________________________________________
> cisco-nsp mailing list  cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/

-- 
Brad Gould, Network Engineer
Internode
PO Box 284, Rundle Mall 5000
Level 3, 132 Grenfell Street, Adelaide 5000
P: 08 8228 2999  F: 08 8235 6999
bradley at internode.com.au; http://www.internode.on.net/