[c-nsp] Ethernet debugging...MTU?
bradley at internode.com.au
Sat Oct 27 10:30:35 EDT 2007
Are you pinging with DF set?
Most PA-FE's I've seen only support an MTU of 1500 bytes.
Each device tends to also have a maximum packet size it can handle -
even after fragmentation. I've seen some cheap CPE not respond to pings
larger than about 8k.
Andy Dills wrote:
> Ok, I have a weird situation that I'd like to get some input on.
> Cliff notes:
> Is the inability to get ping replies with datagrams of larger than 9216
> bytes across a 100mbps ethernet circuit an indication that the far end is
> setup with an MTU consistent with jumbo frames?
> What to do if the far end swears it's set for 1500?
> We're in the process of turning up a new fast-e to a company located in
> Equinix. We are not located in Equinix, but Level3 is, and we have Level3
> fiber in our datacenter. There are two major segments of this circuit: the
> long haul to Ashburn and the cross connect in Equinix. The run at Equinix
> is too long for copper, and Level3 for some reason insisted upon a copper
> handoff, so Equinix supplied and installed transceivers to enable a fiber
> x-con that is delivered via copper to the cage.
> In our datacenter, the ethernet circuit is connected directly via a short
> copper run from Level3's space to a standard FE port on a Cisco router,
> that has previously been in use and is known to be working. If it matters,
> for now it's on a PA-2FEISL-TX (but will later be moved to a more current
> PA when put into production...to my knowledge, even though that PA isn't
> ideal, it should still work fine at lower bandwidth levels and with proper
> full-duplex, etc. The previously attached customer had no problems pushing
>> 30mbps, for example).
> When testing the circuit, using the cisco ping utility, with datagrams of
> 9216 bytes or less, we have no packet loss. When datagrams larger than
> 9216 bytes are used, we have 100% packet loss.
> Given the packet size when total failure occurs, my first reaction is that
> the other company has somehow misconfigured their switch to use jumbo
> frames, as if the circuit was a gig-e.
> According to the company at the far end, their device doesn't even support
> jumbo frames, and other people are attached and working fine. They seem to
> be quite certain the problem isn't on their end. I have a hard time not
> believing them; checking the MTU is a pretty cut and dry thing, and I'm
> working with senior level people at the other company, who I'm assuming
> (perhaps an incorrect assumption) run and manage their international
> To narrow down the problem, we have had Level3 setup a laptop in their
> cage at Equinix, attached to the interface facing our datacenter. When
> testing to that, I had no packet loss with packets of any size up to the
> maximum of 18024 bytes. To me this eliminates the long-haul portion from
> We've also had Equinix double check (at the supervisor level) that the
> transceivers are 10/100, hardcoded for 100 full-duplex (as is everything
> else end to end). I also have a hard time believing that Equinix would
> have any difficulties installing the correct model and properly configured
> ethernet transceivers; Ashburn is a top notch facility with good people.
> (I was hoping with my fingers crossed they had accidently installed gig-e
> transceivers, or that Level3 had accidentally ordered gig-e
> transceivers...no luck).
> As this has been a lingering issue for some time, I'm currently pushing
> for technical reps from all of the companies involved to meet up at
> Equinix, sit in a room, and figure it out. This is a bit difficult for the
> other company as they don't have any real local staffing. So, I'm hoping
> to come up with a solution that doesn't involve them getting on an
> airplane, but it's starting to look like our only avenue of resolution.
> That said, has anybody encountered this before, or has any theories about
> ways I can debug this short of having the other company (who doesn't have
> local staff) visit their cage and attach a laptop facing us, to localize
> the issue to either their switch or the Equinix cross connect?
> Am I likely correct in my theory that something is configured for the
> jumbo frame MTU and thus response packets aren't being properly
> Thanks for any insight.
> Andy Dills
> Xecunet, Inc.
> cisco-nsp mailing list cisco-nsp at puck.nether.net
> archive at http://puck.nether.net/pipermail/cisco-nsp/
Brad Gould, Network Engineer
PO Box 284, Rundle Mall 5000
Level 3, 132 Grenfell Street, Adelaide 5000
P: 08 8228 2999 F: 08 8235 6999
bradley at internode.com.au; http://www.internode.on.net/
More information about the cisco-nsp