[c-nsp] Cisco 3750G backplane throughput
Bryan Holloway
bryan at shout.net
Fri Dec 15 10:21:11 EST 2017
Thanks to everyone who responded both on- and off-list.
To expand on what's happening, here's the scenario:
Five-port copper LACP on Gig1/0/6-10 facing downstream equipment/customers.
Four-port ECMP using the 3750G SFP ports (Gig1/0/25-28) facing upstream
transit.
(Don't laugh ... well, ok ... it's ok to laugh. We inherited this.)
We're seeing equal/nicely-balanced utilization across all ports, but we
can't seem to push more than 2 Gbps. But based on the ASIC layout, it
doesn't seem like that should be an issue.
I'll concede (your honor) that this isn't the best use-case for a 3750G,
but I'll just throw it out there and see if anyone can think of a reason
for the bottleneck.
Thanks again, everyone.
On 12/15/17 6:47 AM, Chris Knipe wrote:
> Yeah,
>
> I don't know about 2G specifically, but we have noticed in the past that
> their performance aren't too great. Especially once you start stacking
> them, or start pushing high BW across multiple ASICs.
>
> It is at the end of the day more of a distribution switch rather than
> anything else (with quite small buffers I may add). Personally, given the
> OPs original message, I wouldn't rule out the switch completely, TBH.
>
> --
> Chris.
>
>
> On Fri, Dec 15, 2017 at 2:42 PM, Nick Cutting <ncutting at edgetg.com> wrote:
>
>> Nice work – The generation 1 and 2 3560 and 3750’s had 2/4 ASICS.
>>
>>
>>
>> I was wrong earlier, (switch replaced on me!) the E and the X have 1 per
>> 24 ports.
>>
>>
>>
>> I did dig one up – and got the same results as below
>>
>>
>>
>> Each SFP is on ONE of the ASICs shared by the copper ports.
>>
>>
>>
>> *From:* cknipe at savage.za.org [mailto:cknipe at savage.za.org] *On Behalf Of *Chris
>> Knipe
>> *Sent:* Friday, December 15, 2017 7:38 AM
>> *To:* Nick Cutting <ncutting at edgetg.com>
>> *Cc:* cisco-nsp (cisco-nsp at puck.nether.net) <cisco-nsp at puck.nether.net>
>> *Subject:* Re: [c-nsp] Cisco 3750G backplane throughput
>>
>>
>>
>> *This message originated from outside your organization.*
>> ------------------------------
>>
>> 3750G-48-TS:
>>
>>
>>
>> Switch Ports Model SW Version SW Image
>>
>>
>> ------ ----- ----- ---------- ----------
>>
>>
>> * 1 52 WS-C3750G-48TS 12.2(40)SE
>> C3750-ADVIPSERVICESK
>>
>>
>>
>>
>>
>> # sh platform pm platform-block
>>
>>
>>
>> interface gid gpn lpn asic hw-i flags sp dp bundle vlan mvid mac so_di
>> i_vlan
>>
>> ------------------------------------------------------------
>> -----------------
>>
>> Gi1/0/1 1 1 1 6 3 U 2 2 no 0 0 0 61441 0
>>
>>
>> Gi1/0/2 2 2 2 6 0 D 0 0 no 0 0 0 61442 0
>>
>>
>> Gi1/0/3 3 3 3 6 1 D 0 0 no 0 0 0 61443 0
>>
>>
>> Gi1/0/4 4 4 4 6 2 U 3 2 no 0 0 0 61444 0
>>
>>
>> Gi1/0/5 5 5 5 5 2 D 0 0 no 0 0 0 61445 0
>>
>>
>> Gi1/0/6 6 6 6 5 3 D 0 0 no 0 0 0 61446 0
>>
>>
>> Gi1/0/7 7 7 7 5 0 D 0 0 no 0 0 0 61447 0
>>
>>
>> Gi1/0/8 8 8 8 5 1 D 0 0 no 0 0 0 61448 0
>>
>>
>> Gi1/0/9 9 9 9 8 3 D 0 0 no 0 0 0 61449 0
>>
>>
>> Gi1/0/10 10 10 10 8 0 D 0 0 no 0 0 0 61450 0
>>
>>
>> Gi1/0/11 11 11 11 8 1 D 0 0 no 0 0 0 61451 0
>>
>>
>> Gi1/0/12 12 12 12 8 2 D 0 0 no 0 0 0 61452 0
>>
>>
>> Gi1/0/13 13 13 13 7 2 D 0 0 no 0 0 0 61453 0
>>
>>
>> Gi1/0/14 14 14 14 7 3 D 0 0 no 0 0 0 61454 0
>>
>>
>> Gi1/0/15 15 15 15 7 0 D 0 0 no 0 0 0 61455 0
>>
>>
>> Gi1/0/16 16 16 16 7 1 D 0 0 no 0 0 0 61456 0
>>
>>
>> Gi1/0/17 17 17 17 4 3 D 0 0 no 0 0 0 61457 0
>>
>>
>> Gi1/0/18 18 18 18 4 0 D 0 0 no 0 0 0 61458 0
>>
>>
>> Gi1/0/19 19 19 19 4 1 D 0 0 no 0 0 0 61459 0
>>
>>
>> Gi1/0/20 20 20 20 4 2 D 0 0 no 0 0 0 61460 0
>>
>>
>> Gi1/0/21 21 21 21 3 2 D 0 0 no 0 0 0 61461 0
>>
>>
>> Gi1/0/22 22 22 22 3 3 D 0 0 no 0 0 0 61462 0
>>
>>
>> Gi1/0/23 23 23 23 3 0 D 0 0 no 0 0 0 61463 0
>>
>>
>> Gi1/0/24 24 24 24 3 1 D 0 0 no 0 0 0 61464 0
>>
>>
>> Gi1/0/25 25 25 25 10 3 D 0 0 no 0 0 0 61465 0
>>
>>
>> Gi1/0/26 26 26 26 10 0 U 2 2 no 0 0 0 61466 0
>>
>>
>> Gi1/0/27 27 27 27 10 1 D 0 0 no 0 0 0 61467 0
>>
>>
>> Gi1/0/28 28 28 28 10 2 U 3 2 no 0 0 0 61468 0
>>
>>
>> Gi1/0/29 29 29 29 9 2 D 0 0 no 0 0 0 61469 0
>>
>>
>> Gi1/0/30 30 30 30 9 3 D 0 0 no 0 0 0 61470 0
>>
>>
>> Gi1/0/31 31 31 31 9 0 U 2 2 no 0 0 0 61471 0
>>
>>
>> Gi1/0/32 32 32 32 9 1 D 0 0 no 0 0 0 61472 0
>>
>>
>> Gi1/0/33 33 33 33 2 3 U 3 2 no 0 0 0 61473 0
>>
>>
>> Gi1/0/34 34 34 34 2 0 D 0 0 no 0 0 0 61474 0
>>
>>
>> Gi1/0/35 35 35 35 2 1 D 0 0 no 0 0 0 61475 0
>>
>>
>> Gi1/0/36 36 36 36 2 2 U 3 2 no 0 0 0 61476 0
>>
>>
>> Gi1/0/37 37 37 37 1 2 U 2 2 no 0 0 0 61477 0
>>
>>
>> Gi1/0/38 38 38 38 1 3 U 3 2 no 0 0 0 61478 0
>>
>>
>> Gi1/0/39 39 39 39 1 0 U 3 2 no 0 0 0 61479 0
>>
>>
>> Gi1/0/40 40 40 40 1 1 U 3 2 no 0 0 0 61480 0
>>
>>
>> Gi1/0/41 41 41 41 12 3 U 1 1 no 0 0 0 61481 0
>>
>>
>> Gi1/0/42 42 42 42 12 0 U 2 2 no 0 0 0 61482 0
>>
>>
>> Gi1/0/43 43 43 43 12 1 D 0 0 no 0 0 0 61483 0
>>
>>
>> Gi1/0/44 44 44 44 12 2 D 0 0 no 0 0 0 61484 0
>>
>>
>> Gi1/0/45 45 45 45 11 2 D 3 2 no 0 0 0 61485 0
>>
>>
>> Gi1/0/46 46 46 46 11 3 D 0 0 no 0 0 0 61486 0
>>
>>
>> Gi1/0/47 47 47 47 11 0 D 3 2 no 0 0 0 61487 0
>>
>>
>> Gi1/0/48 48 48 48 11 1 U 3 2 no 0 0 0 61488 0
>>
>>
>> Gi1/0/49 49 49 49 0 3 D 0 0 no 0 0 0 61489 0
>>
>>
>> Gi1/0/50 50 50 50 0 2 D 0 0 no 0 0 0 61490 0
>>
>>
>> Gi1/0/51 51 51 51 0 1 D 0 0 no 0 0 0 61491 0
>>
>>
>> Gi1/0/52 52 52 52 0 0 U 3 2 no 0 0 0 61492 0
>>
>>
>>
>>
>>
>>
>>
>>
>> On Fri, Dec 15, 2017 at 2:33 PM, Nick Cutting <ncutting at edgetg.com> wrote:
>>
>> I just realized that switch output I pasted was a 2960X - the 3560G was
>> swapped out by a colleague on Tuesday night !
>> It was a 3650G 48TS on Monday.
>>
>> The command should still work for you though
>>
>>
>> -----Original Message-----
>> From: cisco-nsp [mailto:cisco-nsp-bounces at puck.nether.net] On Behalf Of
>> Nick Cutting
>> Sent: Friday, December 15, 2017 7:29 AM
>> To: Bryan Holloway <bryan at shout.net>; cisco-nsp at puck.nether.net
>> Subject: Re: [c-nsp] Cisco 3750G backplane throughput
>>
>> This message originates from outside of your organisation.
>>
>> Use this command:
>>
>> sh platform pm platform-block
>>
>> Should be one ASIC per 24 ports, so a TS should have 1 asic for the normal
>> ports and one for the SFP's.
>> On a my 48portTS, the SFP's are shared across the two normal ASIC's (48
>> copper ports)
>>
>> I just removed my lab 3560g-24TS so I can't be 100 percent sure on the
>> ASIC distribution. I think it had 1 asic for the 24 copper and one for the
>> sfp's.
>>
>> How is your LAG traffic distribution? You need many different flows to
>> get much out of a LAG.
>> It is harder to get bandwidth out of the LAG than be outputted dropped by
>> the ASIC, what I mean is one port in the LAG may hit line rate before the
>> others are even using 50 percent of bandwidth.
>>
>> * 1 52 WS-C2960X-48LPS-L 15.2(2)E6
>> sh platform pm platform-block
>> interface gid gpn lpn asic
>> --------------------------
>> Gi1/0/1 1 1 1 0
>> Gi1/0/2 2 2 2 0
>> Gi1/0/3 3 3 3 0
>> Gi1/0/4 4 4 4 0
>> Gi1/0/5 5 5 5 0
>> Gi1/0/6 6 6 6 0
>> Gi1/0/7 7 7 7 0
>> Gi1/0/8 8 8 8 0
>> Gi1/0/9 9 9 9 0
>> Gi1/0/10 10 10 10 0
>> Gi1/0/11 11 11 11 0
>> Gi1/0/12 12 12 12 0
>> Gi1/0/13 13 13 13 0
>> Gi1/0/14 14 14 14 0
>> Gi1/0/15 15 15 15 0
>> Gi1/0/16 16 16 16 0
>> Gi1/0/17 17 17 17 0
>> Gi1/0/18 18 18 18 0
>> Gi1/0/19 19 19 19 0
>> Gi1/0/20 20 20 20 0
>> Gi1/0/21 21 21 21 0
>> Gi1/0/22 22 22 22 0
>> Gi1/0/23 23 23 23 0
>> Gi1/0/24 24 24 24 0
>> Gi1/0/25 25 25 25 1
>> Gi1/0/26 26 26 26 1
>> Gi1/0/27 27 27 27 1
>> Gi1/0/28 28 28 28 1
>> Gi1/0/29 29 29 29 1
>> Gi1/0/30 30 30 30 1
>> Gi1/0/31 31 31 31 1
>> Gi1/0/32 32 32 32 1
>> Gi1/0/33 33 33 33 1
>> Gi1/0/34 34 34 34 1
>> Gi1/0/35 35 35 35 1
>> Gi1/0/36 36 36 36 1
>> Gi1/0/37 37 37 37 1
>> Gi1/0/38 38 38 38 1
>> Gi1/0/39 39 39 39 1
>> Gi1/0/40 40 40 40 1
>> Gi1/0/41 41 41 41 1
>> Gi1/0/42 42 42 42 1
>> Gi1/0/43 43 43 43 1
>> Gi1/0/44 44 44 44 1
>> Gi1/0/45 45 464 45 1
>> Gi1/0/46 46 465 46 1
>> Gi1/0/47 47 457 47 1
>> Gi1/0/48 48 456 48 1
>> Gi1/0/49 49 49 49 0
>> Gi1/0/50 50 50 50 0
>> Gi1/0/51 51 51 51 1
>> Gi1/0/52 52 52 52 1
>>
>> -----Original Message-----
>> From: cisco-nsp [mailto:cisco-nsp-bounces at puck.nether.net] On Behalf Of
>> Bryan Holloway
>> Sent: Thursday, December 14, 2017 7:55 PM
>> To: cisco-nsp at puck.nether.net
>> Subject: [c-nsp] Cisco 3750G backplane throughput
>>
>> This message originates from outside of your organisation.
>>
>> Hello community,
>>
>> I'm curious if someone is in the know or can point me to a document that
>> describes how the backplane is carved up on a 3750G. I.e., ports per ASIC,
>> etc., if applicable. I've dug around the Cisco docs to no avail.
>>
>> I'm particularly interested to know how the four-port SFP section is
>> handled on, for example, a WS-C3750G-24TS. Does it have its own ASIC for
>> all four SFP ports? Or is that also carved up amongst other ports? If one
>> were to LAG all four SFP ports together, should one expect to be able to
>> reach a full 4 Gbps (assuming no taxation from other switch ports?)
>>
>> We're running into an odd issue where we're unable to achieve more than
>> 2 Gbps of bandwidth, but I have a hard time believing this is a switch
>> limitation.
>>
>> Any input would be most appreciated, thanks!
>>
>> - bryan
>> _______________________________________________
>> cisco-nsp mailing list cisco-nsp at puck.nether.net https://puck.nether.net/
>> mailman/listinfo/cisco-nsp
>> archive at http://puck.nether.net/pipermail/cisco-nsp/
>>
>> _______________________________________________
>> cisco-nsp mailing list cisco-nsp at puck.nether.net https://puck.nether.net/
>> mailman/listinfo/cisco-nsp
>> archive at http://puck.nether.net/pipermail/cisco-nsp/
>>
>> _______________________________________________
>> cisco-nsp mailing list cisco-nsp at puck.nether.net
>> https://puck.nether.net/mailman/listinfo/cisco-nsp
>> archive at http://puck.nether.net/pipermail/cisco-nsp/
>>
>>
>>
>>
>>
>> --
>>
>>
>> Regards,
>> Chris Knipe
>>
>
>
>
More information about the cisco-nsp
mailing list