[c-nsp] Understanding 10G line card oversubscription

[Greg Whynott]

the 6500's are not very well suited for many roles these days in DC land,  especially if your into HPC.    if you are using a policy engine such as the FWSM,  you now only have 4G if your traffic has to pass threw it.  if memory serves me correctly.     We started seeing input drops which lead us to start a deployment OSPF-ECMP so we could balance and route around the core 6500..

kinda sad when you consider the cost of the kit.    we bought a pair of extreme switches last round,  they blow the socks of anything we have had from Cisco (so far!) in terms of features,  performance and cost of ownership.

just my opinion,  i may be way off base.

greg





On Mar 21, 2011, at 5:26 PM, Mack McBride wrote:

> There is the 4 port group which I believe is the same as a port pair on a 6708. Ie. No free switching.
> Based on some comments by Tim Stevenson, the link between the four port group and the fabric asic is 10G
> rather than the 16G in the 6708.  But he mentions some of the same chips (metro, r2d2) so the architecture
> can't be much different from the 6708.
>
> Then there is the fabric group which is switched locally without going over the fabric.
>
> show asic slot <x>
> sh int <interface> capabilities | inc ASIC
>
> Of course a diagram would be really helpful but those usually come with an NDA.
>
> 6708 results:
>
>        ASIC Name      Count      Version
>             KUMA          2      (3.0)
>      METRO_ARGOS          2      (3.0)
>    METRO_KRYPTON          2      (3.0)
>              SSA          2      (9.0)
>             R2D2          8      (2.0)
>         TIANGANG          4      (54.0)
>
> Te<x>/1
>  Ports-in-ASIC (Sub-port ASIC) : 1,4-5,7 (1)
> Te<x>/2
>  Ports-in-ASIC (Sub-port ASIC) : 2-3,6,8 (2)
>
> My understanding of the chips is as follows:
>
> R2D2(per port) -> Tiangang (port pair) -> Metro argos -> SSA (Fabric complex)
> Metro Krypto interfaces to the EARL complex aka PFC (local switching/routing to avoid fabric)
> Kuma acts as a bus bridge (not relevant to fabric switching)
>
> The assumption is that the Tiangang is replace with another chip that can handle
> 4 ports.  I assume the new FPGA - Metro link is somehow different on the 6716 but
> I don't know how.
>
> The 6704 skips the tiangang chips and directly connects the R2D2 to the Metro.
>
> From a Cisco doc (http://www.cisco.com/en/US/docs/ios/vswitch/command/reference/vs_02.pdf):
>    ASIC Name  Count  Version
>         KUMA     2   (2.0)
>  METRO_ARGOS     2   (2.0)
> METRO_KRYPTON     2   (2.0)
>          SSA     2   (8.0)
>         R2D2     4   (2.0)
>
> Short answer seems to be that the 6716 is basically the same despite claims to the contrary
>
> Mack
>
> -----Original Message-----
> From: cisco-nsp-bounces at puck.nether.net [mailto:cisco-nsp-bounces at puck.nether.net] On Behalf Of Phil Mayers
> Sent: Monday, March 21, 2011 1:51 PM
> To: cisco-nsp at puck.nether.net
> Subject: Re: [c-nsp] Understanding 10G line card oversubscription
>
> On 03/21/2011 06:33 PM, Mack McBride wrote:
>
>> The 6716 is going to have similar limitations but I don't have a good document on how the port asics connect
>
> My understanding is that the 6716 is quite different from the 6708.
> There's no "free" local switching within port groups AFAIK. The
> differences have been discussed on the list before.
>
>>
>> If someone can verify the connection method and limitations on the 6716 it would be appreciated.
>
> Do you have specific IOS commands? We've got a few in service.
> _______________________________________________
> cisco-nsp mailing list  cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>
> _______________________________________________
> cisco-nsp mailing list  cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/

Gregory Whynott
Networks and Storage

Ontario Institute for Cancer Research
MaRS Centre, South Tower
101 College Street, Suite 800
Toronto, Ontario, Canada M5G 0A3

647-294-2813 | www.oicr.on.ca


--

This message and any attachments may contain confidential and/or privileged information for the sole use of the intended recipient. Any review or distribution by anyone other than the person for whom it was originally intended is strictly prohibited. If you have received this message in error, please contact the sender and delete all copies. Opinions, conclusions or other information contained in this message may not be that of the organization.