[c-nsp] Please help clarify bus/fabric terminology on the 6500/7600

[Rick Ernst]

Thanks (and Gert, too),

So,

- The 32Gbs bus is shared and the PFC on the sup does the forwarding
- The switch fabric is on the Sup; DFC cards use the fabric, others use the
PFC
- The fabric is limited to 8 or 20Gbs depending on Sup; CEF 256cards use
8Gbs, CEF720 uses 20gbs if the Sup supports it
- Lowest-common-denominator applies to DFC cards; you get the DFC features,
capabilities and TCAM of the least capable card/Sup, but you can
mix-and-match
- If I happen to install a 256K DFC in a 1M TCAM system, can the DFC be
forced off; 1M TCAM via the 32Gbs bus?
- The 32Gbs bus and 20Gbs fabric are total capacity; could you push
1Gbs/31Gbs on the 32Gbs bus?
- Design considerations need to include Sup level, PFC, DFC, 32Gbs shared,
8gbs/20Gbs fabric

>From a practical viewpoint, I'm currently pushing a little less than ~800Mbs
in+out at about 120Kpps. It's getting to be too much for my current software
forwarding, especially during D/DoS.  A Sup720-3BXL gives me 1M routes in
TCAM, and 15Gbs/30Mpps forwarding in the PFC. Control-plane and data-plane
separation, with data-plane in hardware. I could use any combination of line
cards and still be significantly ahead of my current utilization.  As the
bits get bigger and faster, I can offload forwarding onto DFC-enabled cards,
but I'd need to start with DFCs that also have the large TCAM, otherwise
I'm  still using the 32Gbs bus and the PFC.

For D/DoS purposes, policing is handled in hardware at the port ASIC.  If a
1Gbs-connected network were to go nuts and was throttled to 1Mbs, neither
the bus nor fabric would see the .99Gbs?

Rick

On Thu, Nov 5, 2009 at 11:42 PM, Asbjorn Hojmark <Asbjorn at hojmark.org>wrote:

> On Thu, 5 Nov 2009 23:11:56 -0800, you wrote:
>
> >   - The raw capacity of the 32Gbs bus is just that; 32Gbs across the
> > entire bus, combined across all cards
>
> Well, actually it's 16 Gbps shared bus. (The "32G" is marketing, and
> even more so here, because it's not full duplex; it's a bus).
>
> >   - The switch fabric is single or dual channel 20Gbs, dual channel
> > just allowing higher port/speed-density on the line cards
>
> Each slot gets two fabric connections, but some cards use only one of
> those. The channels can be 8 og 20G.
>
> >   - The 20Gbs fabric is used to transfer traffic directly between
> > DFC-enabled line cards, bypassing the Sup.
>
> Not really. With a DFC, the lookup is done on the line card (instead
> of on the PFC on the Sup), but the forwarding is still done via the
> fabric (which is also on the Sup).
>
> >   - The full "720Gbs" capacity is all dual-fabric line cards with DFCs
>
> 9 slots * 2 channels/slot * 20G/channel * 2 for marketing = 720.
>
> The bandwidth is not (directly) dependant on the precense of DFCs,
> only the forwarding capacity. (And other resources, such as NetFlow
> table space).
>
> -A
>