[c-nsp] WS-6904-40G and FourX adapter

[Peter Rathlev]

On Wed, 2014-03-19 at 14:30 -0400, Jay Young wrote:
> Anyone have these in production?

We started using the in several places. We currently have five in
production all using the CVR-CFP-4SFP10G adapter.

> We just moved some 10 Gig links to these and are experiencing some
> problems with traffic getting capped around 1 Gig, moved it to another
> port and things are OK but then one of the other ports starts capping
> traffic around 1 Gig. Seems strange if it is a buffering problem
> unless the buffers are dynamically shared between the 10 Gig
> interfaces. Docs seems to indicate fixed buffers per interface though.

Is the problem on receive or transmit on the X6904 port? Do you see
"Total output drops:" increasing or any other error counters increasing?
Have you verified and possibly qualified the drops with e.g. tcpdump or
similar? 

Regarding buffering you should have ~2.5 MB on egress per queue per port
in 10G mode, totalling 21 MB egress. You can of course adjust how much
each queue receives. Ingress is much less but also less relevant in
light of the the C6k works.

If you suspect buffering problems you can take a look at "show queueing
interface TeX/Y", maybe with " | begin Packets dropped" for the
interesting stuff.

Traffic the enters one port and exist another on the same card are
subject to some bottlenecks in the design, unless you use "performance"
mode and disable half the interfaces.

We have had no problems transferring 7-8 Gbps through these and are
quite happy with them.

-- 
Peter