[c-nsp] WS-X6704-10GE, WS-X6708-10GE

[Artyom Viklenko]

Hi, List!

May be these questions was discussed earlier... can't find it...
Please give me some links than.

I'm tring to clarify my understanding of switching paths on these
line cards. From one point of view, Cisco docs says that if the
traffic should ingress via one port on the line card and then
should egress through another port on the same line card it will
never leave this line card. So it will be switched via internal
bus. Right?

At one of our POPs we have Cisco 7606-S chassis with the folowing:

Mod Ports Card Type                              Model
--- ----- -------------------------------------- ------------------
   2    4  CEF720 4 port 10-Gigabit Ethernet      WS-X6704-10GE
   3   48  CEF720 48 port 1000mb SFP              WS-X6748-SFP
   4   48  CEF720 48 port 10/100/1000mb Ethernet  WS-X6748-GE-TX
   5    2  Route Switch Processor 720 (Active)    RSP720-3CXL-GE

At peaks we see total ingress thraffic on all 10GE ports around
30-32Gbps and increase of overruns on all 10GE ports.

Utilization of the fabric and forwarding performance are as folows:

Switch Fabric Resources
   Bus utilization: current: 10%, peak was 51% at 21:31:26 EET Sat Feb 
18 2012
   Fabric utilization:     Ingress                    Egress
     Module  Chanl  Speed  rate  peak                 rate  peak 

     2       0        20G   15%   46% @22:57 30Jan12   15%   49% @22:02 
31Jan12
     2       1        20G   15%   49% @20:08 01Feb12   15%   46% @20:14 
25Feb12
     3       0        20G    0%    2% @10:49 02Feb12    0%    1% @14:16 
30Jan12
     3       1        20G    1%    1% @14:16 30Jan12    0%    2% @19:25 
30Jan12
     4       0        20G    1%   16% @19:22 11Feb12    3%   11% @19:40 
04Feb12
     4       1        20G    3%    9% @22:23 31Jan12    1%   10% @19:23 
11Feb12
     5       0        20G    0%    1% @14:16 30Jan12    0%    2% @21:24 
10Feb12
   Switching mode: Module 
Switching mode
                   2 
compact
                   3 
compact
                   4 
compact
                   5 
compact
a

L2 Forwarding Resources
            MAC Table usage:   Module  Collisions  Total       Used 
   %Used
                               5                0  98304        292 
      1%

              VPN CAM usage:                       Total       Used 
   %Used
                                                     512          0 
      0%
L3 Forwarding Resources
  Module              FIB TCAM usage:                     Total 
Used     %Used
    5                     72 bits (IPv4, MPLS, EoM)      524288 
13963      3%
                         144 bits (IP mcast, IPv6)      262144 
316      1%

                      detail:      Protocol                    Used 
   %Used
                                   IPv4                        7057 
      1%
                                   MPLS                        6897 
      1%
                                   EoM                            9 
      1%

                                   IPv6                         117 
      1%
                                   IPv4 mcast                   196 
      1%
                                   IPv6 mcast                     3 
      1%

             Adjacency usage:                     Total        Used 
   %Used
                                                1048576        7759 
      1%

      Forwarding engine load:
                      Module       pps   peak-pps 
peak-time
                      5        1965007   10703659  21:31:21 EET Sat Feb 
18 2012

Actually, typical PPS is about 5-6 millions at peak times in evening
and bus utilization is about 20%.

I made simple calculations and found that about 16Gbps switched via
internal bus at line card and about 14-15 Gbps switched via fabric.
So the problem is internal 16Gbps (atually 16Gbps+16Gbps?) bus on
linecard.

So, the possible solution seems to install additional 6704 line
card ad distribute links between them according to main traffic flows.

Is it correct that CFC is not an issue in this particular situation?
D-Bus is not overutilized yet. I agree that this is goog to install
DFC dauter cards (or even 6708 with DFC), but not now.

I kbow that WS-X6708 much better option (and it is DFC), but now we
have no possibility to replace all 6704 by 6708 ones.

Is it all correct or I'm missing something?

Is it possible somehow to disable switching via internal bus on linecard
and reroute all traffic via fabric?

Similar problem was found on another router with WS-X6708 line card.
After swapping some 10GE links between ports most part of traffic
starts to go via fabric. And overruns disappeared.

Thanks in advance!


-- 
            Sincerely yours,
                             Artyom Viklenko.
-------------------------------------------------------
artem at aws-net.org.ua | http://www.aws-net.org.ua/~artem
artem at viklenko.net   | JID: artem at jabber.aws-net.org.ua
FreeBSD: The Power to Serve   -  http://www.freebsd.org