[c-nsp] carving TCAM on Engine 3 linecards due to exhausted IPv6 TCAM regions

[Jan Sandmaier]

Hi Pelle,

thanks for your good answer. I simply can't configure this. What code
are you running? I'm running an quite old code. It seems I have to do an
upgrade first.

(config)#hw-module slot 1 tcam carve rx_ipv6 14
% Ambiguous command:  "hw-module slot 1 tcam carve rx_ipv6 14"
(config)#hw-module slot 1 tcam carve RX_IPv6 14
% Ambiguous command:  "hw-module slot 1 tcam carve RX_IPv6 14"
(config)#hw-module slot 1 tcam carve RX_IPv6 ?
% Ambiguous command:  "hw-module slot 1 tcam carve RX_IPv6 "
(config)#hw-module slot 1 tcam carve RX_IPv6 ?
% Ambiguous command:  "hw-module slot 1 tcam carve RX_IPv6 "

bb-b.fra3.fra.de#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
bb-b.fra3.fra.de(config)#hw-module slot 1 tcam carve ?
  RX_144b_REGION             RX 144b Dynamic Region
  RX_288b_REGION             RX 288b Dynamic Region
  RX_72b_REGION              RX 72b Dynamic Region
  RX_BOT_144b_REGION         RX Bottom 144b Dynamic Region
  RX_BOT_288b_REGION         RX Bottom 288b Dynamic Region
  RX_BOT_72b_REGION          RX Bottom 72b Dynamic Region
  RX_IPV6_MCAST_288b_REGION  RX IPv6 Mcast 288b Region
  RX_IPv6_144b_REGION        RX IPv6 144b Region
  RX_TOP_144b_REGION         RX TOP 144b Fixed Region
  RX_TOP_288b_REGION         RX TOP 288b Fixed Region
  RX_TOP_72b_REGION          RX TOP 72b Fixed Region
  RX_TOP_NF_REGION           RX TOP NetFlow Region
  TX_144b_REGION             TX 144b Dynamic Region
  TX_288b_REGION             TX 288b Dynamic Region
  TX_72b_REGION              TX 72b Dynamic Region
  TX_BOT_144b_REGION         TX Bottom 144b Dynamic Region
  TX_BOT_288b_REGION         TX Bottom 288b Dynamic Region
  TX_BOT_72b_REGION          TX Bottom 72b Dynamic Region
  TX_IPv6_144b_REGION        TX IPv6 144b Region
  TX_TOP_144b_REGION         TX TOP 144b Fixed Region
  TX_TOP_288b_REGION         TX TOP 288b Fixed Region
  TX_TOP_72b_REGION          TX TOP 72b Fixed Region
  TX_TOP_NF_REGION           TX TOP Netflow Region

Thanks,
Jan

>> after some flux of IPv6 routes I got on my last old GSR the error
>> message for each line card:
> 
> We experienced that as well today.
> 
>> Issueing a "sh controllers frfab alpha tcam carve" shows me an
>> utilization of 50-100% of any RX_IPv6_* region while all other regions
>> have 0 to 5% utilization. I've no idea what the other regions exactly
>> are used for except Netflow.
> 
> The others are for ACLs, MQC, policy routing etc. (from the ouput
> above "WARNING: Recarve may affect other input
> features(ACL,CAR,MQC,Netflow)").
> 
>> Does anybody have experience in re-carving this TCAM?
> 
> We have had an discussion with Cisco about the topic today. If you
> lookup CSCsx52504 you will find an explanation of the default TCAM
> carvings and how to change those.
> 
>> Do I need to issue this command for each prefix lengh where a region
>> exists for or is there some magic command?
>>
>> #hw-module slot 1 tcam carve RX_IPv6_144b_REGION ?
>>  <0-128>  Prefix length
>>
>> #hw-module slot 1 tcam carve RX_IPv6_144b_REGION 128 ?
>>  <0-100>  percentage of TCAM
> 
> There is a magic command. You only allocate a percentage for IPv6 and
> it will be divided per prefix length automatically.
> 
> From the bug description: "For the purposes of IPv6 scalability, the
> RX_IPv6 regions are the ones we're interested in. Initially, all IPV6
> cells are placed into the /128 region. As we add other mask lengths
> into the ipv6 routing table, we will "move" blocks of the TCAM into a
> new region that corresponds to the mask length for that prefix.".
> 
> Further on:
> 
> To support more IPv6 routes in hardware:
> Get current TCAM usage with: show controllers ISE tcam
> In config mode, reallocate TCAM regions e.g. reallocate Netflow TCAM to IPv6
> 
> hw-module slot < num> tcam carve rx_ipv6 < v6-percent>
> hw-module slot < num> tcam carve rx_top_nf < nf-percent>
> 
> Verify with show command that sum of all TCAM regions = 100%
> 
> Reload the linecard for the new TCAM carve config to take effect
>