[c-nsp] Catalyst 3550 VRF-lite problems

[Phil Mayers]

On Tue, Apr 03, 2007 at 10:19:37PM +1000, Reuben Farrelly wrote:
>Has anyone or does anyone ever done any small scale VRF configuration on 
>Catalyst 3550s or 3750s?

Small, and large.

>
>The reason I ask is today while diagnosing a fault that a customer was 
>experiencing on a L2 link (we provide L2 transit through the switch only, no 
>L3), I decided to configure a VRF on one of our 3550s, put the newly customer's 
>routed VLAN interface in that VRF with one of their IP addresses, and added a 
>default route to that VRF so I could replicate the fault (which was a routing 
>problem on a specific VLAN).  I would have thought that was fairly 
>straightforward and should pose no problems.  This was with 12.1(22)EA8 i5q3l2.
>
>The testing worked reasonably well, but it caused a total and utter meltdown on 
>the switch when I removed the VRF.  The switch jumped to 99% CPU, experienced a 
>massive decrease in performance (approximately halved throughput), started 
>seeing packet loss on some of the traffic through it but not all, and required a 
>reload mid afternoon to clear it.
>
>Needless to say, this was not an especially favourable outcome especially at 
>that time of day.
>
>After the event I noticed this in the syslog (this did not appear on the vty 
>session when I started to configure the VRF):
>
>
>%L3TCAM-3-SIZE_CONFLICT: VRF requires enabling extended routing

I've never seen this specific symptom, but you must have the extended 
SDM template to do VRFs. We have this on by default. This is mentioned 
in the docs pretty explicitly if I recall correctly, but I don't have a 
link handy.

conf t
sdm prefer routing extended-match

...on a 3550. Needs a reboot to take effect.

It would seem not having it caused the box to get into a broken state.

It's somewhat annoying the CLI doesn't refuse the VRF commands. The VRFs 
even half-work, but as you've found out, it's not a good idea. I 
*believe* what happens is that CPU-switched traffic works, but 
hardware-switched does not.


>
>Was this a known problem with this branch of code?  Or do people just avoid 
>VRF's on these switches altogether?

No - just configure them right ;o)

I agree the device should behave in a more safe manner. A bug with TAC 
might accomplish this, but not on the 3550 given their EoL status.

>
>What effect will enabling extended routing have on my current TCAM parameters?
>
>switch#
>  The current template is the default template.
>  The selected template optimizes the resources in
>  the switch to support this level of features for
>  8 routed interfaces and 1K VLANs.
>
>  number of unicast mac addresses:   5K
>  number of igmp groups:             1K
>  number of qos aces:                1K
>  number of security aces:           1K
>  number of unicast routes:          8K
>  number of multicast routes:        1K
>switch#

 The current template is the routing extended-match template.
 The selected template optimizes the resources in
 the switch to support this level of features for
 8 routed interfaces and 1K VLANs. 

 number of unicast mac addresses:   5K
 number of igmp groups:             1K
 number of qos aces:                512
 number of security aces:           512
 number of unicast routes:          8K
 number of multicast routes:        1K


>
>[Now, that message about only 8 routed interface is interesting given the switch 
>lets you configure many more with no apparent side effects, and I wonder if it 
>is actually correct, but that's another topic entirely.]

That is another topic. We've run 3550s with (far) more routed interfaces 
and apparently no ill effect with not inconsiderable traffic levels. But 
it could go wrong at any time, so I wouldn't advise it.