<html>
<head>
<style><!--
.hmmessage P
{
margin:0px;
padding:0px
}
body.hmmessage
{
font-size: 12pt;
font-family:Calibri
}
--></style></head>
<body class='hmmessage'><div dir='ltr'>Thanks Joseph and Eldon. That seems to be the issue.<br><br>On the XMR in question, I currently have 126 IPs listed in the default-vrf. The others have even more. <br><br>I am wondering though. Let's say I had 29 IPs on my box with this rACL. Then provisioning add three more customers. What happens? That would take you over the limit. Would the entire rACL be removed or just not applied to the three new interfaces? Then again, if the box was rebooted what would then happen?<br><br><br><br>Thanks<br>Darren<br><a href="http://www.mellowd.co.uk/ccie" target="_blank">http://www.mellowd.co.uk/ccie</a><br><br><br><br><div><hr id="stopSpelling">From: Joseph.Kennedy@purchase.edu<br>To: darrenoc@outlook.com<br>CC: esk-puck.nether.net@esk.cs.usu.edu; foundry-nsp@puck.nether.net<br>Subject: Re: [f-nsp] exceed configured CAM size, larger partition size required<br>Date: Sun, 26 Jan 2014 19:51:40 +0000<br><br>
You’ll want to do a "show cam-partition usage” and look under the [Session] section to see the "Receive ACL" CAM usage. The more interface IP’s you have on your XMR16, the more you'll need to increase your receive-cam value accordingly. Rule-based ACL CAM
is not affected by rACL statements. The two are taken out of the same overall pool but they are exclusive.
<div><br>
</div>
<div>Eldon is correct. Every rACL entry you specify is being added to CAM for every interface IP address as a destination. You can calculate your required receive-cam usage as approximately (rACL size) x (total number of interface IP’s). You can count entries
from "show ip interfaces | inc default-vrf” and multiply by 33. Anything over 30 interface IP's on your XMR16 would mean you have to increase your receive-cam value to at least 2048. Yes VE’s and loopbacks count. Based on IP address not physical interface
per-se.</div>
<div><br>
</div>
<div>Also see the note from the guide(since you said you’re using the Multi-service 4 cam-profile) just incase you have an exceptionally large number of interface IP’s or plan to increase your rACL size considerably.</div>
<div><br>
</div>
<div>
<div>
<div>From the 5.4.00a Config Guide:</div>
<div><br>
</div>
<div>NOTES: The following limitations apply when the <number> variable has a maximum limit of 16384.</div>
<div>• The 16K Receive ACL CAM partition is not supported on the cam profiles such as IPv6, </div>
<div>Multi-service 3, and <font color="#e32400">Multi-service 4</font>.</div>
<div>• Depending on the configuration, any of the IPv4 ACL sub-partitions such as IP Source </div>
<div>Guard, Broadcast ACL, IP Multicast, and OpenFlow should be decreased to allow the </div>
<div>creation of the 16K rACL partition. </div>
</div>
<div><br>
</div>
<div>--JK</div>
<div><br>
<div>
<div>On Jan 26, 2014, at 11:00 AM, Darren O'Connor <<a href="mailto:darrenoc@outlook.com">darrenoc@outlook.com</a>> wrote:</div>
<br class="ecxApple-interchange-newline">
<blockquote>
<div class="ecxhmmessage" style="font-size:12pt;font-family:Calibri;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;">
<div dir="ltr">Hi all. I'll answer all three questions altogether.<span class="ecxApple-converted-space"> </span><br>
<br>
All line cards are the same. It's a mixture of copper and sfp 20 and 24 port modules, as well as a mix of 2X10Gb and 4X10Gb cards. All XMR<br>
<br>
The receive-cam values are currently the same on all our XMRs:<br>
show default values | include receive-cam<br>
receive-cam 1024 16384 1024 1024 1024 No<br>
<br>
<br>
The scrubbed ACLs in question are as follows:<br>
<br>
access-list 180 permit ospf any any<br>
access-list 180 permit vrrp any any<br>
access-list 180 permit tcp any any eq bgp<br>
access-list 180 permit tcp any any eq 3784<br>
access-list 180 permit tcp any any eq 3785<br>
access-list 180 permit udp any any eq 3784<br>
access-list 180 permit udp any any eq 3785<br>
access-list 180 permit tcp any any eq 639<br>
access-list 180 permit esp any any<br>
access-list 180 permit mpls-in-ip any any<br>
access-list 180 permit rsvp any any<br>
access-list 180 permit pim any any<br>
access-list 180 permit tcp any any eq 646<br>
access-list 180 permit udp any any eq 646<br>
access-list 180 permit tcp any any eq 646<br>
access-list 180 permit udp any any eq 646<br>
access-list 180 permit tcp any eq 646 any<br>
access-list 180 permit udp any eq 646 any<br>
!<br>
access-list 181 permit icmp any any<br>
!<br>
access-list 182 deny ip any any<br>
!<br>
access-list 199 permit ip 1.1.106.128 0.0.0.7 any<br>
access-list 199 permit ip 1.1.106.152 0.0.0.7 any<br>
access-list 199 permit ip 1.1.111.80 0.0.0.15 any<br>
access-list 199 permit ip 1.1.90.224 0.0.0.31 any<br>
access-list 199 permit ip 1.1.107.104 0.0.0.7 any<br>
access-list 199 permit ip 1.1.94.240 0.0.0.15 any<br>
access-list 199 permit ip 1.1.224.128 0.0.0.31 any<br>
access-list 199 permit ip 1.1.225.128 0.0.0.31 any<br>
access-list 199 permit ip 1.1.226.0 0.0.0.127 any<br>
access-list 199 permit ip 1.1.239.240 0.0.0.15 any<br>
access-list 199 permit ip 1.1.24.96 0.0.0.15 any<br>
access-list 199 permit ip 1.1.22.32 0.0.0.31 any<br>
access-list 199 permit ip host 1.1.60.78 any<span class="ecxApple-converted-space"> </span><br>
<br>
policy-map 1m-management<br>
cir 1042432 cbs 1250<br>
<br>
ip receive access-list 199 sequence 1<br>
ip receive access-list 181 sequence 10 policy-map 1m-management strict-acl<br>
ip receive access-list 180 sequence 20<br>
ip receive access-list 182 sequence 200<br>
<br>
<br>
There is but a handful of ACLs applied to an interface here and there. Hardly any. The above ACLs are to protect the XMR CPU to only allow what is needed to reach the CPU of course.<br>
<br>
<br>
Thanks<br>
Darren<br>
<a href="http://www.mellowd.co.uk/ccie" target="_blank">http://www.mellowd.co.uk/ccie</a><br>
<br>
<br>
<br>
<div>> From:<span class="ecxApple-converted-space"> </span><a href="mailto:Joseph.Kennedy@purchase.edu">Joseph.Kennedy@purchase.edu</a><br>
> To:<span class="ecxApple-converted-space"> </span><a href="mailto:esk-puck.nether.net@esk.cs.usu.edu">esk-puck.nether.net@esk.cs.usu.edu</a>;<span class="ecxApple-converted-space"> </span><a href="mailto:darrenoc@outlook.com">darrenoc@outlook.com</a><br>
> CC:<span class="ecxApple-converted-space"> </span><a href="mailto:foundry-nsp@puck.nether.net">foundry-nsp@puck.nether.net</a><br>
> Subject: RE: [f-nsp] exceed configured CAM size, larger partition size required<br>
> Date: Fri, 24 Jan 2014 21:40:03 +0000<br>
><span class="ecxApple-converted-space"> </span><br>
> IP receive cam is for rACL's aka ip receive ACL's. Basically you would use the "ip receive ..." command to bind an ACL to all your interfaces in the default VRF and limit traffic directed to those interfaces . If you look at your cam-partition usage it shouldn't
use more than a single entry unless you've explicitly configured it. If you aren't using the ACL in question for transit traffic but rather to protect your router IP's I would say you should switch to rACL's (unless you are running VRF's). You might even consider
cam-sharing if the restrictions don't apply to you(like doesn't work with ve's).<br>
><span class="ecxApple-converted-space"> </span><br>
> Darren, do you mind showing us your ACL(with IP ranges omitted or changed if necessary) and your current cam-partition usage for a typical line card with XMR4 vs XMR16? You can limit the usage stats to the [Session] sections.<br>
><span class="ecxApple-converted-space"> </span><br>
> --JK<br>
><span class="ecxApple-converted-space"> </span><br>
> -----Original Message-----<br>
> From: foundry-nsp [<a href="mailto:foundry-nsp-bounces@puck.nether.net">mailto:foundry-nsp-bounces@puck.nether.net</a>] On Behalf Of Eldon Koyle<br>
> Sent: Friday, January 24, 2014 3:54 PM<br>
> To: 'Darren O'Connor'<br>
> Cc: <a href="mailto:foundry-nsp@puck.nether.net">foundry-nsp@puck.nether.net</a><br>
> Subject: Re: [f-nsp] exceed configured CAM size, larger partition size required<br>
><span class="ecxApple-converted-space"> </span><br>
> You might check the receive-cam value. It is documented in the security guide. The default is 1024 on the MLX, not sure if it is different for XMR. It says this is "the maximum number of ACL CAM entries that are allowed"; 1024 sounds a bit low if you are
applying an ACL on every interface.<br>
><span class="ecxApple-converted-space"> </span><br>
> Assuming that each line in an ACL consumes one such entry and each interface requires its own copy of the ACL, you could apply a 20-line ACL to 51 interfaces before you ran out. I could be interpreting this wrong, though.<br>
><span class="ecxApple-converted-space"> </span><br>
> MLXe8-M# show default values | inc receive-cam<br>
> receive-cam 1024 16384 1024 1024 1024 No<span class="ecxApple-converted-space"> </span><br>
><span class="ecxApple-converted-space"> </span><br>
> --<br>
> Eldon Koyle<br>
> --<br>
> BOFH excuse #350:<br>
> paradigm shift...without a clutch<br>
><span class="ecxApple-converted-space"> </span><br>
><span class="ecxApple-converted-space"> </span><br>
> On Jan 24 19:20+0000, Kennedy, Joseph wrote:<br>
> > Are you using the same line cards in the XMR4 as you are in the XMR8's and XMR16's?<br>
> ><span class="ecxApple-converted-space"> </span><br>
> > Are you using tcp/udp port ranges in the ACL in question? (I believe<span class="ecxApple-converted-space"> </span><br>
> > every tcp/udp port in the range may require its own CAM entry)<br>
> ><span class="ecxApple-converted-space"> </span><br>
> > --JK<br>
> ><span class="ecxApple-converted-space"> </span><br>
> > -----Original Message-----<br>
> > From: foundry-nsp [<a href="mailto:foundry-nsp-bounces@puck.nether.net">mailto:foundry-nsp-bounces@puck.nether.net</a>] On<span class="ecxApple-converted-space"> </span><br>
> > Behalf Of Darren O'Connor<br>
> > Sent: Friday, January 24, 2014 1:05 PM<br>
> > To: Eldon Koyle<br>
> > Cc: <a href="mailto:foundry-nsp@puck.nether.net">foundry-nsp@puck.nether.net</a><br>
> > Subject: Re: [f-nsp] exceed configured CAM size, larger partition size<span class="ecxApple-converted-space"> </span><br>
> > required<br>
> ><span class="ecxApple-converted-space"> </span><br>
> > ACL applied is only 20 lines long. I have just a handful of ACLs applied elsewhere.<br>
> ><span class="ecxApple-converted-space"> </span><br>
> > yes the XMR16 has a load more ports, but that should not cause an issue due to the tiny ACLs I'm using.<br>
> ><span class="ecxApple-converted-space"> </span><br>
> > same tcam profile used on all boxes: CAM partitioning profile:<span class="ecxApple-converted-space"> </span><br>
> > multi-service-4<br>
> ><span class="ecxApple-converted-space"> </span><br>
> > system max:<br>
> ><span class="ecxApple-converted-space"> </span><br>
> > sh run | inc system-max<br>
> > system-max vlan 4095<br>
> > system-max ip-cache 768000<br>
> > system-max ip-route 768000<br>
> > system-max virtual-interface 4095<br>
> > system-max ipv6-cache 32000<br>
> > system-max ipv6-route 32000<br>
> > system-max lsp-out-acl-cam 1000<br>
> ><span class="ecxApple-converted-space"> </span><br>
> ><span class="ecxApple-converted-space"> </span><br>
> > Thanks<br>
> > Darren<br>
> > <a href="http://www.mellowd.co.uk/ccie" target="_blank">http://www.mellowd.co.uk/ccie</a><br>
> ><span class="ecxApple-converted-space"> </span><br>
> ><span class="ecxApple-converted-space"> </span><br>
> ><span class="ecxApple-converted-space"> </span><br>
> ><span class="ecxApple-converted-space"> </span><br>
> > > Date: Fri, 24 Jan 2014 09:58:06 -0700<br>
> > > From: <a href="mailto:ekoyle@gmail.com">ekoyle@gmail.com</a><br>
> > > To: <a href="mailto:darrenoc@outlook.com">darrenoc@outlook.com</a><br>
> > > CC: <a href="mailto:foundry-nsp@puck.nether.net">foundry-nsp@puck.nether.net</a><br>
> > > Subject: Re: [f-nsp] exceed configured CAM size, larger partition<span class="ecxApple-converted-space"> </span><br>
> > > size required<br>
> > ><span class="ecxApple-converted-space"> </span><br>
> > > Which cam-partition profile are you using? How long are your ACLs?<span class="ecxApple-converted-space"> </span><br>
> > > I'm guessing your XMR16 has a lot more ports than any of your XMR4s,<span class="ecxApple-converted-space"> </span><br>
> > > so that could explain why you are having issues there.<br>
> > ><span class="ecxApple-converted-space"> </span><br>
> > > --<br>
> > > Eldon Koyle<br>
> > ><span class="ecxApple-converted-space"> </span><br>
> > > On Jan 24 9:01+0000, Darren O'Connor wrote:<br>
> > > > Most interfaces have a single IP, some have 2. No more than that<br>
> > > ><span class="ecxApple-converted-space"> </span><br>
> > > > Thanks<br>
> > > > Darren<br>
> > > > <a href="http://www.mellowd.co.uk/ccie" target="_blank">http://www.mellowd.co.uk/ccie</a><br>
> > > ><span class="ecxApple-converted-space"> </span><br>
> > > ><span class="ecxApple-converted-space"> </span><br>
> > > ><span class="ecxApple-converted-space"> </span><br>
> > > > Subject: Re: [f-nsp] exceed configured CAM size, larger partition<span class="ecxApple-converted-space"> </span><br>
> > > > size required<br>
> > > > From: <a href="mailto:mtindle@he.net">mtindle@he.net</a><br>
> > > > Date: Thu, 23 Jan 2014 15:54:25 -0800<br>
> > > > CC: <a href="mailto:foundry-nsp@puck.nether.net">foundry-nsp@puck.nether.net</a><br>
> > > > To: <a href="mailto:darrenoc@outlook.com">darrenoc@outlook.com</a><br>
> > > ><span class="ecxApple-converted-space"> </span><br>
> > > > Check if you have a lot of IP addresses configured on interfaces. The rACL has to be applied for each inbound IP address the router could be listening on. The limited CAM size for rACLs can have an impact if there are a lot of IPs and the ACL is long.<span class="ecxApple-converted-space"> </span><br>
> > > > Regards,Mike<br>
> > > ><span class="ecxApple-converted-space"> </span><br>
> > > > On Jan 23, 2014, at 8:14 AM, Darren O'Connor <<a href="mailto:darrenoc@outlook.com">darrenoc@outlook.com</a>> wrote:<br>
> > > ><span class="ecxApple-converted-space"> </span><br>
> > > ><span class="ecxApple-converted-space"> </span><br>
> > > > Last weekend we added new receive ACLs to our XMRs. All our XMRs (4, 8, and 16) have identical TCAM profiles set up.<br>
> > > ><span class="ecxApple-converted-space"> </span><br>
> > > > I had applied the new receive ACL to 4 XMR4s with no problems. When applying it to an XMR16 in-band I lost connection to the box. Going through OOB I removed and re-added the ACL. I was shown this error:<br>
> > > ><span class="ecxApple-converted-space"> </span><br>
> > > > Port 16/1, IP Receive ACL 199 exceed configured CAM size, larger partition size required.<br>
> > > > Unbinding IP Receive ACL 199<br>
> > > > Port 2/1, IP Receive ACL 199 exceed configured CAM size, larger partition size required.<br>
> > > > Unbinding IP Receive ACL 199<br>
> > > > Port 9/1, IP Receive ACL 199 exceed configured CAM size, larger partition size required.<br>
> > > > Unbinding IP Receive ACL 199<br>
> > > > Port 6/1, IP Receive ACL 199 exceed configured CAM size, larger partition size required.<br>
> > > > Unbinding IP Receive ACL 199<br>
> > > > Port 5/1, IP Receive ACL 199 exceed configured CAM size, larger partition size required.<br>
> > > > Unbinding IP Receive ACL 199<br>
> > > > Port 3/1, IP Receive ACL 199 exceed configured CAM size, larger partition size required.<br>
> > > > Unbinding IP Receive ACL 199<br>
> > > > Port 1/1, IP Receive ACL 199 exceed configured CAM size, larger partition size required.<br>
> > > > Unbinding IP Receive ACL 199<br>
> > > > Port 4/1, IP Receive ACL 199 exceed configured CAM size, larger partition size required.<br>
> > > > Unbinding IP Receive ACL 199<br>
> > > ><span class="ecxApple-converted-space"> </span><br>
> > > > Port 16/3, IP Receive ACL 180 exceed configured CAM size, larger partition size required.<br>
> > > > Unbinding IP Receive ACL 180<br>
> > > ><span class="ecxApple-converted-space"> </span><br>
> > > ><span class="ecxApple-converted-space"> </span><br>
> > > > Odd, as mentioned all my cam-partitions are identical across all boxes. After this happened I did not try and add it to any other box as it was too disruptive.<br>
> > > ><span class="ecxApple-converted-space"> </span><br>
> > > > Any ideas why I would get this? Currently on 5.4d and was upgrading to 5.4e on the night.<br>
> > > ><span class="ecxApple-converted-space"> </span><br>
> > > ><span class="ecxApple-converted-space"> </span><br>
> > > > Thanks<br>
> > > > Darren<br>
> > > > <a href="http://www.mellowd.co.uk/ccie" target="_blank">http://www.mellowd.co.uk/ccie</a><br>
> > > ><span class="ecxApple-converted-space"> </span><br>
> > > ><span class="ecxApple-converted-space"> </span><br>
> > > > _______________________________________________<br>
> > > > foundry-nsp mailing list<br>
> > > > <a href="mailto:foundry-nsp@puck.nether.net">foundry-nsp@puck.nether.net</a><br>
> > > > <a href="http://puck.nether.net/mailman/listinfo/foundry-nsp" target="_blank">http://puck.nether.net/mailman/listinfo/foundry-nsp</a><br>
> > > ><span class="ecxApple-converted-space"> </span><br>
> > > > *----------- H U R R I C A N E - E L E C T R I C ---------->><br>
> > > > | Mike Tindle | Senior Network Engineer | <a href="mailto:mtindle@he.net">
mtindle@he.net</a> ASN 6939<span class="ecxApple-converted-space"> </span><br>
> > > > | | <a href="http://www.he.net" target="_blank">http://www.he.net</a> | 510-580-4126<br>
> > > > *--------------------------------------------------->><br>
> > > ><span class="ecxApple-converted-space"> </span><br>
> > > ><span class="ecxApple-converted-space"> </span><br>
> > > ><span class="ecxApple-converted-space"> </span><br>
> > ><span class="ecxApple-converted-space"> </span><br>
> > > > _______________________________________________<br>
> > > > foundry-nsp mailing list<br>
> > > > <a href="mailto:foundry-nsp@puck.nether.net">foundry-nsp@puck.nether.net</a><br>
> > > > <a href="http://puck.nether.net/mailman/listinfo/foundry-nsp" target="_blank">http://puck.nether.net/mailman/listinfo/foundry-nsp</a><br>
> > ><span class="ecxApple-converted-space"> </span><br>
> ><span class="ecxApple-converted-space"> </span><br>
> ><span class="ecxApple-converted-space"> </span><br>
> > _______________________________________________<br>
> > foundry-nsp mailing list<br>
> > <a href="mailto:foundry-nsp@puck.nether.net">foundry-nsp@puck.nether.net</a><br>
> > <a href="http://puck.nether.net/mailman/listinfo/foundry-nsp" target="_blank">http://puck.nether.net/mailman/listinfo/foundry-nsp</a><br>
> ><span class="ecxApple-converted-space"> </span><br>
> _______________________________________________<br>
> foundry-nsp mailing list<br>
> <a href="mailto:foundry-nsp@puck.nether.net">foundry-nsp@puck.nether.net</a><br>
> <a href="http://puck.nether.net/mailman/listinfo/foundry-nsp" target="_blank">http://puck.nether.net/mailman/listinfo/foundry-nsp</a></div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div></div> </div></body>
</html>