[j-nsp] Cannot program filter pfe-cos-cl-631-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries
Saku Ytti
saku at ytti.fi
Thu Oct 13 02:39:42 EDT 2022
You chose a filter which doesn't seem to complain about TCAM in the
initial post. Two filters just state 'not programmed' Others about
TCAM? Could you choose another filter which complains about TCAM?
But certainly that output confirms 'Programmed: NO', just not entirely
clear why. Maybe TCAM issue, maybe invalid bind-point, maybe invalid
match, maybe invalid action. I am not familiar with the VFP_IL2L3_COS
type filter.
Is this filter you created? What are the terms you expect it to have?
Single term to accept ether-type 0x8100? What actions? What is the
bind point?
On Wed, 12 Oct 2022 at 21:36, Chuck Anderson <cra at fea.st> wrote:
>
> On Wed, Oct 12, 2022 at 08:40:46AM +0300, Saku Ytti wrote:
> > - show filter dram
> > - show filter hw X
> > - show filter hw X show_term_info
> >
> > I lost a fight with JTAC about whether the TCAM exhausting filter
> > should be a commit failure or not. Argument was along the line 'well
> > you can keep adding routes even if you exhaust TCAM, so this should be
> > the same'.
> > I'm absolutely certain there are many QFX and EX networks out there
> > with wildly different filters programmed than what they believe they
> > have.
>
> Switching platform (2199 Mhz Pentium processor, 511MB memory, 0KB flash)
>
> FPC0(ex4300-48mp vty)# show filter dram
> Name Bytes Allocs Frees Failures
> ---------------------------------------------------------------------------
> filter 62940 1198 395 0
> filter-halp 0 0 0 0
> ---------------------------------------------------------------------------
>
> Total DFW Dram Usage obtained from global handle:
> Total DFW Dram Usage: 78680 bytes
> Total DFW allocs: 740
> Total DFW frees: 0
> Outstanding DFW allocs: 740
>
> Total DFW Dram Usage obtained from all filters:
> Total DFW Dram Usage: 78704 bytes
> Total DFW allocs: 740
> Total DFW frees: 0
> Outstanding DFW allocs: 740
>
> FPC0(ex4300-48mp vty)#
> FPC0(ex4300-48mp vty)# show filter
> Program Filters:
> ---------------
> Index Dir Cnt Text Bss Name
> -------- ------ ------ ------ ------ --------
>
> Term Filters:
> ------------
> Index Semantic Name
> -------- ----------------
> 1 Classic ROUTING-ENGINE
> 2 Classic ROUTING-ENGINE6
> 3 Classic ACCESS-FILTER
> 17000 Classic __default_arp_policer__
> 57006 Classic __jdhcpd__
> 57007 Classic __dhcpv6__
> 65008 Classic __jdhcpd_l2_snoop_filter__
> 16777216 Classic fnp-filter-level-all
> 46137360 Classic pfe-cos-cl-610-5-1
> 46137361 Classic pfe-cos-cl-611-5-1
> 46137362 Classic pfe-cos-cl-612-5-1
> 46137363 Classic pfe-cos-cl-613-5-1
> 46137364 Classic pfe-cos-cl-614-5-1
> 46137365 Classic pfe-cos-cl-615-5-1
> 46137366 Classic pfe-cos-cl-616-5-1
> 46137367 Classic pfe-cos-cl-617-5-1
> 46137368 Classic pfe-cos-cl-618-5-1
> 46137369 Classic pfe-cos-cl-619-5-1
> 46137370 Classic pfe-cos-cl-620-5-1
> 46137371 Classic pfe-cos-cl-621-5-1
> 46137372 Classic pfe-cos-cl-622-5-1
> 46137373 Classic pfe-cos-cl-623-5-1
> 46137374 Classic pfe-cos-cl-624-5-1
> 46137375 Classic pfe-cos-cl-625-5-1
> 46137376 Classic pfe-cos-cl-626-5-1
> 46137377 Classic pfe-cos-cl-627-5-1
> 46137378 Classic pfe-cos-cl-628-5-1
> 46137379 Classic pfe-cos-cl-629-5-1
> 46137380 Classic pfe-cos-cl-630-5-1
> 46137381 Classic pfe-cos-cl-631-5-1
> 46137382 Classic pfe-cos-cl-632-5-1
> 46137383 Classic pfe-cos-cl-633-5-1
> 46137384 Classic pfe-cos-cl-634-5-1
> 46137385 Classic pfe-cos-cl-635-5-1
> 46137386 Classic pfe-cos-cl-636-5-1
> 46137387 Classic pfe-cos-cl-637-5-1
> 46137388 Classic pfe-cos-cl-638-5-1
> 46137389 Classic pfe-cos-cl-639-5-1
> 46137390 Classic pfe-cos-cl-640-5-1
> 46137391 Classic pfe-cos-cl-641-5-1
> 46137392 Classic pfe-cos-cl-642-5-1
> 46137393 Classic pfe-cos-cl-643-5-1
> 46137394 Classic pfe-cos-cl-644-5-1
> 46137395 Classic pfe-cos-cl-645-5-1
> 46137396 Classic pfe-cos-cl-646-5-1
> 46137397 Classic pfe-cos-cl-647-5-1
> 46137398 Classic pfe-cos-cl-648-5-1
> 46137399 Classic pfe-cos-cl-649-5-1
> 46137400 Classic pfe-cos-cl-656-5-1
> 46137401 Classic pfe-cos-cl-657-5-1
> 46137402 Classic pfe-cos-cl-658-5-1
> 46137403 Classic pfe-cos-cl-655-5-1
> 46137404 Classic pfe-cos-cl-650-5-1
> 46137405 Classic pfe-cos-cl-651-5-1
> 46137406 Classic pfe-cos-cl-652-5-1
> 46137407 Classic pfe-cos-cl-653-5-1
> 46137408 Classic pfe-cos-cl-654-5-1
> 92274688 Classic __jdhcpd_l2_dai_filter__
> 125829120 Classic __jdhcpd_security_dhcpv6_l2_snoop_filter__
> 130023424 Classic __jdhcpd_security_icmpv6_l2_snoop_filter__
> 142606336 Classic __jdhcpd_l3_tag__
> 142606337 Classic __dhcpv6_l3_tag__
>
> Resolve Filters:
> ---------------
> Index
> --------
>
> FPC0(ex4300-48mp vty)# show filter hw 46137374
> ======================
> Filter index : 46137374
> ======================
>
> - Filter name : pfe-cos-cl-624-5-1
>
> + Hardware Instance : 1
> + Hardware key (struct brcm_dfw_hw_key_t):
> - Type : VFP_IL2L3_COS
> - Vlan id : 0
> - Direction : ingress
> - Protocol : 35 (L2 Bridge)
> - Port class id : 0
> - Class id : 0
> - Loopback : 0
> - Port : 0(xe-1)
> - Vlan tag : 0
> - Non-overflow : 1
> + FP usage info (struct brcm_dfw_fp_t):
> - Group : VFP IL2L3 COS group (143)
> - My Mac : 00:00:00:00:00:00
> - Loopback Reference Count : 00000000
> - IFL Type : unknown (0)
> + List of tcam entries : [ total: 0; ]
> - Pipe: 0; []
> + List of ranges : [ total: 0; ]
> - Pipe: 0 []
> + List of interface match entries : [ total: 0; ]
> - Pipe: 0 []
> + List of dot1q-tag match entries : [ total: 0; ]
> - Pipe: 0 []
> - List of l3 ifl index entries : [ total: 0; ]
> + List of vfp tcam entries : [ total: 0; ]
> - Pipe: 0 []
> + Misc info (struct brcm_dfw_misc_info_t):
> - List of <anlz_id, entry_id> : [ total: 0; ]
> + Bind point info (union brcm_dfw_bind_point_info_t):
> - No grouping possible
> + Programmed: NO
> + BD ID : 500
> + Total TCAM entries available: 512
> + Total TCAM entries needed : 1
> + Term Expansion:
> - Term 1: will expand to 1 term : Name "cos-cl-624-5-1"
> + Term TCAM entry requirements:
> - Term 1: needs 1 TCAM entry : Name "cos-cl-624-5-1"
> + Total TCAM entries available: 512
> + Total TCAM entries needed : 1
>
> Total hardware instances: 1
>
> FPC0(ex4300-48mp vty)#
> FPC0(ex4300-48mp vty)# show filter hw 46137374 show_term_info
> ======================
> Filter index : 46137374
> ======================
>
> - Filter name : pfe-cos-cl-624-5-1
>
> + Hardware Instance : 1
> + Hardware key (struct brcm_dfw_hw_key_t):
> - Type : VFP_IL2L3_COS
> - Vlan id : 0
> - Direction : ingress
> - Protocol : 35 (L2 Bridge)
> - Port class id : 0
> - Class id : 0
> - Loopback : 0
> - Port : 0(xe-1)
> - Vlan tag : 0
> - Non-overflow : 1
> + FP usage info (struct brcm_dfw_fp_t):
> - Group : VFP IL2L3 COS group (143)
> - My Mac : 00:00:00:00:00:00
> - Loopback Reference Count : 00000000
> - IFL Type : unknown (0)
> + List of tcam entries : [ total: 0; ]
> - Pipe: 0; []
> + List of ranges : [ total: 0; ]
> - Pipe: 0 []
> + List of interface match entries : [ total: 0; ]
> - Pipe: 0 []
> + List of dot1q-tag match entries : [ total: 0; ]
> - Pipe: 0 []
> - List of l3 ifl index entries : [ total: 0; ]
> + List of vfp tcam entries : [ total: 0; ]
> - Pipe: 0 []
> + Misc info (struct brcm_dfw_misc_info_t):
> - List of <anlz_id, entry_id> : [ total: 0; ]
> + Bind point info (union brcm_dfw_bind_point_info_t):
> - No grouping possible
> + Programmed: NO
> + BD ID : 500
> + Total TCAM entries available: 512
> + Total TCAM entries needed : 1
> + Term Expansion:
> - Term 1: will expand to 1 term : Name "cos-cl-624-5-1"
> + Term TCAM entry requirements:
> - Term 1: needs 1 TCAM entry : Name "cos-cl-624-5-1"
> + Total TCAM entries available: 512
> + Total TCAM entries needed : 1
>
> Total hardware instances: 1
>
>
>
> FPC0(ex4300-48mp vty)# show filter hw groups
> Unit:0 Group Information:
> > VFP groups:
> VFP group for DYN IPSG group id: 231. Pipe: 0 Entries: 238 Max Entries(total_available): 256(1024) Pri: 3 Slice: 2 Def Entries: 0
> > IFP groups:
> iRACL group id: 33. Pipe: 0 Entries: 1148 Max Entries(total_available): 2048(2560) Pri: 6 Slice: 4 Def Entries: 0
> iPACL group id: 25. Pipe: 0 Entries: 12 Max Entries(total_available): 512(1024) Pri: 8 Slice: 2 Def Entries: 0
> Dynamic group id: 17. Pipe: 0 Entries: 141 Max Entries(total_available): 512(1024) Pri: 5 Slice: 2 Def Entries: 0
> Dynamic HiGig group id: 37. Pipe: 0 Entries: 46 Max Entries(total_available): 512(2048) Pri: 4 Slice: 1 Def Entries: 0
> > EFP groups:
>
> FPC0(ex4300-48mp vty)# show filter summary
> Filter Summary:
> Number of Program Filters = 0
> Number of Term Filters = 62
> Number of Resolve Filters = 0
> Number of Simple Filters = 0
> Number of Implicit Filters = 58
> Number of Other Filters = 0
> Number of cached non-local Filters = 0
> Number of post cli Implicit Filters = 0
> Number of service filter templates = 0
>
> Total Number of Filters = 62
>
> Protocol Filters:
> Number of PROTO_IP Filters = 4
> Number of PROTO_IP6 Filters = 3
> Number of PROTO_MPLS Filters = 0
> Number of PROTO_VPLS Filters = 55
> Number of PROTO_CCC Filters = 0
> Number of PROTO_ANY Filters = 0
> Number of PROTO_BRIDGE Filters = 0
> Number of Filters with unknown protocols = 0
>
> Application Filters:
> Number of Implicit Filters from OAM = 0
> Number of Implicit Filters from JDHCPD = 2
> Number of Implicit Filters from L2CPD = 1
> Number of Implicit Filters from DHCP L2 = 1
> Number of Implicit Filters from PGCPD_COMPILED = 0
> Number of Implicit Filters from PGCPD_FUF = 0
> Number of Implicit Filters from FTAPLITE = 0
> Number of Implicit Filters from SA_VAL = 0
> Number of Implicit Filters from JDHCPD_L2_FUF = 0
> Number of Implicit Filters from SDK1 = 0
> Number of Implicit Filters from SDK2 = 0
> Number of Implicit Filters from SDK3 = 0
> Number of Implicit Filters from SDK4 = 0
> Number of Implicit Filters from SDK5 = 0
> Number of Implicit Filters from TEST1 = 0
> Number of Implicit Filters from TEST2 = 0
> Number of Implicit Filters from DOT1XD = 0
> Number of Implicit Filters from DYNAMIC SERVICES = 0
> Number of Implicit Filters from LPDFD = 0
> Number of Implicit Filters from Propagated Config = 0
> Number of Implicit Filters from RPD = 0
> Number of Implicit Filters from unknown = 0
> Number of Implicit Filters from unknown = 49
> Number of Implicit Filters from LI_DFCD = 0
> Number of Implicit Filters from ESWD = 0
> Number of Implicit Filters from ESWD EVB = 0
> Number of Implicit Filters from PPMD = 0
> Number of Implicit Filters from SPD = 0
> Number of Implicit Filters from RMOPD = 0
> Number of Implicit Filters from unknown = 0
> Number of Implicit Filters from UACD = 0
> Number of Implicit Filters from MCOS = 0
> Number of Implicit Filters from RMPSD = 0
> Number of Implicit Filters from DHCP L2 DAI = 1
> Number of Implicit Filters from OPENFLOWD = 0
> Number of Implicit Filters from VMOND = 0
> Number of Implicit Filters from DHCP IPV6 = 1
> Number of Implicit Filters from DHCP ICMPV6 = 1
> Number of Implicit Filters from JDHCPD L3 TAG = 2
> Number of Implicit Filters from BBE1 = 0
> Number of Implicit Filters from BBE1_LI = 0
> Number of Implicit Filters from BBE1_BGF = 0
> Number of Implicit Filters from URL-FILTERD = 0
> Number of Implicit Filters from URL 2 FILTERD = 0
> Number of Implicit Filters from TWAMP = 0
> Number of Implicit Filters from CAED-GLOBAL = 0
> Number of Implicit Filters from IDS_FILTER = 0
> Number of Implicit Filters from RTLOGD = 0
> Number of Implicit Filters from CAED-LOCAL = 0
>
>
> FPC0(ex4300-48mp vty)# show filter statistcs
> Filter Manager Statistics:
> 0 program filter add
> 0 program filter delete
> 0 program filter change
> 0 filter log get
> 2291 filter counter get
> 0 filter counter reset
> 0 filter log clear
> 0 filter dcu stats get
> 0 filter scu stats get
> 0 rpf stats get
> 0 prefix action stats get
> 0 prefix action stats clear
> 385 filter apply
> 2 policer
> 0 three-color policer
> 0 hierarchical policer
> 0 global msg
> 0 prefix specifc action
> 0 timer
> 0 notify owner id
> 13 filter head
> 47 filter term
> 14 filter state
> 0 multicast statistics support
>
> Filter Manager Errors:
> 0 program filter add
> 0 program filter delete
> 0 program filter change
> 0 filter log get
> 0 filter counter get
> 0 filter counter reset
> 0 filter log clear
> 0 filter dcu stats get
> 0 filter scu stats get
> 0 rpf stats get
> 0 prefix action stats get
> 0 prefix action stats clear
> 0 filter apply
> 0 policer
> 0 three-color policer
> 0 hierarchical policer
> 0 global msg
> 0 prefix specifc action
> 0 timer
> 0 notify owner id
> 0 filter head
> 0 filter term
> 0 filter state
> 0 multicast statistics support
>
> More details extracted from the full "show filter hw all" output:
>
> ======================
> Filter index : 46137374
> ======================
>
> - Filter name : pfe-cos-cl-624-5-1
> Filter index = 46137374
> Optimization flag: 0x0
> Filter notify host id = 0
> Pfe Mask = 0x0
> jnh inst = 0x0
> Filter properties: None
> Filter state = CONSISTENT
> term cos-cl-624-5-1
> term priority 0
> ethertype
> 33024
>
> then
> accept
>
> + Hardware Instance : 1
> + Hardware key (struct brcm_dfw_hw_key_t):
> - Type : VFP_IL2L3_COS
> - Vlan id : 0
> - Direction : ingress
> - Protocol : 35 (L2 Bridge)
> - Port class id : 0
> - Class id : 0
> - Loopback : 0
> - Port : 0(xe-1)
> - Vlan tag : 0
> - Non-overflow : 1
> + FP usage info (struct brcm_dfw_fp_t):
> - Group : VFP IL2L3 COS group (143)
> - My Mac : 00:00:00:00:00:00
> - Loopback Reference Count : 00000000
> - IFL Type : unknown (0)
> + List of tcam entries : [ total: 0; ]
> - Pipe: 0; []
> + List of ranges : [ total: 0; ]
> - Pipe: 0 []
> + List of interface match entries : [ total: 0; ]
> - Pipe: 0 []
> + List of dot1q-tag match entries : [ total: 0; ]
> - Pipe: 0 []
> - List of l3 ifl index entries : [ total: 0; ]
> + List of vfp tcam entries : [ total: 0; ]
> - Pipe: 0 []
> + Misc info (struct brcm_dfw_misc_info_t):
> - List of <anlz_id, entry_id> : [ total: 0; ]
> + Bind point info (union brcm_dfw_bind_point_info_t):
> - No grouping possible
> + Programmed: NO
> + BD ID : 500
> + Total TCAM entries available: 512
> + Total TCAM entries needed : 1
> + Term Expansion:
> - Term 1: will expand to 1 term : Name "cos-cl-624-5-1"
> + Term TCAM entry requirements:
> - Term 1: needs 1 TCAM entry : Name "cos-cl-624-5-1"
> + Total TCAM entries available: 512
> + Total TCAM entries needed : 1
>
> Total hardware instances: 1
> ======================
>
>
> > On Wed, 12 Oct 2022 at 05:33, Chuck Anderson via juniper-nsp
> > <juniper-nsp at puck.nether.net> wrote:
> > >
> > > Has anyone seen these errors and know what the cause is?
> > >
> > > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : "pfe-cos-cl-624-5-1" is NOT programmed in HW
> > > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : "pfe-cos-cl-626-5-1" is NOT programmed in HW
> > > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter pfe-cos-cl-631-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries
> > > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : "pfe-cos-cl-631-5-1" is NOT programmed in HW
> > > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter pfe-cos-cl-632-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries
> > > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : "pfe-cos-cl-632-5-1" is NOT programmed in HW
> > > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter pfe-cos-cl-633-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries
> > > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : "pfe-cos-cl-633-5-1" is NOT programmed in HW
> > > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter pfe-cos-cl-634-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries
> > > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : "pfe-cos-cl-634-5-1" is NOT programmed in HW
> > > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter pfe-cos-cl-638-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries
> > > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : "pfe-cos-cl-638-5-1" is NOT programmed in HW
> > > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter pfe-cos-cl-647-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries
> > > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : "pfe-cos-cl-647-5-1" is NOT programmed in HW
> > > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter pfe-cos-cl-656-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries
> > > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : "pfe-cos-cl-656-5-1" is NOT programmed in HW
> > > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter pfe-cos-cl-657-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries
> > > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : "pfe-cos-cl-657-5-1" is NOT programmed in HW
> > > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter pfe-cos-cl-655-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries
> > > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter pfe-cos-cl-652-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries
> > > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : "pfe-cos-cl-652-5-1" is NOT programmed in HW
> > > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter pfe-cos-cl-653-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries
> > > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : "pfe-cos-cl-653-5-1" is NOT programmed in HW
> > > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter pfe-cos-cl-654-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries
> > > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : "pfe-cos-cl-654-5-1" is NOT programmed in HW
> > >
> > > There is plenty of TCAM space for IRACL/IPACL entries, so this seems to be some issue with a different TCAM partition?
> > >
> > > ex4300-48mp> show pfe filter hw summary
> > >
> > > Slot 0
> > >
> > > Unit:0:
> > > Group Group-ID Allocated Used Free
> > > ---------------------------------------------------------------------------
> > > > Ingress filter groups:
> > > iRACL group 33 2048 1148 900
> > > iPACL group 25 512 12 500
> > > > Egress filter groups:
> > >
> > > Slot 1
> > >
> > > Unit:0:
> > > Group Group-ID Allocated Used Free
> > > ---------------------------------------------------------------------------
> > > > Ingress filter groups:
> > > iRACL group 33 2048 1148 900
> > > iPACL group 25 512 12 500
> > > > Egress filter groups:
--
++ytti
More information about the juniper-nsp
mailing list