[j-nsp] Cannot program filter pfe-cos-cl-631-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries
Chuck Anderson
cra at fea.st
Wed Oct 12 14:36:02 EDT 2022
On Wed, Oct 12, 2022 at 08:40:46AM +0300, Saku Ytti wrote:
> - show filter dram
> - show filter hw X
> - show filter hw X show_term_info
>
> I lost a fight with JTAC about whether the TCAM exhausting filter
> should be a commit failure or not. Argument was along the line 'well
> you can keep adding routes even if you exhaust TCAM, so this should be
> the same'.
> I'm absolutely certain there are many QFX and EX networks out there
> with wildly different filters programmed than what they believe they
> have.
Switching platform (2199 Mhz Pentium processor, 511MB memory, 0KB flash)
FPC0(ex4300-48mp vty)# show filter dram
Name Bytes Allocs Frees Failures
---------------------------------------------------------------------------
filter 62940 1198 395 0
filter-halp 0 0 0 0
---------------------------------------------------------------------------
Total DFW Dram Usage obtained from global handle:
Total DFW Dram Usage: 78680 bytes
Total DFW allocs: 740
Total DFW frees: 0
Outstanding DFW allocs: 740
Total DFW Dram Usage obtained from all filters:
Total DFW Dram Usage: 78704 bytes
Total DFW allocs: 740
Total DFW frees: 0
Outstanding DFW allocs: 740
FPC0(ex4300-48mp vty)#
FPC0(ex4300-48mp vty)# show filter
Program Filters:
---------------
Index Dir Cnt Text Bss Name
-------- ------ ------ ------ ------ --------
Term Filters:
------------
Index Semantic Name
-------- ----------------
1 Classic ROUTING-ENGINE
2 Classic ROUTING-ENGINE6
3 Classic ACCESS-FILTER
17000 Classic __default_arp_policer__
57006 Classic __jdhcpd__
57007 Classic __dhcpv6__
65008 Classic __jdhcpd_l2_snoop_filter__
16777216 Classic fnp-filter-level-all
46137360 Classic pfe-cos-cl-610-5-1
46137361 Classic pfe-cos-cl-611-5-1
46137362 Classic pfe-cos-cl-612-5-1
46137363 Classic pfe-cos-cl-613-5-1
46137364 Classic pfe-cos-cl-614-5-1
46137365 Classic pfe-cos-cl-615-5-1
46137366 Classic pfe-cos-cl-616-5-1
46137367 Classic pfe-cos-cl-617-5-1
46137368 Classic pfe-cos-cl-618-5-1
46137369 Classic pfe-cos-cl-619-5-1
46137370 Classic pfe-cos-cl-620-5-1
46137371 Classic pfe-cos-cl-621-5-1
46137372 Classic pfe-cos-cl-622-5-1
46137373 Classic pfe-cos-cl-623-5-1
46137374 Classic pfe-cos-cl-624-5-1
46137375 Classic pfe-cos-cl-625-5-1
46137376 Classic pfe-cos-cl-626-5-1
46137377 Classic pfe-cos-cl-627-5-1
46137378 Classic pfe-cos-cl-628-5-1
46137379 Classic pfe-cos-cl-629-5-1
46137380 Classic pfe-cos-cl-630-5-1
46137381 Classic pfe-cos-cl-631-5-1
46137382 Classic pfe-cos-cl-632-5-1
46137383 Classic pfe-cos-cl-633-5-1
46137384 Classic pfe-cos-cl-634-5-1
46137385 Classic pfe-cos-cl-635-5-1
46137386 Classic pfe-cos-cl-636-5-1
46137387 Classic pfe-cos-cl-637-5-1
46137388 Classic pfe-cos-cl-638-5-1
46137389 Classic pfe-cos-cl-639-5-1
46137390 Classic pfe-cos-cl-640-5-1
46137391 Classic pfe-cos-cl-641-5-1
46137392 Classic pfe-cos-cl-642-5-1
46137393 Classic pfe-cos-cl-643-5-1
46137394 Classic pfe-cos-cl-644-5-1
46137395 Classic pfe-cos-cl-645-5-1
46137396 Classic pfe-cos-cl-646-5-1
46137397 Classic pfe-cos-cl-647-5-1
46137398 Classic pfe-cos-cl-648-5-1
46137399 Classic pfe-cos-cl-649-5-1
46137400 Classic pfe-cos-cl-656-5-1
46137401 Classic pfe-cos-cl-657-5-1
46137402 Classic pfe-cos-cl-658-5-1
46137403 Classic pfe-cos-cl-655-5-1
46137404 Classic pfe-cos-cl-650-5-1
46137405 Classic pfe-cos-cl-651-5-1
46137406 Classic pfe-cos-cl-652-5-1
46137407 Classic pfe-cos-cl-653-5-1
46137408 Classic pfe-cos-cl-654-5-1
92274688 Classic __jdhcpd_l2_dai_filter__
125829120 Classic __jdhcpd_security_dhcpv6_l2_snoop_filter__
130023424 Classic __jdhcpd_security_icmpv6_l2_snoop_filter__
142606336 Classic __jdhcpd_l3_tag__
142606337 Classic __dhcpv6_l3_tag__
Resolve Filters:
---------------
Index
--------
FPC0(ex4300-48mp vty)# show filter hw 46137374
======================
Filter index : 46137374
======================
- Filter name : pfe-cos-cl-624-5-1
+ Hardware Instance : 1
+ Hardware key (struct brcm_dfw_hw_key_t):
- Type : VFP_IL2L3_COS
- Vlan id : 0
- Direction : ingress
- Protocol : 35 (L2 Bridge)
- Port class id : 0
- Class id : 0
- Loopback : 0
- Port : 0(xe-1)
- Vlan tag : 0
- Non-overflow : 1
+ FP usage info (struct brcm_dfw_fp_t):
- Group : VFP IL2L3 COS group (143)
- My Mac : 00:00:00:00:00:00
- Loopback Reference Count : 00000000
- IFL Type : unknown (0)
+ List of tcam entries : [ total: 0; ]
- Pipe: 0; []
+ List of ranges : [ total: 0; ]
- Pipe: 0 []
+ List of interface match entries : [ total: 0; ]
- Pipe: 0 []
+ List of dot1q-tag match entries : [ total: 0; ]
- Pipe: 0 []
- List of l3 ifl index entries : [ total: 0; ]
+ List of vfp tcam entries : [ total: 0; ]
- Pipe: 0 []
+ Misc info (struct brcm_dfw_misc_info_t):
- List of <anlz_id, entry_id> : [ total: 0; ]
+ Bind point info (union brcm_dfw_bind_point_info_t):
- No grouping possible
+ Programmed: NO
+ BD ID : 500
+ Total TCAM entries available: 512
+ Total TCAM entries needed : 1
+ Term Expansion:
- Term 1: will expand to 1 term : Name "cos-cl-624-5-1"
+ Term TCAM entry requirements:
- Term 1: needs 1 TCAM entry : Name "cos-cl-624-5-1"
+ Total TCAM entries available: 512
+ Total TCAM entries needed : 1
Total hardware instances: 1
FPC0(ex4300-48mp vty)#
FPC0(ex4300-48mp vty)# show filter hw 46137374 show_term_info
======================
Filter index : 46137374
======================
- Filter name : pfe-cos-cl-624-5-1
+ Hardware Instance : 1
+ Hardware key (struct brcm_dfw_hw_key_t):
- Type : VFP_IL2L3_COS
- Vlan id : 0
- Direction : ingress
- Protocol : 35 (L2 Bridge)
- Port class id : 0
- Class id : 0
- Loopback : 0
- Port : 0(xe-1)
- Vlan tag : 0
- Non-overflow : 1
+ FP usage info (struct brcm_dfw_fp_t):
- Group : VFP IL2L3 COS group (143)
- My Mac : 00:00:00:00:00:00
- Loopback Reference Count : 00000000
- IFL Type : unknown (0)
+ List of tcam entries : [ total: 0; ]
- Pipe: 0; []
+ List of ranges : [ total: 0; ]
- Pipe: 0 []
+ List of interface match entries : [ total: 0; ]
- Pipe: 0 []
+ List of dot1q-tag match entries : [ total: 0; ]
- Pipe: 0 []
- List of l3 ifl index entries : [ total: 0; ]
+ List of vfp tcam entries : [ total: 0; ]
- Pipe: 0 []
+ Misc info (struct brcm_dfw_misc_info_t):
- List of <anlz_id, entry_id> : [ total: 0; ]
+ Bind point info (union brcm_dfw_bind_point_info_t):
- No grouping possible
+ Programmed: NO
+ BD ID : 500
+ Total TCAM entries available: 512
+ Total TCAM entries needed : 1
+ Term Expansion:
- Term 1: will expand to 1 term : Name "cos-cl-624-5-1"
+ Term TCAM entry requirements:
- Term 1: needs 1 TCAM entry : Name "cos-cl-624-5-1"
+ Total TCAM entries available: 512
+ Total TCAM entries needed : 1
Total hardware instances: 1
FPC0(ex4300-48mp vty)# show filter hw groups
Unit:0 Group Information:
> VFP groups:
VFP group for DYN IPSG group id: 231. Pipe: 0 Entries: 238 Max Entries(total_available): 256(1024) Pri: 3 Slice: 2 Def Entries: 0
> IFP groups:
iRACL group id: 33. Pipe: 0 Entries: 1148 Max Entries(total_available): 2048(2560) Pri: 6 Slice: 4 Def Entries: 0
iPACL group id: 25. Pipe: 0 Entries: 12 Max Entries(total_available): 512(1024) Pri: 8 Slice: 2 Def Entries: 0
Dynamic group id: 17. Pipe: 0 Entries: 141 Max Entries(total_available): 512(1024) Pri: 5 Slice: 2 Def Entries: 0
Dynamic HiGig group id: 37. Pipe: 0 Entries: 46 Max Entries(total_available): 512(2048) Pri: 4 Slice: 1 Def Entries: 0
> EFP groups:
FPC0(ex4300-48mp vty)# show filter summary
Filter Summary:
Number of Program Filters = 0
Number of Term Filters = 62
Number of Resolve Filters = 0
Number of Simple Filters = 0
Number of Implicit Filters = 58
Number of Other Filters = 0
Number of cached non-local Filters = 0
Number of post cli Implicit Filters = 0
Number of service filter templates = 0
Total Number of Filters = 62
Protocol Filters:
Number of PROTO_IP Filters = 4
Number of PROTO_IP6 Filters = 3
Number of PROTO_MPLS Filters = 0
Number of PROTO_VPLS Filters = 55
Number of PROTO_CCC Filters = 0
Number of PROTO_ANY Filters = 0
Number of PROTO_BRIDGE Filters = 0
Number of Filters with unknown protocols = 0
Application Filters:
Number of Implicit Filters from OAM = 0
Number of Implicit Filters from JDHCPD = 2
Number of Implicit Filters from L2CPD = 1
Number of Implicit Filters from DHCP L2 = 1
Number of Implicit Filters from PGCPD_COMPILED = 0
Number of Implicit Filters from PGCPD_FUF = 0
Number of Implicit Filters from FTAPLITE = 0
Number of Implicit Filters from SA_VAL = 0
Number of Implicit Filters from JDHCPD_L2_FUF = 0
Number of Implicit Filters from SDK1 = 0
Number of Implicit Filters from SDK2 = 0
Number of Implicit Filters from SDK3 = 0
Number of Implicit Filters from SDK4 = 0
Number of Implicit Filters from SDK5 = 0
Number of Implicit Filters from TEST1 = 0
Number of Implicit Filters from TEST2 = 0
Number of Implicit Filters from DOT1XD = 0
Number of Implicit Filters from DYNAMIC SERVICES = 0
Number of Implicit Filters from LPDFD = 0
Number of Implicit Filters from Propagated Config = 0
Number of Implicit Filters from RPD = 0
Number of Implicit Filters from unknown = 0
Number of Implicit Filters from unknown = 49
Number of Implicit Filters from LI_DFCD = 0
Number of Implicit Filters from ESWD = 0
Number of Implicit Filters from ESWD EVB = 0
Number of Implicit Filters from PPMD = 0
Number of Implicit Filters from SPD = 0
Number of Implicit Filters from RMOPD = 0
Number of Implicit Filters from unknown = 0
Number of Implicit Filters from UACD = 0
Number of Implicit Filters from MCOS = 0
Number of Implicit Filters from RMPSD = 0
Number of Implicit Filters from DHCP L2 DAI = 1
Number of Implicit Filters from OPENFLOWD = 0
Number of Implicit Filters from VMOND = 0
Number of Implicit Filters from DHCP IPV6 = 1
Number of Implicit Filters from DHCP ICMPV6 = 1
Number of Implicit Filters from JDHCPD L3 TAG = 2
Number of Implicit Filters from BBE1 = 0
Number of Implicit Filters from BBE1_LI = 0
Number of Implicit Filters from BBE1_BGF = 0
Number of Implicit Filters from URL-FILTERD = 0
Number of Implicit Filters from URL 2 FILTERD = 0
Number of Implicit Filters from TWAMP = 0
Number of Implicit Filters from CAED-GLOBAL = 0
Number of Implicit Filters from IDS_FILTER = 0
Number of Implicit Filters from RTLOGD = 0
Number of Implicit Filters from CAED-LOCAL = 0
FPC0(ex4300-48mp vty)# show filter statistcs
Filter Manager Statistics:
0 program filter add
0 program filter delete
0 program filter change
0 filter log get
2291 filter counter get
0 filter counter reset
0 filter log clear
0 filter dcu stats get
0 filter scu stats get
0 rpf stats get
0 prefix action stats get
0 prefix action stats clear
385 filter apply
2 policer
0 three-color policer
0 hierarchical policer
0 global msg
0 prefix specifc action
0 timer
0 notify owner id
13 filter head
47 filter term
14 filter state
0 multicast statistics support
Filter Manager Errors:
0 program filter add
0 program filter delete
0 program filter change
0 filter log get
0 filter counter get
0 filter counter reset
0 filter log clear
0 filter dcu stats get
0 filter scu stats get
0 rpf stats get
0 prefix action stats get
0 prefix action stats clear
0 filter apply
0 policer
0 three-color policer
0 hierarchical policer
0 global msg
0 prefix specifc action
0 timer
0 notify owner id
0 filter head
0 filter term
0 filter state
0 multicast statistics support
More details extracted from the full "show filter hw all" output:
======================
Filter index : 46137374
======================
- Filter name : pfe-cos-cl-624-5-1
Filter index = 46137374
Optimization flag: 0x0
Filter notify host id = 0
Pfe Mask = 0x0
jnh inst = 0x0
Filter properties: None
Filter state = CONSISTENT
term cos-cl-624-5-1
term priority 0
ethertype
33024
then
accept
+ Hardware Instance : 1
+ Hardware key (struct brcm_dfw_hw_key_t):
- Type : VFP_IL2L3_COS
- Vlan id : 0
- Direction : ingress
- Protocol : 35 (L2 Bridge)
- Port class id : 0
- Class id : 0
- Loopback : 0
- Port : 0(xe-1)
- Vlan tag : 0
- Non-overflow : 1
+ FP usage info (struct brcm_dfw_fp_t):
- Group : VFP IL2L3 COS group (143)
- My Mac : 00:00:00:00:00:00
- Loopback Reference Count : 00000000
- IFL Type : unknown (0)
+ List of tcam entries : [ total: 0; ]
- Pipe: 0; []
+ List of ranges : [ total: 0; ]
- Pipe: 0 []
+ List of interface match entries : [ total: 0; ]
- Pipe: 0 []
+ List of dot1q-tag match entries : [ total: 0; ]
- Pipe: 0 []
- List of l3 ifl index entries : [ total: 0; ]
+ List of vfp tcam entries : [ total: 0; ]
- Pipe: 0 []
+ Misc info (struct brcm_dfw_misc_info_t):
- List of <anlz_id, entry_id> : [ total: 0; ]
+ Bind point info (union brcm_dfw_bind_point_info_t):
- No grouping possible
+ Programmed: NO
+ BD ID : 500
+ Total TCAM entries available: 512
+ Total TCAM entries needed : 1
+ Term Expansion:
- Term 1: will expand to 1 term : Name "cos-cl-624-5-1"
+ Term TCAM entry requirements:
- Term 1: needs 1 TCAM entry : Name "cos-cl-624-5-1"
+ Total TCAM entries available: 512
+ Total TCAM entries needed : 1
Total hardware instances: 1
======================
> On Wed, 12 Oct 2022 at 05:33, Chuck Anderson via juniper-nsp
> <juniper-nsp at puck.nether.net> wrote:
> >
> > Has anyone seen these errors and know what the cause is?
> >
> > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : "pfe-cos-cl-624-5-1" is NOT programmed in HW
> > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : "pfe-cos-cl-626-5-1" is NOT programmed in HW
> > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter pfe-cos-cl-631-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries
> > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : "pfe-cos-cl-631-5-1" is NOT programmed in HW
> > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter pfe-cos-cl-632-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries
> > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : "pfe-cos-cl-632-5-1" is NOT programmed in HW
> > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter pfe-cos-cl-633-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries
> > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : "pfe-cos-cl-633-5-1" is NOT programmed in HW
> > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter pfe-cos-cl-634-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries
> > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : "pfe-cos-cl-634-5-1" is NOT programmed in HW
> > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter pfe-cos-cl-638-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries
> > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : "pfe-cos-cl-638-5-1" is NOT programmed in HW
> > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter pfe-cos-cl-647-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries
> > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : "pfe-cos-cl-647-5-1" is NOT programmed in HW
> > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter pfe-cos-cl-656-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries
> > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : "pfe-cos-cl-656-5-1" is NOT programmed in HW
> > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter pfe-cos-cl-657-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries
> > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : "pfe-cos-cl-657-5-1" is NOT programmed in HW
> > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter pfe-cos-cl-655-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries
> > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter pfe-cos-cl-652-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries
> > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : "pfe-cos-cl-652-5-1" is NOT programmed in HW
> > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter pfe-cos-cl-653-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries
> > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : "pfe-cos-cl-653-5-1" is NOT programmed in HW
> > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Cannot program filter pfe-cos-cl-654-5-1 (type VFP_IL2L3_COS) -TCAM has 0 free entries
> > Oct 11 21:41:02 ex4300-48mp fpc0 DFWE ERROR DFW: Filter : "pfe-cos-cl-654-5-1" is NOT programmed in HW
> >
> > There is plenty of TCAM space for IRACL/IPACL entries, so this seems to be some issue with a different TCAM partition?
> >
> > ex4300-48mp> show pfe filter hw summary
> >
> > Slot 0
> >
> > Unit:0:
> > Group Group-ID Allocated Used Free
> > ---------------------------------------------------------------------------
> > > Ingress filter groups:
> > iRACL group 33 2048 1148 900
> > iPACL group 25 512 12 500
> > > Egress filter groups:
> >
> > Slot 1
> >
> > Unit:0:
> > Group Group-ID Allocated Used Free
> > ---------------------------------------------------------------------------
> > > Ingress filter groups:
> > iRACL group 33 2048 1148 900
> > iPACL group 25 512 12 500
> > > Egress filter groups:
More information about the juniper-nsp
mailing list