[c-nsp] issue on old but running device
Brian Turnbow
brian.turnbow at cdlan.it
Thu Jan 30 17:25:01 UTC 2025
Hi James,
Looks like microburst overdriving the interface to me.
You are seeing rx misses and rx_overruns on internal driver.
There are few misses in the buffers but no failures and I don't think that
buffer tuning will help.
AFAIK you cannot tune the rx ring on the router but I have been proven
wrong before....
Otherwise it could be a software bug so check if the devices' releases are
the same and check on bug tracker.
You could ask for a shaping to be placed upstream or see if flow control
can be used.
Sorry for no good news
Brian
Il giorno gio 30 gen 2025 alle ore 17:08 james list <jameslist72 at gmail.com>
ha scritto:
>
> Hi Brian
> thanks for responding.
>
> I've already done some actions, reboot did not help, the point is that the
> same C3925 getting the same feeds in other location doesn't have the same
> behaviour.
>
> One difference: 3925 having the issue is connected 1 Gbs to a C9407 from
> the market data feed broker, while the 3925 not having the issue is
> connected to an old C4948...
>
> Unfortunately I have already done some of the actions, not tried
> rate-limit to log the exceed:
>
> 1) reboot doesn't helped
> 2) buffer miss doesn't increase a lot
>
> #sh buffer
> Buffer elements:
> 603 in free list
> 96825593 hits, 0 misses, 1242 created
>
> Public buffer pools:
> Small buffers, 104 bytes (total 1024, permanent 1024):
> 1016 in free list (100 min, 2048 max allowed)
> 39265134 hits, 0 misses, 0 trims, 0 created
> 0 failures (0 no memory)
> Middle buffers, 600 bytes (total 1024, permanent 1024):
> 1024 in free list (100 min, 2048 max allowed)
> 12407239 hits, 0 misses, 0 trims, 0 created
> 0 failures (0 no memory)
> Big buffers, 1536 bytes (total 512, permanent 512):
> 512 in free list (100 min, 1024 max allowed)
> 25923483 hits, 0 misses, 0 trims, 0 created
> 0 failures (0 no memory)
> VeryBig buffers, 4520 bytes (total 150, permanent 150, peak 151 @ 7w0d):
> 150 in free list (30 min, 300 max allowed)
> 78341 hits, 0 misses, 1 trims, 1 created
> 0 failures (0 no memory)
> Large buffers, 5024 bytes (total 1, permanent 0, peak 1 @ 7w0d):
> 1 in free list (0 min, 10 max allowed)
> 0 hits, 0 misses, 486 trims, 487 created
> 0 failures (0 no memory)
> Huge buffers, 18024 bytes (total 5, permanent 0, peak 8 @ 7w0d):
> 5 in free list (4 min, 10 max allowed)
> 234041 hits, 972 misses, 2917 trims, 2922 created
> 0 failures (0 no memory)
>
> Interface buffer pools:
> CF Small buffers, 104 bytes (total 101, permanent 100, peak 101 @ 7w0d):
> 101 in free list (100 min, 200 max allowed)
> 0 hits, 0 misses, 486 trims, 487 created
> 0 failures (0 no memory)
> Generic ED Pool buffers, 512 bytes (total 101, permanent 100, peak 101 @
> 7w0d):
> 101 in free list (100 min, 100 max allowed)
> 0 hits, 0 misses
> CF Middle buffers, 600 bytes (total 101, permanent 100, peak 101 @ 7w0d):
> 101 in free list (100 min, 200 max allowed)
> 0 hits, 0 misses, 486 trims, 487 created
> 0 failures (0 no memory)
> Syslog ED Pool buffers, 600 bytes (total 133, permanent 132, peak 133 @
> 7w0d):
> 101 in free list (132 min, 132 max allowed)
> 318 hits, 0 misses
> CF Big buffers, 1536 bytes (total 26, permanent 25, peak 26 @ 7w0d):
> 26 in free list (25 min, 50 max allowed)
> 0 hits, 0 misses, 486 trims, 487 created
> 0 failures (0 no memory)
> IPC buffers, 4096 bytes (total 2, permanent 2):
> 1 in free list (1 min, 8 max allowed)
> 1 hits, 0 fallbacks, 0 trims, 0 created
> 0 failures (0 no memory)
> CF VeryBig buffers, 4520 bytes (total 3, permanent 2, peak 3 @ 7w0d):
> 3 in free list (2 min, 4 max allowed)
> 0 hits, 0 misses, 486 trims, 487 created
> 0 failures (0 no memory)
> CF Large buffers, 5024 bytes (total 2, permanent 1, peak 2 @ 7w0d):
> 2 in free list (1 min, 2 max allowed)
> 0 hits, 0 misses, 486 trims, 487 created
> 0 failures (0 no memory)
> IPC Medium buffers, 16384 bytes (total 2, permanent 2):
> 2 in free list (1 min, 8 max allowed)
> 0 hits, 0 fallbacks, 0 trims, 0 created
> 0 failures (0 no memory)
> IPC Large buffers, 65535 bytes (total 17, permanent 16, peak 17 @ 7w0d):
> 17 in free list (16 min, 16 max allowed)
> 0 hits, 0 misses, 78809 trims, 78810 created
> 0 failures (0 no memory)
>
> Header pools:
> Header buffers, 0 bytes (total 8705, permanent 8192, peak 8705 @ 7w0d):
> 513 in free list (512 min, 16384 max allowed)
> 8021 hits, 171 misses, 972 trims, 1485 created
> 0 failures (0 no memory)
> 8192 max cache size, 8192 in cache
> 13200 hits in cache, 0 misses in cache
>
> Particle Clones:
> 1024 clones, 19303 hits, 0 misses
>
> Public particle pools:
> F/S buffers, 256 bytes (total 3072, permanent 3072):
> 1024 in free list (512 min, 4096 max allowed)
> 2048 hits, 0 misses, 0 trims, 0 created
> 0 failures (0 no memory)
> 2048 max cache size, 2048 in cache
> 24526 hits in cache, 0 misses in cache
> Normal buffers, 1548 bytes (total 3840, permanent 3840):
> 3712 in free list (128 min, 4096 max allowed)
> 128 hits, 0 misses, 0 trims, 0 created
> 0 failures (0 no memory)
> 128 max cache size, 128 in cache
> 0 hits in cache, 0 misses in cache
> CRYPTO_HW buffers, 1600 bytes (total 12288, permanent 12288):
> 0 in free list (0 min, 12288 max allowed)
> 12288 hits, 0 misses
> 12288 max cache size, 12288 in cache
> 234482 hits in cache, 0 misses in cache
>
> Private particle pools:
> HQF Particle buffers, 0 bytes (total 2000, permanent 2000):
> 2000 in free list (500 min, 2000 max allowed)
> 0 hits, 0 misses, 0 trims, 0 created
> 0 failures (0 no memory)
> IDS SM buffers, 240 bytes (total 128, permanent 128):
> 0 in free list (0 min, 128 max allowed)
> 128 hits, 0 fallbacks
> 128 max cache size, 128 in cache
> 0 hits in cache, 0 misses in cache
> GigabitEthernet0/0 buffers, 2304 bytes (total 2561, permanent 2048):
> 513 in free list (512 min, 6144 max allowed)
> 1877 hits, 171 misses, 972 trims, 1485 created
> 0 failures (0 no memory)
> 2048 max cache size, 1536 in cache
> 31641103 hits in cache, 0 misses in cache
> 14 buffer threshold, 0 threshold transitions
> GigabitEthernet0/1 buffers, 2304 bytes (total 2561, permanent 2048):
> 513 in free list (512 min, 6144 max allowed)
> 1877 hits, 171 misses, 972 trims, 1485 created
> 0 failures (0 no memory)
> 2048 max cache size, 1536 in cache
> 2796374359 hits in cache, 0 misses in cache
> 14 buffer threshold, 0 threshold transitions
> GigabitEthernet0/2 buffers, 2304 bytes (total 2561, permanent 2048):
> 513 in free list (512 min, 6144 max allowed)
> 1877 hits, 171 misses, 972 trims, 1485 created
> 0 failures (0 no memory)
> 2048 max cache size, 1536 in cache
> 1095825533 hits in cache, 0 misses in cache
> 14 buffer threshold, 0 threshold transitions
> GigabitEthernet0/3 buffers, 2304 bytes (total 2561, permanent 2048):
> 513 in free list (512 min, 6144 max allowed)
> 1877 hits, 171 misses, 972 trims, 1485 created
> 0 failures (0 no memory)
> 2048 max cache size, 1536 in cache
> 512 hits in cache, 0 misses in cache
> 14 buffer threshold, 0 threshold transitions
>
>
> 3) show controller
>
>
> # sh controllers gigabitEthernet 0/2
> Interface GigabitEthernet0/2 (idb 0x22B2F87C)
> Hardware is iGbE
> Register base address: 0xFD120000
> PCI REV: A1
> network link is up
> loopback type is none
>
> = Media RJ45 =
>
> Media Change Count: 1
>
> MAC registers:
> CTRL =0x000C1A41, STATUS=0x80080783, CTRL_X=0x12800480, IMS
> =0x00000091
> EECD =0x00000218, EERD =0x00000000 ETT =0x00000000,
> TXDMAC=0x00000000
> RCTL =0x00408032, RDBAL =0x10092000, RDBAH =0x00000000, RDLEN
> =0x00002000
> RDH =0x00000000, RDT =0x000001FF, RDTR =0x0000130D, RLPML
> =0x000005F4
> RFCTL =0x00800000
> TCTL =0xA40400FA, TDBAL =0x10095000, TDBAH =0x00000000, TDLEN
> =0x00004000
> TDH =0x00000178, TDT =0x00000178, TIPG =0x00701808
> TDWAH =0x00000000, TDWAL =0x0FA13803
> FCRTH =0x00000000, FCRTL =0x00000000
> FCAH =0x00000100, FCAL =0x00C28001, FCT =0x00008808, FCTTV
> =0x00000000
> RDFH =0x40021A70, RDFT =0x00001A24, RDFPC =0x00000000
> TDFH =0x4002130E, TDFT =0x0000130C, TDFPC =0x00000000
> RDFHS =0x00001AB0, RDFTS =0x00001AB0, RXDCTL=0x02010000
> TDFHS =0x0000130E, TDFTS =0x0000130E, TXDCTL=0x02000000
> RXCSUM=0x00000300, TIDV=0x00000020
> PCS_CFG =0x00000008, PCS_LCTL=0x02110000, PCS_LSTS =0x0061001D
> PCS_ANADV =0x00000020, PCS_LPAB=0x0000D800, PCS_NPTX =0x00000000
> PCS_LPABNP=0x00000000, I2CCMD =0x00000000, I2CPARAMS=0x00001746
>
> Rxdescriptor addr=10532100 len=48310437, stat=00000000
> Txdescriptor addr=0F8400FC cmd/len=0B00004E, stat=00000000
> RX is enabled, TX is enabled
> Device status = full-duplex, link up
> PCI width = 32, PCI66=0, PCIX=0, PCIXSPD is 50-66MHz Device Speed =
> 1000Mbps
>
> PCI configuration registers:
> bus_no=2, device_no=0, subdevice 0
> DeviceID=0x10C9, VendorID=0x8086, Command=0x0146, Status=0x0010
> Class=0x02/0x00/0x00, Revision=0x01, LatencyTimer=0x00,
> CacheLineSize=0x10
> BaseAddr0=0xFD120000, BaseAddr1=0x00000000, MaxLat=0x00,MinGnt=0x00
> SubsysDeviceID=0x0000, SubsysVendorID=0x8086
> Interrupt Line 0x0B, Pin 0x01
> Cap_Ptr=0x00000040 Retry/TRDY Timeout=0x48235001
> PMC=0x00000000 PMCSR=0x00000003
> DEVCTRL=0x00090800
>
> Internal Driver Information:
> lc_ip_turbo_fs=0x0, ip_routecache=0x11(dfs=0/mdfs=0)
> i82576_ds=0x22B30BB8, registers=0xFD120000
> mci_txcount 1024 rx cache size=2048, rx cache end=1536, rx_nobuffer=0
> max_mtu=1528
> ring sizes: RX=512, TX=1024
> rxring=0x10092000, shadow=0x22B3165C, head=0, rx_buf_size=2048
> txring=0x10095000, shadow=0x22B31E90, sw_tdh=376, tail=376
> chip_state=2, pci_rev=1
> congested = 0 tx_count=0, tx_ring_max=1024, tx_limited=0 (cur ring size
> 1024)
> rx_overrun=58854, rx_seq=0, rx_no_enp=0, rx_discard=0
> throttled=0, enabled=0, disabled=0, bypassed=0
> reset=4(init=1, check=0, restart=3, pci=0), auto_restart=8
> link_reset=0, tx_carrier_loss=4, fatal_tx_err=0
> wait_for_last_tdt=0, rx_stuck=0
> tx_stuck=0, rx_max_spin=1, rr_port_next=1
>
> Statistics:
> CRC error 0 Symbol error 0
> Missed Packets 58854 Excessive Coll 0
> Multiple Coll 0 Length Error 0
> Late Coll 0 Collision 0
> Defer 0 Receive Length 0
> XON RX 0
> XON TX 0 XOFF RX 0
> XOFF TX 0 FC RX Unsupport 0
> Packet RX (64) 0 Packet RX (127) 5486324
> Packet RX (255) 2568423 Packet RX (511) 764976
> Packet RX (1023) 598611 Packet RX (1522) 3299107
> Good Packet RX 12717441 Broadcast RX 0
> Multicast RX 12688463 Good Packet TX 44078
> Good Octets RX.H 1 Good Octets RX.L 1954032566
> Good Octets TX.H 0 Good Octets TX.L 5414177
> RX No Buff 9421 RX Undersize 0
> RX Fragment 0 RX Oversize 0
> RX Octets High 1 RX Octets Low 1961371318
> TX Octets High 0 TX Octets Low 5414177
> TX Packet 44078 RX Packet 12776868
> TX Broadcast 0 TX Multicast 5968
> Packet TX (64) 3912 Packet TX (127) 32710
> Packet TX (255) 2567 Packet TX (511) 4889
> Packet TX (1023) 0 Packet TX (1522) 0
> TX No CSR 0
> RX Error Count 0 TCP Segmentation 0
> Host Trasmit Discarded packets by MAC Count 0
> Interrupt Assertion Count 5809488
> Receive Packets to Host Count 12717440
> Host Good Packets Transmitted Count 44078
>
>
> PHY REGISTERS:
> Display for CRC check for page:6
>
> PACKET_GENERATION reg_addr: 16: 0x0010
> CRC COUNTERS reg_addr: 17: 0xFF00
>
> page number:0
>
> Copper Contrl @ reg_addr: 0: 0x1140
> Copper Status @ reg_addr: 1: 0x796D
> PHY Idf1 @ reg_addr: 2: 0x0141
> PHY Idf2 @ reg_addr: 3: 0x0C97
> Autoneg Adv @ reg_addr: 4: 0x0C01
> Autoneg Lnk Ptr abty @ reg_addr: 5: 0xC001
> Autoneg Exp @ reg_addr: 6: 0x000D
> Autoneg Nx Pg Tx @ reg_addr: 7: 0x2801
> Autoneg Lk Prt Nx Pg @ reg_addr: 8: 0x4D26
> 1000B-T Ctr @ reg_addr: 9: 0x0E00
> 1000B-T St @ reg_addr: 10: 0x3800
> Copper Ext St @ reg_addr: 15: 0x3000
> Copper Specific Ctl1 @ reg_addr: 16: 0x6060
> Copper Specific St1 @ reg_addr: 17: 0xAC08
> Copper Interrupt En @ reg_addr: 18: 0x0000
> Copper Specific St2 @ reg_addr: 19: 0x0000
> Copper Rcv Er Ctr @ reg_addr: 21: 0x0000
> Copper Specific Ctl2 @ reg_addr: 26: 0x8040
>
> page number:1
>
> Fiber Contrl @ reg_addr: 0: 0x1140
> Fiber Status @ reg_addr: 1: 0x794D
> PHY Idf1 @ reg_addr: 2: 0x0141
> PHY Idf2 @ reg_addr: 3: 0x0C97
> Autoneg Adv @ reg_addr: 4: 0x0060
> Autoneg Lnk Ptr abty @ reg_addr: 5: 0x0000
> Autoneg Exp @ reg_addr: 6: 0x0004
> Autoneg Nx Pg Tx @ reg_addr: 7: 0x2001
> Autoneg Lk Prt Nx Pg @ reg_addr: 8: 0x0000
> Fiber Ext St @ reg_addr: 15: 0x3000
> Fiber Specific Ctl1 @ reg_addr: 16: 0x0000
> Fiber Specific St1 @ reg_addr: 17: 0x0018
> Fiber Interrupt En @ reg_addr: 18: 0x0000
> Fiber Specific St2 @ reg_addr: 19: 0x0000
> Fiber Rcv Er Ctr @ reg_addr: 21: 0x0000
> Fiber Specific Ctl2 @ reg_addr: 26: 0x8085
>
> page number:2
>
> MAC Contrl @ reg_addr: 0: 0x1040
> PHY Idf1 @ reg_addr: 2: 0x0141
> PHY Idf2 @ reg_addr: 3: 0x0C97
> MAC Specific Ctl1 @ reg_addr: 16: 0x0288
> MAC Specific St1 @ reg_addr: 17: 0x0440
> MAC Interrupt En @ reg_addr: 18: 0x0000
> MAC Specific St2 @ reg_addr: 19: 0x0000
> MAC Specific Ctl2 @ reg_addr: 26: 0x0005
>
> HW addr filter: 0x22B33EC4
>
> Chip address filter[0] 80000293.16032B44
> Chip address filter[1] 80000E00.00C28001
> Chip address filter[2] 8000CCCC.CC0C0001
> Chip address filter[3] 80000700.00C28001
> Chip address filter[4] 80000200.00C28001
> Chip address filter[5] 03FFFFFF.005E0001
> Chip address filter[6] 03FFFFFF.005E0001
> Chip address filter[7] 03FFFFFF.005E0001
> Chip address filter[8] 03FFFFFF.005E0001
> Chip address filter[9] 03FFFFFF.00000000
> Chip address filter[10] 03FFFFFF.00000000
> Chip address filter[11] 03FFFFFF.00000000
> Chip address filter[12] 03FFFFFF.00000000
> Chip address filter[13] 03FFFFFF.00000000
> Chip address filter[14] 03FFFFFF.00000000
> Chip address filter[15] 03FFFFFF.00000000
> Entry= 0: Addr=442B.0316.9302
> Entry= 1: Addr=0180.C200.000E
> Entry= 2: Addr=0100.0CCC.CCCC
> Entry= 3: Addr=0180.C200.0007
> Entry= 4: Addr=0180.C200.0002
> (All other entries are empty)
>
> Software MAC address filter(hash:length/addr/mask/hits)
> -------------------------------------------------------
> 0x00: 0 ffff.ffff.ffff 0000.0000.0000 5
> 0x01: 0 442b.0316.9302 0000.0000.0000 1035374
> 0xC0: 0 0100.0ccc.cccc 0000.0000.0000 12399
> 0xC0: 1 0180.c200.0002 0000.0000.0000 0
> 0xC5: 0 0180.c200.0007 0000.0000.0000 0
> 0xCC: 0 0180.c200.000e 0000.0000.0000 2
> 0x00: 0 0100.5e7f.ffff 0000.007f.ffff 1094516179
>
> Software filtered frames: 261439
> unicast overflow mode off
> multicast overflow mode on
> Promiscuous mode off
>
>
>
>
>
>
>
>
>
>
>
>
>
> Il giorno gio 30 gen 2025 alle ore 16:41 Brian Turnbow <
> brian.turnbow at cdlan.it> ha scritto:
>
>> Hi James
>>
>> You can use
>> sh buffers and sh controller g0/2 for more info and then try and tune
>> buffers, but this doesn't help a lot usually.
>> Another thing to check would be for microbursts using the old rate limit
>> trick setting a policer with exceed action transmit
>> Then you can check to see if you are overflowing therate limit bucket as
>> sign of microburst. Like maybe due to speed steps 10G/1G.
>> Overruns are a sign that the router can't handle the traffic coming in
>> and may be due to traffic levels or traffic types that take longer
>> to process.
>> Flow control could help but I see unsupported listed on the interface
>> and admit I am not well versed on the 3925 as a platform
>> Have you tried a reboot ? I have seen routers get out of whack that
>> magically return to form after a reboot..
>>
>>
>>
>> Brian Turnbow
>>
>>
>>
>> Il giorno gio 30 gen 2025 alle ore 14:41 james list via cisco-nsp <
>> cisco-nsp at puck.nether.net> ha scritto:
>>
>>> Dear expert
>>> I'm trying to get any useful insight and I can also understand not polite
>>> comments...
>>>
>>> We've a customer which is experiencing multicast gaps on feed A and we've
>>> identified an old C3925-SPE200 in a remote location (hence difficult to
>>> be
>>> replaced in a manner) which is experiencing input errors/overrun also on
>>> low bandwidth and packet rate.
>>> The customer has the same kind of device in another remote location
>>> receiving the same multicast feed (feed B) without issues.
>>> I think these gaps are due to packet dropped ... customer has also tried
>>> to
>>> move interface from g0/1 to g0/2 but no improvement has obtained.
>>>
>>> Any idea what could be the reason ?
>>>
>>> Thanks in advance
>>> James
>>>
>>> xxx>sh inventory
>>> NAME: "CISCO3925-CHASSIS", DESCR: "CISCO3925-CHASSIS"
>>> PID: CISCO3925-CHASSIS , VID: V02, SN: FCZxxx
>>>
>>> NAME: "Cisco Services Performance Engine 200 for Cisco 3900 ISR on Slot
>>> 0",
>>> DESCR: "Cisco Services Performance Engine 200 for Cisco 3900 ISR"
>>> PID: C3900-SPE200/K9 , VID: V04 , SN: FOC160320B7xxx
>>>
>>> NAME: "C3900 AC Power Supply 1", DESCR: "C3900 AC Power Supply 1"
>>> PID: PWR-3900-AC , VID: V03, SN: SNIxxx
>>>
>>> NAME: "C3900 AC Power Supply 2", DESCR: "C3900 AC Power Supply 2"
>>> PID: PWR-3900-AC , VID: V03 , SN: QCSxxx
>>>
>>>
>>> xxx>sh int g0/2
>>> GigabitEthernet0/2 is up, line protocol is up
>>> Hardware is iGbE, address is 442b.0316.9302 (bia 442b.0316.9302)
>>> Description: To Market
>>> MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,
>>> reliability 255/255, txload 1/255, rxload 2/255
>>> Encapsulation 802.1Q Virtual LAN, Vlan ID 1., loopback not set
>>> Keepalive set (10 sec)
>>> Full Duplex, 1Gbps, media type is RJ45
>>> output flow-control is unsupported, input flow-control is unsupported
>>> ARP type: ARPA, ARP Timeout 04:00:00
>>> Last input 00:00:00, output 00:00:00, output hang never
>>> Last clearing of "show interface" counters 01:18:46
>>> Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0
>>> Queueing strategy: fifo
>>> Output queue: 0/40 (size/max)
>>> 30 second input rate 9004000 bits/sec, 2628 packets/sec
>>> 30 second output rate 9000 bits/sec, 7 packets/sec
>>> 7936911 packets input, 4224448667 bytes, 0 no buffer
>>> Received 7914597 broadcasts (7218973 IP multicasts)
>>> 0 runts, 0 giants, 0 throttles
>>> 56932 input errors, 0 CRC, 0 frame, 56932 overrun, 0 ignored
>>> 0 watchdog, 7914597 multicast, 0 pause input
>>> 30904 packets output, 2951099 bytes, 0 underruns
>>> 0 output errors, 0 collisions, 0 interface resets
>>> 158 unknown protocol drops
>>> 0 babbles, 0 late collision, 0 deferred
>>> 0 lost carrier, 0 no carrier, 0 pause output
>>> 0 output buffer failures, 0 output buffers swapped out
>>>
>>>
>>> interface GigabitEthernet0/2
>>> description To Market
>>> no ip address
>>> load-interval 30
>>> duplex full
>>> speed 1000
>>> hold-queue 2000 in
>>> !
>>> interface GigabitEthernet0/2.90
>>> description VLAN multicast
>>> encapsulation dot1Q 90
>>> ip address xx.xx.221.1 255.255.255.0
>>> ip pim query-interval 5
>>> ip pim bsr-border
>>> ip pim dense-mode proxy-register list xx
>>> ip multicast boundary xx
>>> ip nat outside
>>> ip virtual-reassembly in
>>> !
>>> interface GigabitEthernet0/2.1683
>>> description To market flex vpn
>>> encapsulation dot1Q 1683
>>> ip address 10.xx.xx.85 255.255.255.252
>>> ip access-group CRYPTO-xxx
>>> _______________________________________________
>>> cisco-nsp mailing list cisco-nsp at puck.nether.net
>>> https://puck.nether.net/mailman/listinfo/cisco-nsp
>>> archive at http://puck.nether.net/pipermail/cisco-nsp/
>>>
>>
More information about the cisco-nsp
mailing list