[c-nsp] Dynamic ARP timeout on ASR 1001

Wed Oct 16 17:55:15 EDT 2013

I have been tweaking the timers and think have fixed the issue, thanks everyone for the replies.

ARP timeout of 60 was ok, I was however testing over maybe a flakey VPN this morning; when I got into the office I could not reproduce the issue anymore.

Cheers,

Chris Gibbs
Network and Security Engineer | Information Management & Technology
Gosford City Council
www.gosford.nsw.gov.au

PO Box 21 Gosford NSW 2250
Phone: (02) 43258888
Mobile: 0408 222 496
Fax:    (02) 4323 2477
chris.gibbs at gosford.nsw.gov.au 

-----Original Message-----
From: Chris Gibbs 
Sent: Thursday, 17 October 2013 6:56 AM
To: 'Mourad Berkane'
Cc: cisco-nsp at puck.nether.net
Subject: RE: Dynamic ARP timeout on ASR 1001

Thanks Mourad.

I have lowered the ARP timeout on the bdi and port channel interface to 60 seconds.

The problem still seems to be occurring, it has however reduced the occurance of the issue, instead of once or twice a minute, around one or twice every 10 minutues depending on how aligned the arp timeouts are to the DHCP renew messages.

When I check on an upstream switch when I loose IP connectivity, I found the mac addresses for the IPoE are appearing to originate from the ASR. Not sure if the ASR somehow spoofing the IPoE clients MAC. I thought perhaps some default DHCP snooping or dynamic ARP inspection feature doing something funny, I haven't turned any of it on though.

I can get the problem to reproduce everytime the ASR sees a DHCP Request for a lease RENEW (or possibly the resulting G-ARP), IP connectivity is now restored faster due to the smaller arp timeout. I can see on some occasions there is an outage of upto the 60 seconds.

I would still like to resolve this issue.

Cheers,

Chris Gibbs
Network and Security Engineer | Information Management & Technology Gosford City Council www.gosford.nsw.gov.au

PO Box 21 Gosford NSW 2250
Phone: (02) 43258888
Mobile: 0408 222 496
Fax:    (02) 4323 2477
chris.gibbs at gosford.nsw.gov.au 

-----Original Message-----
From: Mourad Berkane [mailto:berkane at unhcr.org]
Sent: Wednesday, 16 October 2013 11:22 PM
To: Chris Gibbs
Cc: cisco-nsp at puck.nether.net
Subject: RE: Dynamic ARP timeout on ASR 1001

Default EVC MAC aging-time is 300s (5 minutes) while ARP timeout is 14400s (4 hours).

You may have to lower ARP timeout to 5 minutes also

-----Original Message-----
From: cisco-nsp [mailto:cisco-nsp-bounces at puck.nether.net] On Behalf Of Chris Gibbs
Sent: Wednesday, October 16, 2013 3:30 AM
To: cisco-nsp at puck.nether.net
Subject: [c-nsp] Dynamic ARP timeout on ASR 1001

Hey all,

Having a bit of an issue with bridge-domains on ASR 1001 and dynamic ARP entries.

Looking through the packet captures, I see the following events

1.       DHCP request from CPE

2.       DHCP ACK and assignment from DHCP server to CPE

3.       Gratuitous ARP sent from CPE.

4.       Packets flow as normal.

5.       Something triggers dynamic ARP entry to timeout on the BNG (Cisco ASR 1001).

a.       Suspect this may be triggered by the DHCP Request renew or the following gratuitous ARP received.

b.      See the debug message on the ASR:

*Oct 15 18:18:29.376: IP ARP: rcvd rep src 10.0.30.11 5475.d0df.7f48, dst 10.0.30.11 BDI100 *Oct 15 18:18:29.376: ARP DB: ARP entry of key 10.0.30.11 found *Oct 15 18:18:29.376: ARP TABLE: modifying entry 10.0.30.11/5475.d0df.7f48 on BD100 for Dynamic *Oct 15 18:18:29.376: ARP DYNAMIC[N]: Dynamic timeout occurredtimeout = 14400000, refresh_token = 2,refresh_timeout = 60000 *Oct 15 18:18:29.376: ARP DB: ARP entry of key 10.0.30.11 found

6.       DHCP client on the CPE eventually sends through a DHCP request for the IP 10.0.30.11.

7.       DHCP server replies with ACK.

8.       Gratuitous ARP sent from CPE.

9.       Dynamic ARP entry is populated.

10.   Packets flow as normal.

If I attempt to ping manually from the CPE, the dynamic arp entry is restored on the ASR.

Further details:

                Platform: ASR 1001
                Software: 3.10a (asr1001-universalk9.03.10.00a.S.153-3.S0a-ext.bin)
                Interfaces:

                interface Port-channel2
description Uplink - <redacted>
mtu 2000
ip dhcp relay information option-insert
 ip dhcp relay information check-reply none no ip address no ip unreachables no negotiation auto lacp fast-switchover lacp max-bundle 1 service instance 1101 ethernet
  encapsulation dot1q 80 second-dot1q 1101
  rewrite ingress tag pop 2 symmetric
  ip dhcp relay information option subscriber-id GCC-CPE-1-1
  service-policy output pm_BNG-WAN-wVoice-Out-12Mbps
  bridge-domain 100
!
service instance 1102 ethernet
  encapsulation dot1q 80 second-dot1q 1102
  rewrite ingress tag pop 2 symmetric
  ip dhcp relay information option subscriber-id GCC-CPE-2-1
  service-policy output pm_BNG-WAN-wVoice-Out-25Mbps
  bridge-domain 100
                interface BDI100
ip address 10.0.30.1 255.255.255.0
ip helper-address 2.2.1.2

GCC-BNG-1#sh run | i bridge
bridge-domain 100
bridge-domain 912
bridge irb
  bridge-domain 100
  bridge-domain 100
bridge 100 protocol vlan-bridge
bridge 100 route ip

Any ideas?

Cheers,

[X]<http://www.gosford.nsw.gov.au/>

Chris Gibbs
Network and Security Engineer | Information Management & Technology Gosford City Council www.gosford.nsw.gov.au<http://www.gosford.nsw.gov.au/>

PO Box 21 Gosford NSW 2250
Phone: (02) 4325 8888
Mobile: 0408 222 496
Fax:    (02) 4323 2477
chris.gibbs at gosford.nsw.gov.au<mailto:chris.gibbs at gosford.nsw.gov.au>

The information contained in this email may be confidential. 
You should only disclose, re-transmit, copy, distribute, act in reliance on or commercialise the information if you are authorised to do so. Gosford City Council does not represent, warrant or guarantee that the communication is free of errors, virus or interference.

Gosford City Council complies with the Privacy and Personal Information Protection Act (1998). 
See Council's Privacy Statement at
http://www.gosford.nsw.gov.au/council/privacy.html
_______________________________________________
cisco-nsp mailing list  cisco-nsp at puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
We believe 1 family torn apart by war is too many.
Tell the world you do too: http://www.unhcr.org/1family?link=email

The information contained in this email may be confidential. 
You should only disclose, re-transmit, copy, distribute, 
act in reliance on or commercialise the information if you 
are authorised to do so. Gosford City Council does not 
represent, warrant or guarantee that the communication is 
free of errors, virus or interference.

Gosford City Council complies with the Privacy and 
Personal Information Protection Act (1998). 
See Council's Privacy Statement at 
http://www.gosford.nsw.gov.au/council/privacy.html