[c-nsp] vss switch 2 reloading every 15 minutes
C and C Dominte
domintefamily at yahoo.co.uk
Wed Aug 26 07:55:28 EDT 2009
Hi,
I recently configured two catalyst 6509 switches into a
VSS cluster. After I experimented issues with unknown unicast, the secondary
chassis reloaded itself with no apparent reason, .
The cluster is configured with two VSL 10G links, one
link is on the supervisor, and the secondary one located on the WS-X6708-10GE linecard,
to provide redundancy.
I pasted the details from the logs below. The sequence of
events is as follows:
1. For some
reason VSL interfaces go down
Aug 25 15:54:10.563: %LINK-3-UPDOWN: Interface
TenGigabitEthernet1/6/4, changed state to down
2. This
triggers the VSL failure
Aug 25 15:54:10.535: %VSLP-SW1_SP-3-VSLP_LMP_FAIL_REASON:
Te1/6/4: Link down Aug 25 15:54:10.535: %VSLP-SW1_SP-2-VSL_DOWN: Last VSL
interface Te1/6/4 went down Aug 25 15:54:10.559: %VSLP-SW1_SP-2-VSL_DOWN: All
VSL links went down while switch is in ACTIVE role Aug 25 15:54:10.567:
%LINK-SW1_SP-3-UPDOWN: Interface TenGigabitEthernet1/6/4, changed state to down
3. Another
VSL error:
Aug 25 15:54:10.655:
%NTI-SW1_SP-3-TRIG_INIT_ALREADY_IN_PROGRESS: Cannot initiate NTI trigger for EP
ID 0x216 at this time: trigger processing of trigger type NTI_EP_NOT_PRESENT,
trigger group NTI_TRIGGER_GROUP_NONE already in progress
4. Chassis 2 reloads,
and the failover mode is changed to simplex
Aug 25 15:54:11.259: %PFREDUN-SW1_SP-6-ACTIVE: Standby
processor removed or reloaded, changing to Simplex mode
5. A few more
errors for 5 minutes. The cached IP-over-EoBC packets are being dropped:
Aug 25 15:55:01.174:
%SATVS_IBC-SW1_SP-5-VSL_DOWN_SCP_DROP: VSL inactive - dropping cached SCP
packet: (SA/DA:0x5/0x5, SSAP/DSAP:0x1D/0xAB, OP/SEQ:0x500/0x9BF5,
SIG/INFO:0x1/0x501, eSA:0000.0600.0000) Aug 25 15:56:04.064:
%SATVS_IBC-SW1_SP-5-VSL_DOWN_SCP_DROP: VSL inactive - dropping cached SCP
packet: (SA/DA:0x5/0x5, SSAP/DSAP:0x1D/0xAB, OP/SEQ:0x500/0x9D72,
SIG/INFO:0x1/0x501, eSA:0000.0600.0000) Aug 25 15:58:23.016:
%SATVS_IBC-SW1_SP-5-VSL_DOWN_SCP_DROP: VSL inactive - dropping cached SCP
packet: (SA/DA:0x5/0x5, SSAP/DSAP:0x1D/0xAB, OP/SEQ:0x500/0xA123,
SIG/INFO:0x1/0x501, eSA:0000.0600.0000)
6. VSL
Interface on the supervisor comes back up:
Aug 25 16:02:19.037: %LINK-3-UPDOWN: Interface
TenGigabitEthernet1/6/4, changed state to up Aug 25 16:02:18.737:
%LINK-SW1_SP-3-UPDOWN: Interface TenGigabitEthernet1/6/4, changed state to up
7. VSL
reinitialises
Aug 25 16:02:50.721: %VSLP-SW1_SP-5-RRP_ROLE_RESOLVED:
Role resolved as ACTIVE by VSLP Aug 25 16:02:50.721:
%VSL-SW1_SP-5-VSL_CNTRL_LINK: New VSL Control Link Te1/6/4 Aug 25 16:02:55.972:
%VSLP-SW1_SP-5-VSL_UP: Ready for control traffic Aug 25 16:05:22.325:
%PFREDUN-SW1_SP-6-ACTIVE: Standby initializing for SSO mode Aug 25
16:09:47.278: %PFINIT-SW1_SP-5-CONFIG_SYNC: Sync'ing the startup configuration
to the standby Router.
8. Interface
on the supervisor goes down again
Aug 25 16:14:37.314: %LINK-3-UPDOWN: Interface
TenGigabitEthernet1/6/4, changed state to down
The same loop repeats at regular intervals, and the
secondary VSL link never comes back online. No other line card comes back
online on switch 2, as the secondary supervisor does not initialise fully.
The interfaces went down initially with the following
message logged:
Aug 25 15:16:49.254: %SYS-SW2_SPSTBY-5-RELOAD: Reload
requested - From Active Switch (Reload peer unit).
I pasted below the full initial log, when the secondary
chassis went down:
Yesterday
16:20:51
Syslog [Message
Details] Aug 25 15:16:53.686: %SATVS_IBC-SW1_SP-5-VSL_DOWN_SCP_DROP: VSL
inactive - dropping cached SCP packet: (SA/DA:0x5/0x5, SSAP/DSAP:0x2A/0x0,
OP/SEQ:0x3 ...
Yesterday 16:20:51
Syslog [Message
Details] Aug 25 15:16:52.994: %LINK-SW1_SP-3-UPDOWN: Interface Port-channel120,
changed state to down
Yesterday 16:20:51
Syslog [Message
Details] Aug 25 15:16:52.290: %LINK-SW1_SP-3-UPDOWN: Interface
TenGigabitEthernet2/6/4, changed state to down
Yesterday 16:20:40
Syslog [Message
Details] Aug 25 15:16:52.270: %PFREDUN-SW1_SP-6-ACTIVE: Standby processor
removed or reloaded, changing to Simplex mode
Yesterday 16:20:39
Syslog [Message
Details] Aug 25 15:16:51.502: %LINK-SW1_SP-3-UPDOWN: Interface
TenGigabitEthernet1/6/4, changed state to down
Yesterday 16:20:39
Syslog [Message
Details] Aug 25 15:16:51.502: %LINK-SW1_SP-3-UPDOWN: Interface
TenGigabitEthernet1/7/1, changed state to down
Yesterday 16:20:39
Syslog [Message
Details] Aug 25 15:16:51.502: %LINK-SW1_SP-3-UPDOWN: Interface Port-channel256,
changed state to down
Yesterday 16:20:39
Syslog [Message
Details] Aug 25 15:16:51.502: %LINEPROTO-SW1_SP-5-UPDOWN: Line protocol on
Interface Port-channel256, changed state to down
Yesterday 16:20:39
Syslog [Message
Details] Aug 25 15:16:51.502: %LINEPROTO-SW1_SP-5-UPDOWN: Line protocol on
Interface TenGigabitEthernet1/7/1, changed state to down
Yesterday 16:20:39
Syslog [Message
Details] Aug 25 15:16:51.334: %VSLP-SW1_SP-2-VSL_DOWN: All VSL links went down
while switch is in ACTIVE role
Yesterday 16:20:39
Syslog [Message Details] Aug 25 15:16:51.334:
%LINEPROTO-SW1_SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet2/6/4,
changed state to down
Yesterday 16:20:39
Syslog [Message Details] Aug 25 15:16:51.330:
%LINEPROTO-SW1_SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/6/4,
changed state to down
Yesterday 16:20:39
Syslog [Message Details] Aug 25 15:16:51.114:
%VSLP-SW1_SP-2-VSL_DOWN: Last VSL interface Te1/7/1 went down
Yesterday 16:20:39
Syslog [Message
Details] Aug 25 15:16:51.114: %VSLP-SW1_SP-3-VSLP_LMP_FAIL_REASON: Te1/7/1:
Disabled by Peer Reload Request
Yesterday 16:20:39
Syslog [Message
Details] Aug 25 15:16:51.030: %VSL-SW1_SP-5-VSL_CNTRL_LINK: New VSL Control
Link Te1/7/1
Yesterday 16:20:39
Syslog [Message
Details] Aug 25 15:16:51.030: %VSLP-SW1_SP-3-VSLP_LMP_FAIL_REASON: Te1/6/4:
Disabled by Peer Reload Request
Yesterday 16:20:39
Syslog [Message Details] Aug 25 15:16:48.990:
%RF-SW1_SP-5-RF_RELOAD: Peer reload. Reason: Proxy request to reload peer
Yesterday 16:20:39
Syslog [Message
Details] Aug 25 15:17:01.590: %LINEPROTO-5-UPDOWN: Line protocol on Interface
TenGigabitEthernet2/7/1, changed state to down
Yesterday 16:20:39
Syslog [Message
Details] Aug 25 15:17:01.578: %LINK-3-UPDOWN: Interface
TenGigabitEthernet2/7/1, changed state to down
Yesterday 16:20:39
Syslog [Message
Details] Aug 25 15:17:01.574: %LINK-3-UPDOWN: Interface Port-channel255, changed
state to down
Yesterday 16:20:39
Syslog [Message
Details] Aug 25 15:17:01.574: %LINEPROTO-5-UPDOWN: Line protocol on Interface
Port-channel255, changed state to down
Yesterday 16:20:30
Syslog [Message
Details] Aug 25 15:16:51.534: %LINK-3-UPDOWN: Interface
TenGigabitEthernet1/6/4, changed state to down
Yesterday 16:20:30
Syslog [Message
Details] Aug 25 15:16:51.446: %LINK-3-UPDOWN: Interface
TenGigabitEthernet1/7/1, changed state to down
Yesterday 16:20:30
Syslog [Message
Details] Aug 25 15:16:51.446: %LINK-3-UPDOWN: Interface Port-channel256,
changed state to down
Yesterday 16:20:29
Syslog [Message
Details] Aug 25 15:16:51.446: %LINEPROTO-5-UPDOWN: Line protocol on Interface
Port-channel256, changed state to down
Yesterday 16:20:29
Syslog [Message
Details] Aug 25 15:16:51.382: %LINEPROTO-5-UPDOWN: Line protocol on Interface
TenGigabitEthernet1/7/1, changed state to down
Yesterday 16:20:28
Syslog [Message
Details] Aug 25 15:16:51.102: %LINEPROTO-5-UPDOWN: Line protocol on Interface
TenGigabitEthernet2/6/4, changed state to down
Yesterday 16:20:28
Syslog [Message
Details] Aug 25 15:16:51.046: %LINEPROTO-5-UPDOWN: Line protocol on Interface
TenGigabitEthernet1/6/4, changed state to down
Yesterday 16:20:28
Syslog [Message
Details] Aug 25 15:16:49.254: %SYS-SW2_SPSTBY-5-RELOAD: Reload requested - From
Active Switch (Reload peer unit).
Yesterday 16:20:26
Syslog [Message
Details] Aug 25 15:16:48.138: %IDBMAN-4-CONFIG_WRITE_FAIL: Failed to generate
configuration for interface Po120
Has anyone seen something similar?
Regards,
Catalin
More information about the cisco-nsp
mailing list