[c-nsp] SPA issue on ASR1006

Wed Apr 22 07:27:28 EDT 2015

Dear colleagues,

I have an ASR1006 with ESP20, RP2 and two SIP10. Recently I have inserted one SPA 1x10G into one of the SIP10 and I have seen the following log messages:

Apr 22 12:00:53: %IOSXE_OIR-6-INSSPA: SPA inserted in
subslot 1/2

Apr 22 12:01:00: %LINK-3-UPDOWN: SIP1/2: Interface
EOBC1/1, changed state to up

ASR1006-LIS01-02#sh ip int  

Apr 22 12:01:03: %IOSXE_SPA-6-UPDOWN: Interface
TenGigabitEthernet1/2/0, link down due to local fault

Apr 22 12:01:03: %SPA_OIR-6-ONLINECARD: SPA
(SPA-1X10GE-L-V2) online in subslot 1/2

Apr 22 12:01:11: %IOSXE_SIP_SPA-4-SPABUS: SIP1/2:
Subslot 2, Read SPA BUS access failed. timeout=0 err=1 par=0 err_c=0x10883
addr=0x00045008 data =0x0 

ASR1006-LIS01-02#

Apr 22 12:01:12: %IOSXE_SIP_SPA-3-DEVICE_ERROR:
SIP1/2: subSlot 2, spatype 0x50C. Device error: critical fault occured

ASR1006-LIS01-02#

Apr 22 12:01:15: %CMCC-3-PLIM_STATUS: SIP1:
cmcc:  A PLIM driver informational error RXSPI2 Out of Frame, block 15
count 3

ASR1006-LIS01-02#

Apr 22 12:01:17: %SPA_OIR-3-RECOVERY_RELOAD: subslot
1/2: Attempting recovery by reloading SPA

ASR1006-LIS01-02#

ASR1006-LIS01-02#

Apr 22 12:01:17: %SPA_OIR-6-OFFLINECARD: SPA
(SPA-1X10GE-L-V2) offline in subslot 1/2

ASR1006-LIS01-02#

ASR1006-LIS01-02#

ASR1006-LIS01-02#

ASR1006-LIS01-02#

Apr 22 12:01:26: %IOSXE_SIP_SPA-3-SPI4_NOTSYNC:
SIP1/2: SPA-1X10GE-L-V2[1/2]: Can not synchronize SPI4 bus (host: src in sync,
sink not in sync, spa: src in sync, sink not in sync).

ASR1006-LIS01-02#

Apr 22 12:01:31: %SPA_OIR-3-RECOVERY_RELOAD: subslot
1/2: Attempting recovery by reloading SPA

ASR1006-LIS01-02#

Apr 22 12:01:31: %SPA_OIR-6-OFFLINECARD: SPA
(SPA-1X10GE-L-V2) offline in subslot 1/2

ASR1006-LIS01-02#

Apr 22 12:01:39: %IOSXE_SIP_SPA-3-SPI4_NOTSYNC:
SIP1/2: SPA-1X10GE-L-V2[1/2]: Can not synchronize SPI4 bus (host: src in sync,
sink not in sync, spa: src in sync, sink not in sync).

ASR1006-LIS01-02#

Apr 22 12:01:44: %SPA_OIR-3-RECOVERY_RELOAD: subslot
1/2: Attempting recovery by reloading SPA

ASR1006-LIS01-02#

Apr 22 12:01:44: %SPA_OIR-6-OFFLINECARD: SPA
(SPA-1X10GE-L-V2) offline in subslot 1/2

ASR1006-LIS01-02#

Apr 22 12:01:52: %IOSXE_SIP_SPA-3-SPI4_NOTSYNC:
SIP1/2: SPA-1X10GE-L-V2[1/2]: Can not synchronize SPI4 bus (host: src in sync,
sink not in sync, spa: src in sync, sink not in sync).

ASR1006-LIS01-02#

Apr 22 12:01:57: %SPA_OIR-3-RECOVERY_RELOAD: subslot
1/2: Attempting recovery by reloading SPA

ASR1006-LIS01-02#

Apr 22 12:01:57: %SPA_OIR-6-OFFLINECARD: SPA
(SPA-1X10GE-L-V2) offline in subslot 1/2

ASR1006-LIS01-02#

ASR1006-LIS01-02#

Apr 22 12:02:05: %IOSXE_SIP_SPA-3-SPI4_NOTSYNC:
SIP1/2: SPA-1X10GE-L-V2[1/2]: Can not synchronize SPI4 bus (host: src in sync,
sink not in sync, spa: src in sync, sink not in sync).

ASR1006-LIS01-02#

Apr 22 12:02:10: %SPA_OIR-3-RECOVERY_RELOAD: subslot
1/2: Attempting recovery by reloading SPA

ASR1006-LIS01-02#

Apr 22 12:02:10: %SPA_OIR-6-OFFLINECARD: SPA
(SPA-1X10GE-L-V2) offline in subslot 1/2

ASR1006-LIS01-02#

Apr 22 12:02:18: %SPA_OIR-3-SPA_POWERED_OFF: subslot
1/2: SPA 1x10GE XFP SPA powered off after 5 failures within 1200 seconds

ASR1006-LIS01-02#

Apr 22 12:02:18: %SPA_OIR-6-OFFLINECARD: SPA
(SPA-1X10GE-L-V2) offline in subslot 1/2

ASR1006-LIS01-02#

Apr 22 12:02:18: %IOSXE_SIP_SPA-3-SPI4_NOTSYNC:
SIP1/2: SPA-1X10GE-L-V2[1/2]: Can not synchronize SPI4 bus (host: src in sync,
sink not in sync, spa: src in sync, sink not in sync).

ASR1006-LIS01-02#

ASR1006-LIS01-02#

ASR1006-LIS01-02#

ASR1006-LIS01-02#

ASR1006-LIS01-02#

ASR1006-LIS01-02#

Apr 22 12:02:47: %IOSXE_OIR-6-REMSPA: SPA removed from
subslot 1/2, interfaces disabled

Any ideas about this issue?

ASR1006-LIS01-02#sh platform   

Chassis type:
ASR1006            

Slot     
Type               
State                
Insert time (ago) 

--------- ------------------- ---------------------
----------------- 

0        
ASR1000-SIP10      
ok                   
13w5d         

 0/0     
SPA-5X1GE-V2       
ok                   
13w5d         

 0/1     
SPA-4XOC3-POS      
ok                   
13w5d         

 0/2     
SPA-1XOC12-POS     
ok                   
13w5d         

 0/3     
SPA-4XOC3-POS      
ok                   
13w5d         

1        
ASR1000-SIP10      
ok                   
13w5d         

 1/0     
SPA-5X1GE-V2       
ok                
   13w5d         

 1/1     
SPA-4XOC3-POS      
ok                   
13w5d         

 1/2     
SPA-1X10GE-L-V2    
booting              
00:00:01      

 1/3     
SPA-4XOC12-POS     
ok                   
13w5d         

R0       
ASR1000-RP2         ok, active           
13w5d         

R1       
ASR1000-RP2         ok,
standby          
13w5d         

F0       
ASR1000-ESP20       ok,
active           
13w5d         

F1       
ASR1000-ESP20       ok,
standby          
13w5d         

P0       
ASR1006-PWR-AC      ok                   
13w5d         

P1       
ASR1006-PWR-AC     
ok                   
13w5d         

Slot      CPLD
Version        Firmware
Version                       

--------- -------------------
--------------------------------------- 

0         09111601           
12.2(33r)XNC                       

1        
09111601           
12.2(33r)XNC                       

R0       
10021901           
12.2(33r)XND                       

R1       
10021901           
12.2(33r)XND                       

F0       
08041102           
12.2(33r)XNC                       

F1       
08041102           
12.2(33r)XNC                       

ASR1006-LIS01-02#

ASR1006-LIS01-02#sh ver
Cisco IOS Software, IOS-XE Software (X86_64_LINUX_IOSD-ADVIPSERVICESK9-M), Version 15.2(4)S6, RELEASE SOFTWARE (fc4)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2014 by Cisco Systems, Inc.
Compiled Thu 14-Aug-14 19:30 by mcpre

IOS XE Version: 03.07.06.S

Cisco IOS-XE software, Copyright (c) 2005-2014 by cisco Systems, Inc.
All rights reserved.  Certain components of Cisco IOS-XE software are
licensed under the GNU General Public License ("GPL") Version 2.0.  The
software code licensed under GPL Version 2.0 is free software that comes
with ABSOLUTELY NO WARRANTY.  You can redistribute and/or modify such
GPL code under the terms of GPL Version 2.0.  For more details, see the
documentation or "License Notice" file accompanying the IOS-XE software,
or the applicable URL provided on the flyer accompanying the IOS-XE
software.

ROM: IOS-XE ROMMON

ASR1006-LIS01-02 uptime is 13 weeks, 5 days, 11 hours, 5 minutes
Uptime for this control processor is 13 weeks, 5 days, 11 hours, 6 minutes
System returned to ROM by reload at 00:18:14 CET Fri Jan 16 2015
System restarted at 00:21:28 CET Fri Jan 16 2015
System image file is "bootflash:asr1000rp2-advipservicesk9.03.07.06.S.152-4.S6.bin"
Last reload reason: Reload Command

This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to
export at cisco.com.

cisco ASR1006 (RP2) processor with 4238183K/6147K bytes of memory.
Processor board ID FOX1608H96D
10 Gigabit Ethernet interfaces
17 Packet over SONET interfaces
32768K bytes of non-volatile configuration memory.
8388608K bytes of physical memory.
1826815K bytes of eUSB flash at bootflash:.
78085207K bytes of SATA hard disk at harddisk:.

Configuration register is 0x2102

Thanks in advance for your support.

Best regards,
Jordi