[c-nsp] ME3600 BFD session to A9K breaks after upgrade to 15.3(3)S1a

[Jason Lixfeld]

Hi all,

I got an answer on this and thought I'd share.  It bit me in the ass and I'd hate for it to bite anyone else.

The root cause was due to a fix implemented in 15.3(3)S1a for CSCtl54835.  Essentially, the CLNS mtu is now properly calculated from the L3 interface MTU whereas before, the CLNS MTU was always 1497 no matter what the L3 interface MTU was set to.  This fix was not listed in the list of resolved caveats for that release, but it is now.

So,

After the upgrade, the CLNS MTU was calculated to be 9213 (based on a L3 interface MTU of 9216) which caused an incompatibility with it's adjacent 9K which has a CLNS MTU of 9202 (based on a L3 interface MTU of 9216).

The fix was to lower the L3 MTU on the ME3600 to 9202 (which is the correct MTU anyway when the A9K MTU is 9216).  This lowered the ME3600 CLNS MTU to 9199 at which point ISIS/BFD was again operational.

I haven't read the CLNS or ISIS RFC, so I don't understand what the expected behaviour is terms of CLNS MTU, but my findings are as follows:

Pre-15.3(3)S1a upgrade:
ME3600 interface MTU: 9216
ME3600 inherited CLNS MTU: 1497
ASR9K interface MTU: 9216
ASR9K inherited CLNS MTU: 9202

ME3600 CLNS MTU < ASR9K CLNS MTU

Result: ISIS/BFD Up
--
Post-15.3(3)S1a upgrade (no config change):

ME3600 interface MTU: 9216
ME3600 inherited CLNS MTU: 9213
ASR9K interface MTU: 9216
ASR9K inherited CLNS MTU: 9202

ME3600 CLNS MTU > ASR9K CLNS MTU

Result: ISIS/BFD Down
--
Post-15.3(3)S1a upgrade (config changed):

ME3600 interface MTU: 9202
ME3600 inherited CLNS MTU: 9199
ASR9K interface MTU: 9216
ASR9K inherited CLNS MTU: 9202

ME3600 CLNS MTU < ASR9K CLNS MTU

Result: ISIS/BFD Up

I have heard that it is the understanding of other people (who have presumably read the RFC) that this configuration should still not work because the CLNS MTU always needs to match on both ends.  So it seems that there might be another issue afoot here?  To me, it seems that the A9K cannot negotiate an ISIS adjacency when a neighbour CLNS MTU is larger than itself, but the ME3600 can, but if the CLNS MTU is supposed to match always, then both devices are actually misbehaving?

On Nov 12, 2013, at 9:57 AM, Jason Lixfeld <jason at lixfeld.ca> wrote:

> Before an upgrade to 15.3(3)S1a, a BFD session between a 9K and an ME3600 worked just fine.  After the upgrade, BFD session wouldn't come up.  I looked at the release notes and couldn't see any notes about behaviour changes in BFD or any specific caveats.  BFD still works fine if the adjacent device is another ME3600 (only tested adjacent ME3600s that were *not* upgraded to 15.3(3)S1a).
> 
> I've got Cisco looking at it, but I'm just curious if anyone else has seen this change in behaviour.  Previously, this ME3600 was running 15.3(3)S and it worked just fine to this same 9K (which is running XR 4.3.1).
> 
> ! A9K
> !
> router isis
> is-type level-2-only
> net 00.0720.1504.8009.0000.00
> nsf cisco
> log adjacency changes
> log pdu drops
> address-family ipv4 unicast
>  metric-style transition
> !
> interface TenGigE0/0/0/6
>  bfd minimum-interval 300
>  bfd multiplier 3
>  bfd fast-detect ipv4
>  link-down fast-detect
>  address-family ipv4 unicast
> !
> 
> ! ME3600
> !
> interface TenGigabitEthernet0/1
> no switchport
> dampening
> mtu 9216
> ip address 72.15.51.45 255.255.255.254
> ip router isis 21949
> logging event link-status
> carrier-delay msec 0
> mpls ip
> bfd interval 300 min_rx 300 multiplier 3
> isis bfd
> !
> router isis 21949
> net 00.0720.1505.0138.0000.00
> is-type level-2-only
> metric-style transition
> spf-interval 5 1 50
> prc-interval 5 1 50
> lsp-gen-interval 5 1 50
> bfd all-interfaces
> !
> 
> Thanks in advance.