[j-nsp] Apply-group brain fart

[Tom Storey]

Hi everyone. I am trying to build some configuration groups with the
intention of keeping related configuration for some IPSEC VPNs etc nicely
contained in one spot - define all relevant configuration in a group and
apply it in one go, and also remove it *all* when you delete and remove the
configuration group. This way, hopefully, little bits and pieces dont get
left behind as VPNs are set up and torn down, ideally maintaining a cleaner
configuration.

But I have an odd situation. I am working with a cluster of SRX345, and it
seems that when ever I apply a config group with an st interface defined,
my default route disappears and some connected routes seem to disappear or
change from local to reject, and I lose the ability to manage the cluster
via SSH and have to use the console. The same does not happen on a
standalone SRX110.

For example, prior to applying the configuration, my routing table looks
like:

0.0.0.0/0          *[Static/5] 16:51:49
                    > to 10.32.31.1 via reth2.0
10.32.31.0/24      *[Direct/0] 16:51:49
                    > via reth2.0
10.32.31.230/32    *[Local/0] 16w6d 23:56:30
                      Local via reth2.0
10.32.31.231/32    *[Static/1] 6d 22:02:40
                      Receive
100.64.0.0/24      *[Direct/0] 16:51:49
                    > via reth3.0
100.64.0.1/32      *[Local/0] 1w0d 19:43:19
                      Local via reth3.0

And after applying the apply group:

10.32.31.230/32    *[Local/0] 16w6d 23:58:13
                      Reject
10.32.31.231/32    *[Static/1] 6d 22:04:23
                      Receive
100.64.0.1/32      *[Local/0] 1w0d 19:45:02
                      Reject
100.64.255.0/30    *[Direct/0] 00:00:04
                    > via st0.1
100.64.255.1/32    *[Local/0] 00:00:04
                      Local via st0.1

The very simple configuration group that I have defined is (I removed a lot
of it during debugging, and this is all that is left):

groups {
    EXAMPLE-VPN {
        interfaces {
            st0 {
                unit 1 {
                    description "DCN VPN to srx110:st0.0";
                    family inet {
                        address 100.64.255.1/30;
                    }
                }
            }
        }
    }
}

I set this with:

{primary:node0}[edit]
root at node0-srx345# set apply-groups EXAMPLE-VPN

Pretty straight forward I thought...

In the configuration, after the commit has completed (no complaints) I do
see my st0 configuration inherited, and all of the configuration for my
reth interfaces is also inherited, and show int terse shows them all there
with their IP addresses.

# show interfaces | display inheritance
...
reth2 {
    description UNTRUST;
    ##
    ## 'redundant-ether-options' was inherited from group 'SRX34X-CLUSTER'
    ##
    redundant-ether-options {
        ##
        ## '1' was inherited from group 'SRX34X-CLUSTER'
        ##
        redundancy-group 1;
    }
    unit 0 {
        family inet {
            address 10.32.31.230/24;
        }
    }
}
reth3 {
    description "AVAILABLE - Parent for ge-[05]/0/3";
    ##
    ## 'redundant-ether-options' was inherited from group 'SRX34X-CLUSTER'
    ##
    redundant-ether-options {
        ##
        ## '1' was inherited from group 'SRX34X-CLUSTER'
        ##
        redundancy-group 1;
    }
    unit 0 {
        family inet {
            address 100.64.0.1/24;
        }
    }
}
...
##
## 'st0' was inherited from group 'EXAMPLE-VPN'
##
st0 {
    ##
    ## '1' was inherited from group 'EXAMPLE-VPN'
    ##
    unit 1 {
        ##
        ## 'DCN VPN to srx110:st0.0' was inherited from group 'EXAMPLE-VPN'
        ##
        description "DCN VPN to srx110:st0.0";
        ##
        ## 'inet' was inherited from group 'EXAMPLE-VPN'
        ##
        family inet {
            ##
            ## '100.64.255.1/30' was inherited from group 'EXAMPLE-VPN'
            ##
            address 100.64.255.1/30;
        }
    }
}

So Im a bit struck as to what is going wrong here. The only smoking gun I
see is that my reth subinterfaces appear to be "hardware down" with the
parents claiming "physical link down" when the config group is applied.
Remove it and everything returns to normal.

Ive tried defining the st0 interface and unit 1 within the interfaces
stanza, but that didnt help. What am I missing? What can I look at? Am I
trying to do something that simply cant be done?