[j-nsp] flow-active-timeout apparently ignored on MX204

[Leon Kramer]

Hello,

I see a problem with IPFIX Flow Export on an Juniper MX204 device.

We have approximately 80 Gbps Egress and 40 Ingress Traffic going
through this router with input sampling enabled on every interface and
output sampling disabled. The sampling rate is set to 2500 and works
basically but the flows seem flawed.
This has been noticed when we received a notification of supposedly
250 Gbps egress DDOS attack. Analysis showed that there was only a
single flow sent by the MX204 router for this particular flow. The
flow was probably a backup job through an IPSEC-Tunnel with constantly
100 Mbps for around 12 hours resulting in the very big flow export of
250 Gbps. So it seems that the MX204 is accumulating active flows for
even a very long period ignoring the flow-active-timeout setting. This
of course is creating trouble for flow based services.

Looking at flow exports / second statistics for MX204 we also see a
very flat line once there are around 3200 flows/second. In comparison:
Our MX10003 running with the same version and configuration has a nice
curve for flow exports / second.

We have had MX480 running with IPFIX and same sampling rate without
any issues.  I wonder if The MX204 really cannot handle more than 3200
flows.

This situation with active flow accumulation could only be improved by
lowering the sample rate to even lower values. After doing this the
active-flow-timeout apparently also worked again. As MX480 worked
perfectly fine I hope the MX204 can do so either.

If anyone can help with this issue to improve the situation I am
thankful for your help.


Kind Regards
Leon Kramer



> show version
Model: mx204
Junos: 18.4R2-S3





> show services accounting flow inline-jflow fpc-slot 0
  Flow information
    FPC Slot: 0
    Flow Packets: 13228303295, Flow Bytes: 6472971946326
    Active Flows: 41659, Total Flows: 16231808905
    Flows Exported: 17168991038, Flow Packets Exported: 4317826420
    Flows Inactive Timed Out: 7017316323, Flows Active Timed Out: 8789666564
    Total Flow Insert Count: 7442142341

    IPv4 Flows:
    IPv4 Flow Packets: 13195410076, IPv4 Flow Bytes: 6446418303883
    IPv4 Active Flows: 41547, IPv4 Total Flows: 16203134906
    IPv4 Flows Exported: 17139188812, IPv4 Flow Packets exported: 4290996954
    IPv4 Flows Inactive Timed Out: 7003594699, IPv4 Flows Active Timed
Out: 8774863492
    IPv4 Flow Insert Count: 7428271414

    IPv6 Flows:
    IPv6 Flow Packets: 32893219, IPv6 Flow Bytes: 26553642443
    IPv6 Active Flows: 112, IPv6 Total Flows: 28673999
    IPv6 Flows Exported: 29802226, IPv6 Flow Packets Exported: 26829466
    IPv6 Flows Inactive Timed Out: 13721624, IPv6 Flows Active Timed
Out: 14803072
    IPv6 Flow Insert Count: 13870927





> show services accounting errors inline-jflow fpc-slot 0
  Error information
    FPC Slot: 0
    Flow Creation Failures: 15998072
    Route Record Lookup Failures: 9623950, AS Lookup Failures: 9623950
    Export Packet Failures: 174967
    Memory Overload: No, Memory Alloc Fail Count: 0

    IPv4:
    IPv4 Flow Creation Failures: 15998069
    IPv4 Route Record Lookup Failures: 9283780, IPv4 AS Lookup Failures: 9283780
    IPv4 Export Packet Failures: 174836

    IPv6:
    IPv6 Flow Creation Failures: 3
    IPv6 Route Record Lookup Failures: 340170, IPv6 AS Lookup Failures: 340170
    IPv6 Export Packet Failures: 131

> show services accounting status inline-jflow fpc-slot 0
  Status information
    FPC Slot: 0
    IPV4 export format: Version-IPFIX, IPV6 export format: Version-IPFIX
    BRIDGE export format: Not set, MPLS export format: Not set
    IPv4 Route Record Count: 1612737, IPv6 Route Record Count: 175463,
MPLS Route Record Count: 0
    Route Record Count: 1788200, AS Record Count: 835618
    Route-Records Set: Yes, Config Set: Yes
    Service Status: PFE-0: Steady
    Using Extended Flow Memory?: PFE-0: No
    Flex Flow Sizing ENABLED?: PFE-0: No
    IPv4 MAX FLOW Count: 4891446, IPv6 MAX FLOW Count: 349389
    BRIDGE MAX FLOW Count: 1024, MPLS MAX FLOW Count: 1024





> show configuration forwarding-options
sampling {
    instance {
        export_flows {
            input {
                rate 2500;
                run-length 0;
                max-packets-per-second 65535;
            }
            family inet {
                output {
                    flow-server x.x.x.x {
                        port 4739;
                        source-address x.x.x.x;
                        version-ipfix {
                            template {
                                IPv4;
                            }
                        }
                    }
                    flow-server x.x.x.x {
                        port 4739;
                        source-address x.x.x.x;
                        version-ipfix {
                            template {
                                IPv4;
                            }
                        }
                    }
                    inline-jflow {
                        source-address x.x.x.x;
                    }
                }
            }
            family inet6 {
                output {
                    flow-server x.x.x.x {
                        port 4739;
                        source-address x.x.x.x;
                        version-ipfix {
                            template {
                                IPv6;
                            }
                        }
                    }
                    flow-server x.x.x.x {
                        port 4739;
                        source-address x.x.x.x;
                        version-ipfix {
                            template {
                                IPv6;
                            }
                        }
                    }
                    inline-jflow {
                        source-address x.x.x.x;
                    }
                }
            }
        }
    }
}





> show configuration services
flow-monitoring {
    version-ipfix {
        template IPv4 {
            flow-active-timeout 10;
            flow-inactive-timeout 10;
            ipv4-template;
        }
        template IPv6 {
            flow-active-timeout 10;
            flow-inactive-timeout 10;
            ipv6-template;
        }
    }
}




> show configuration chassis
routing-engine-power-off-button-disable;
aggregated-devices {
    ethernet {
        device-count 1;
    }
}
fpc 0 {
    pic 0 {
        port 0 {
            speed 100g;
        }
        port 1 {
            speed 100g;
        }
        port 2 {
            speed 100g;
        }
        port 3 {
            speed 100g;
        }
    }
    pic 1 {
        number-of-ports 0;
    }
    sampling-instance export_flows;
    inline-services {
        flow-table-size {
            ipv4-flow-table-size 14;
            ipv6-flow-table-size 1;
        }
    }
}