[j-nsp] 880 second RPM probes

[Ron Stroobant]

Hi Folks,

This is my first post to Puck juniper-nsp. I'm hoping someone has seen this issue before and can comment.

We enable RPM probes on our Juniper network. They poll the RPM server (an EX4200 VC) every 5 mins. Approximately once/month/device we will see an 880 second Ping RTT or Jitter RTT probe result. That one result is so high it ruins the averages for the entire month. The issue also seems to be linked to lost probes. When we see a device that has lost probes, it will usually show the 880 second probe issue as well. I believe that Juniper, instead of ignoring lost probes, is actually assigning a max RTT value to a lost probe of 880 seconds.

There are no PR's about this and I have had a JTAC case open for 3 months and worked it thru the escalation levels. There have been no explanations or workarounds provided. Finally, a couple weeks ago I created my own workaround using a network automation approach. Now when a "bad" probe occurs, the device will automatically delete the results before our performance poller sees it. It would be nice to fix the root cause but we had no choice. The issue had to be fixed immediately.

Here is a summary output. I can provide more detail if someone wants to see more but I'm only trying to find out if anyone else has come across this issue.

rstroobant at xxxxxxxxcpswt001> show services rpm probe-results | match 880
        Samples: 324929, Minimum: 2201 usec, Maximum: 880456303 usec, Average: 5736 usec, Peak to peak: 880454102 usec, Stddev: 1544584 usec,
        Samples: 163530, Minimum: 0 usec, Maximum: 880453581 usec, Average: 5831 usec, Peak to peak: 880453581 usec, Stddev: 2177240 usec,
        Samples: 161361, Minimum: 1 usec, Maximum: 880453455 usec, Average: 5909 usec, Peak to peak: 880453454 usec, Stddev: 2191823 usec,
        Samples: 324930, Minimum: 2038 usec, Maximum: 880454315 usec, Average: 5662 usec, Peak to peak: 880452277 usec, Stddev: 1544578 usec,
        Samples: 164331, Minimum: 0 usec, Maximum: 880451603 usec, Average: 5822 usec, Peak to peak: 880451603 usec, Stddev: 2171922 usec,
        Samples: 160556, Minimum: 1 usec, Maximum: 880451057 usec, Average: 5959 usec, Peak to peak: 880451056 usec, Stddev: 2197305 usec,


Many thanks.............Ron