[c-nsp] Catching OSPF hello problems

[Jee Kay]

I currently have a not-entirely-right metro ethernet solution that
seems to go unidirectional for a few seconds every so often. We've had
to change our OSPF timers across these links to hello 1, dead 9 to
stop OSPF flapping when this occurs.

(Un)fortunately, this means we no longer get big red alarmbells going
off when the line does its pseudo-flap thing. We do however get logs
that looks very much like the following:

Sep 26 07:20:20.701 CDT: OSPF: Send hello to 224.0.0.5 area 0 on
FastEthernet2/1 from x.y.z.b
Sep 26 07:20:20.717 CDT: OSPF: End of hello processing
Sep 26 07:20:20.937 CDT: OSPF: Rcv hello from 216.210.122.174 area 0
from FastEthernet2/1 x.y.z.a
Sep 26 07:20:20.937 CDT: OSPF: End of hello processing
Sep 26 07:20:21.701 CDT: OSPF: Send hello to 224.0.0.5 area 0 on
FastEthernet2/1 from x.y.z.b
Sep 26 07:20:21.717 CDT: OSPF: End of hello processing
Sep 26 07:20:21.945 CDT: OSPF: Rcv hello from 216.210.122.174 area 0
from FastEthernet2/1 x.y.z.a
Sep 26 07:20:21.945 CDT: OSPF: End of hello processing
Sep 26 07:20:22.701 CDT: OSPF: Send hello to 224.0.0.5 area 0 on
FastEthernet2/1 from x.y.z.b
Sep 26 07:20:22.717 CDT: OSPF: End of hello processing
Sep 26 07:20:23.701 CDT: OSPF: Send hello to 224.0.0.5 area 0 on
FastEthernet2/1 from x.y.z.b
Sep 26 07:20:23.717 CDT: OSPF: End of hello processing
Sep 26 07:20:24.701 CDT: OSPF: Send hello to 224.0.0.5 area 0 on
FastEthernet2/1 from x.y.z.b
Sep 26 07:20:24.717 CDT: OSPF: End of hello processing

(I'm guessing gmail is going to wrap that to unreadable death, so have
also attached it)

Basically what has happened here is that for 3 seconds we're sending
hellos, but not receiving any. After this, the hellos start being
received again and everyone is once again happy.

What I'm looking for is some way to watch for this happening (send 2
or 3 consecutive hellos on the same interface without receiving one
back) and alert on it.

Can anyone think of a relatively easy way to do this?

Thank,
Ras
-------------- next part --------------
Sep 26 07:20:20.701 CDT: OSPF: Send hello to 224.0.0.5 area 0 on FastEthernet2/1 from x.y.z.b
Sep 26 07:20:20.717 CDT: OSPF: End of hello processing
Sep 26 07:20:20.937 CDT: OSPF: Rcv hello from 216.210.122.174 area 0 from FastEthernet2/1 x.y.z.a
Sep 26 07:20:20.937 CDT: OSPF: End of hello processing
Sep 26 07:20:21.701 CDT: OSPF: Send hello to 224.0.0.5 area 0 on FastEthernet2/1 from x.y.z.b
Sep 26 07:20:21.717 CDT: OSPF: End of hello processing
Sep 26 07:20:21.945 CDT: OSPF: Rcv hello from 216.210.122.174 area 0 from FastEthernet2/1 x.y.z.a
Sep 26 07:20:21.945 CDT: OSPF: End of hello processing
Sep 26 07:20:22.701 CDT: OSPF: Send hello to 224.0.0.5 area 0 on FastEthernet2/1 from x.y.z.b
Sep 26 07:20:22.717 CDT: OSPF: End of hello processing
Sep 26 07:20:23.701 CDT: OSPF: Send hello to 224.0.0.5 area 0 on FastEthernet2/1 from x.y.z.b
Sep 26 07:20:23.717 CDT: OSPF: End of hello processing
Sep 26 07:20:24.701 CDT: OSPF: Send hello to 224.0.0.5 area 0 on FastEthernet2/1 from x.y.z.b
Sep 26 07:20:24.717 CDT: OSPF: End of hello processing