[c-nsp] Problem encountered while securing NTP

[Justin Shore]

Given the recent NTP PSIRT from Cisco (cisco-sa-20090923-ntp) I decided 
to spend the morning revisiting my NTP practices to lessen the chance of 
getting kicked in the teeth by this router-crashing bug later on.  In my 
networks I usually have a pair (or more sometimes) of border routers as 
local NTP servers, advertising themselves internally as stratum-3.  They 
use stratum-1s on the Internet in geographically diverse areas.  The 
local network devices are configured to use both of the local NTP 
servers with authentication.

Here's a border router's config:

ntp authentication-key 1 md5 BLAHBLAHBLAH 7
ntp authenticate
ntp trusted-key 1
ntp source Loopback1
ntp access-group peer 5
ntp access-group serve-only 6
ntp master 3
ntp update-calendar
ntp server a.a.a.a prefer
ntp server b.b.b.b
ntp peer LOCAL.BORDER.RTR.TWO key 1
ntp server c.c.c.c
!
access-list 5 remark NTP Peers
access-list 5 permit 127.127.7.1
access-list 5 permit a.a.a.a.
access-list 5 permit b.b.b.b
access-list 5 permit c.c.c.c
access-list 5 permit LOCAL.BORDER.RTR.ONE
access-list 5 permit LOCAL.BORDER.RTR.TWO
access-list 5 deny   any log
!
access-list 6 remark NTP Serve-Only
access-list 6 permit LOCAL.NETWORK.MANAGEMENT.PREFIXES x.x.x.mask
access-list 6 permit LOCAL.RIR.ASSIGNED.PREFIXES x.x.x.mask
access-list 6 deny   any log

Here's a client device's config:

ntp authentication-key 1 md5 105D020D0619061B4851 7
ntp authenticate
ntp trusted-key 1
ntp clock-period 17179513
ntp source Vlan224
ntp access-group serve-only 6
ntp server 64.71.98.51 key 1
ntp server 64.71.98.59 key 1
!
access-list 6 remark NTP Serve-Only
access-list 6 permit LOCAL.NETWORK.MANAGEMENT.PREFIXES x.x.x.mask
access-list 6 permit LOCAL.RIR.ASSIGNED.PREFIXES x.x.x.mask
access-list 6 deny   any log
(Same as above for simplicity's sake)

That's my standard NTP config that I run on several networks without any 
real problems, normally.  There may be other or better ways to help 
secure NTP that I either haven't implemented yet (iACLs, CoPP) or don't 
know about but I'm always open to suggestions.

The problem I'm running into today is that the 'access-group peer' 
statements on the NTP servers are matching local clients with ACL 6 as 
well as configured stratum-1 peers (successfully matching the peers at 
that).  The local clients should be matched with the 'access-group 
serve-only' ACL 6, but for some reason they are not.

Strangely enough I have ACE counters increasing on ACL 6's lines that 
correspond to my network infrastructure, even though the 'deny any log' 
ACE in ACL 5 is also reporting denied hits for the very same subnets. 
It's as if I've freaked out NTP.  I made changes to the ACL earlier 
today as part of my initial review.  Then NTP for the network 
infrastructure stopped working.  Could this be an IOS bug by chance?  My 
next step is to blow away the NTP config from the borders, hopefully 
killing the NTP process at the same time, and re-add the config.

Thoughts before I blow away the config and start over?
  Justin