[c-nsp] [2nd Try] Decent Network Documentation and Topology

[Justin Shore]

Kim Onnel wrote:
>> From your perspective, what is to be considered enough documentation to
>> troubleshoot problems in a corp.(switches + PIX + WAN routers) environment??

Kim,

Unfortunately this is about as vague of a question and it gets. 
Basically you have to document everything.


High-level logical diagrams
	Include	device names
		management Loopback IPs
	Overall 40k foot view of the SP
	10k foot view of each POP

Low-level physical diagrams
	Label	interfaces
		trunks (and the VLANs they carry)
		link speeds by varying the width of the lines in Visio

IGP diagrams
	Include	area information
		IS-IS level information
		Special config details like SA, TSA, NSSA etc
		If it's designed for equal-cost load-balancing indicate that.

EGP diagrams
	Note	peering IPs, both your's and their's
		ebgp-multihop details
		security measures
			TTL limiting
			MD5
			CoPP
			rACLs
			max prefix limits
		expected data
			full tables
			default route
			RTBH trigger routes
		expected baseline volume and date
			ie, 240k routes in the full table, 2/08
		basic filtering measures
			sanity
			locally originated prefixes
			BOGON
			transit
		dampening

iBGP diagrams
	All the same stuff from the EGP diagrams plus advertised prefixes per 
speaker

You should document the implementation and use of uRPF, where it's 
deployed, what L3 interfaces it's deployed on, what version is deployed, 
drop/exception ACLs, integration with RTBH, etc.

Document AAA policies, implementations, local auth backup accounts, etc. 
  Include all management access details such as access-class statements 
on VTYs and the ACLs that go with them.  Lock and Key configuration. 
SNMP community details and ACLs.  TACACS or RADIUS config.  CoPP config. 
  Core dump details.  HTTP(S) config and ACLs.  SCP server config. 
Netflow config.  Role-based CLI config.  All management functions.

Document every single line of config that pertains to each and every 
customer.  Does a customer have a specific static route?  Name the 
static route and document it in their file.  Does a customer depend on 
special eBGP config, route-maps, prefix-lists, and community-lists? 
Name them appropriately, give them descriptions, and document them.  QoS 
settings for customer transit traffic?  Same as above.  Customer ACLs? 
Ditto.

Document QoS policies.  Take your high-level diagram from above and show 
QoS re-coloring policies graphically.

Document your NTP infrastructure including the external lower stratum 
peers, radio clocks, etc.  Include NTP access lists for clients and peers.

Document your syslog infrastructure (everyone forgets this one). 
Inevitably you will disable certain syslog messages from being sent on 
certain devices (link state messages from your dialin servers for 
example).  Document anything that varies from your standard logging 
config template such as when you disable messages of certain types. 
Document that standard logging options in a template that you apply to 
all devices (logging monitor debug, no logging con, logging trap debug, 
logging facility local5, service timestamps..., logging buffered 65536, 
etc).

Document contracts, contacts, contact procedures, notification policies, 
authorized contacts, meet-point address, CIDs, peering IPs, NOC numbers, 
account team numbers, and everything else you can think of to document 
about your peers.  You will need this information sooner rather than later.


Document your policies and procedures with regards to DMCA Takedown 
Notices.  Document your policies and procedures for dealing with 
infected customers, spammers, crackers, and other network nuisances. 
Document what you do with bandwidth hogs.  Document your policies and 
procedures with regards to CALEA (if you haven't already done this then 
you are already breaking the law...).

Document you policies and procedures for turning up new customers in 
each of the possible and supported connection methods.

Document your public and private IP usage.  Document what subnet 
corresponds to, which devices it touches and what interface on each 
device is in the subnet (label the interfaces too).

Document your patch panels.  Document your fiber panels.  Do you have a 
SMF simplex link going through multiple fiber panels, crossing bundles, 
between multiple sites?  You'd better have that documented or you should 
just get used to outages.


Basically you have to document everything.  You should have template 
configs for routers, switches, firewalls, etc.  You should have template 
configs for interfaces (customer-facing, core-facing, management-facing, 
L2, L3, VRF, w/ HSRP, wo/ HSRP, SVIs, with customer-ACLS,
etc).  You should have eBGP (customer, transit, and upstream peering) 
and iBGP (core, distribution, access edge for RTBH only) template 
configs.  AAA templates.  MPLS templates.  Mcast templates.

Document everything.  When you think you've documented everything look 
again and you'll find something else you forgot to document.  Rinse. 
Rest.  Repeat.

Justin