[c-nsp] [2nd Try] Decent Network Documentation and Topology

[Wink]

I would add that a tool like Alfresca, or an internal engineering wiki, 
is a great way to provide a quick way of seeing customer specific 
engineering data, POP-specific engineering data, and so on.  You can 
create banners and tie-ins such as a list of the last 10 tickets opened 
on a device or for a customer, recent engineering changes, diagrams, 
general notes to other engineers etc in plain english on the front page 
of that customers information "portal." Especially put in links to 
documents on shared drives etc that people can reference.

Program into your people that no matter what they are doing, they need 
to immediately go to this wiki/portal and READ IT and if they make 
changes to an environment, DOCUMENT IT.  Integrate this into your 
engineers annual reviews.  This is the hard part if you're not already 
doing something like this.  But it has to be done at every level and it 
needs to be apparent at every engineering meeting.  If something falls 
through the cracks everyone should be asking why.



Justin Shore wrote:
> Kim Onnel wrote:
>   
>>> From your perspective, what is to be considered enough documentation to
>>> troubleshoot problems in a corp.(switches + PIX + WAN routers) environment??
>>>       
>
> Kim,
>
> Unfortunately this is about as vague of a question and it gets. 
> Basically you have to document everything.
>
>
> High-level logical diagrams
> 	Include	device names
> 		management Loopback IPs
> 	Overall 40k foot view of the SP
> 	10k foot view of each POP
>
> Low-level physical diagrams
> 	Label	interfaces
> 		trunks (and the VLANs they carry)
> 		link speeds by varying the width of the lines in Visio
>
> IGP diagrams
> 	Include	area information
> 		IS-IS level information
> 		Special config details like SA, TSA, NSSA etc
> 		If it's designed for equal-cost load-balancing indicate that.
>
> EGP diagrams
> 	Note	peering IPs, both your's and their's
> 		ebgp-multihop details
> 		security measures
> 			TTL limiting
> 			MD5
> 			CoPP
> 			rACLs
> 			max prefix limits
> 		expected data
> 			full tables
> 			default route
> 			RTBH trigger routes
> 		expected baseline volume and date
> 			ie, 240k routes in the full table, 2/08
> 		basic filtering measures
> 			sanity
> 			locally originated prefixes
> 			BOGON
> 			transit
> 		dampening
>
> iBGP diagrams
> 	All the same stuff from the EGP diagrams plus advertised prefixes per 
> speaker
>
> You should document the implementation and use of uRPF, where it's 
> deployed, what L3 interfaces it's deployed on, what version is deployed, 
> drop/exception ACLs, integration with RTBH, etc.
>
> Document AAA policies, implementations, local auth backup accounts, etc. 
>   Include all management access details such as access-class statements 
> on VTYs and the ACLs that go with them.  Lock and Key configuration. 
> SNMP community details and ACLs.  TACACS or RADIUS config.  CoPP config. 
>   Core dump details.  HTTP(S) config and ACLs.  SCP server config. 
> Netflow config.  Role-based CLI config.  All management functions.
>
> Document every single line of config that pertains to each and every 
> customer.  Does a customer have a specific static route?  Name the 
> static route and document it in their file.  Does a customer depend on 
> special eBGP config, route-maps, prefix-lists, and community-lists? 
> Name them appropriately, give them descriptions, and document them.  QoS 
> settings for customer transit traffic?  Same as above.  Customer ACLs? 
> Ditto.
>
> Document QoS policies.  Take your high-level diagram from above and show 
> QoS re-coloring policies graphically.
>
> Document your NTP infrastructure including the external lower stratum 
> peers, radio clocks, etc.  Include NTP access lists for clients and peers.
>
> Document your syslog infrastructure (everyone forgets this one). 
> Inevitably you will disable certain syslog messages from being sent on 
> certain devices (link state messages from your dialin servers for 
> example).  Document anything that varies from your standard logging 
> config template such as when you disable messages of certain types. 
> Document that standard logging options in a template that you apply to 
> all devices (logging monitor debug, no logging con, logging trap debug, 
> logging facility local5, service timestamps..., logging buffered 65536, 
> etc).
>
> Document contracts, contacts, contact procedures, notification policies, 
> authorized contacts, meet-point address, CIDs, peering IPs, NOC numbers, 
> account team numbers, and everything else you can think of to document 
> about your peers.  You will need this information sooner rather than later.
>
>
> Document your policies and procedures with regards to DMCA Takedown 
> Notices.  Document your policies and procedures for dealing with 
> infected customers, spammers, crackers, and other network nuisances. 
> Document what you do with bandwidth hogs.  Document your policies and 
> procedures with regards to CALEA (if you haven't already done this then 
> you are already breaking the law...).
>
> Document you policies and procedures for turning up new customers in 
> each of the possible and supported connection methods.
>
> Document your public and private IP usage.  Document what subnet 
> corresponds to, which devices it touches and what interface on each 
> device is in the subnet (label the interfaces too).
>
> Document your patch panels.  Document your fiber panels.  Do you have a 
> SMF simplex link going through multiple fiber panels, crossing bundles, 
> between multiple sites?  You'd better have that documented or you should 
> just get used to outages.
>
>
> Basically you have to document everything.  You should have template 
> configs for routers, switches, firewalls, etc.  You should have template 
> configs for interfaces (customer-facing, core-facing, management-facing, 
> L2, L3, VRF, w/ HSRP, wo/ HSRP, SVIs, with customer-ACLS,
> etc).  You should have eBGP (customer, transit, and upstream peering) 
> and iBGP (core, distribution, access edge for RTBH only) template 
> configs.  AAA templates.  MPLS templates.  Mcast templates.
>
> Document everything.  When you think you've documented everything look 
> again and you'll find something else you forgot to document.  Rinse. 
> Rest.  Repeat.
>
> Justin
> _______________________________________________
> cisco-nsp mailing list  cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>
>
>