[j-nsp] Separate internet transit network versus converged

[Mark Tees]

The main reasons were around security.

As Adam said it is a factor of me being absolutely sure that nothing
from the internet can effect the stability of the network here.

Effectively it comes down to ensuring:

* filters are set to drop traffic towards our infrastructure on public IP space

* filter various bogons from the Team Cymru feed /
http://www.team-cymru.org/bogon-dotted-decimal.html

* filters for Internet / transit / peering facing ports reset traffic
for almost everything back to BE class.

Our lo0 filters are already reasonably comprehensive. I still need to
go over and adjust the ddos protection config.

It is also a worry that somebody could forget to ensure a filter/QOS
is in place but that could be handled with commit scripts and by CRON
scripts that run some sanity checks on the network periodically.
Effectively, we need to ensure an interface is pretty much never
brought up without a filter and QOS applied to ensure stability and
security.

I like the separated edge functionality and a BGP free core is what we
are aiming for.

Will definitely at bare minimum have separate RR's be it VPNv4 or plain inet.












On 29 March 2016 at 12:32, Mark Tees <marktees at gmail.com> wrote:
> On 27 March 2016 at 22:02, Saku Ytti <saku at ytti.fi> wrote:
>> On 27 March 2016 at 13:37, Mark Tinka <mark.tinka at seacom.mu> wrote:
>>
>> Hey,
>>
>>> As costs and management got out of control, they run l3vpn's and
>>> Internet in the same chassis, but on different line cards.
>>>
>>> Eventually, everything converged.
>>
>> I tend to agree. If there is significant CAPEX delta buying L3 MPLS
>> VPN + HQoS capable boxes and Internet transit capable boxes, then it
>> might make sense to buy two networks, as likely L3 MPLS VPN traffic
>> rates are very minor but service requires much higher touch hardware.
>> But I don't suspect the delta is high these days and more importantly
>> I don't think the IP device CAPEX is very large part of TCO.
>>
>> Another justification might be, if the software stack is very
>> different, but for L3 MPLS VPN will need all services IP Transit uses,
>> so having IP Transit on same devices does not require turning on
>> additional services, so it is not really creating additional risk on
>> the premium services.
>> If your bread and butter would be L2 VPN, then separating IP transit
>> on another edge device might be very prudent, as you could do away
>> with BGP and IP lookups completely on the edge.
>>
>> I am fan of Internet-in-VRF, mainly because global-table is special
>> case and it's hard to import/export route between global and VRF, and
>> this complexity has forced me to do some really bad/ugly solutions,
>> which would have been clean and simple if Internet was in VRF. It does
>> not have to mean ugly traceroutes, you can configure device on TTL
>> exceeded to pop labels and do IP lookup in transit for returning TTL
>> exceeded messages. This does not even exclude BGP free core, as your
>> core can have static route pointing to anycast IGP loopback added to
>> all edge devices with full BGP, so TTL exceeded message goes to
>> closest edge device for lookup, probably creating less than
>> millisecond additional delay on traceroute.
>
> Yes, ICMP tunnelling possibly seems to be what I need for that.
>
>>
>> OP states that mistakes in IGP do not affect each other, but mistakes
>> in the 'core' network IGP where the L2 circuits run, still break
>> everything.
>
> True, there is shared risk here but not in all cases for us.
>
>>
>> I'd say you need solid arguments to separate  the networks, state
>> exact specific problems and how it solves them, default to converged
>> network in absence of such arguments. For background it might be
>> interesting to hear what problems you've had, what is prompting this
>> separate edge.
>
> Agree. Rather than being paranoid about it I need a strict list.
>
>
>>
>>
>> --
>>   ++ytti
>
>
>
> --
> Regards,
>
> Mark L. Tees



-- 
Regards,

Mark L. Tees