[j-nsp] For info: NAT64 on M7i using adaptive services

[Phil Mayers]

All,

Some time ago, I asked whether this was possible. tl;dr - it is, and 
mostly works with a few caveats.

The config I used was basically the same as that in the "Configuring 
Stateful NAT64 for Handling IPv4  Address Depletion" document from 
Juniper, but with one important change (that document assumes an MX 3D 
router - maybe the unvarnished config works there).

In that document, they give an interface config like so:

     ge-1/3/0 {
         description "IPv6 only domain";
         unit 0 {
             family inet;
             family inet6 {
                 service {
                     input {
                         service-set nat64;
                     }
                     output {
                         service-set nat64;
                     }
                 }
                 address 2001:db8::x/112;
             }
         }
     }

This did NOT work until I actually added an IPv4 address under "family 
inet" i.e.

   set interface ge-1/3/0 unit 0 family inet address ....

Without an actual IPv4 address, the M7i simply didn't work, and it very 
much did NOT give any useful logging or diagnostics - it just failed in 
weird ways, with weird log messages.

With this config, the NAT64 worked just fine. I then provisioned a 2nd 
NAT64 prefix, out of our local /48, and used a firewall filter that 
basically said this:

  1. from local addresses -> 64:FF9B::/96 permit
  2. from ::/0 -> 2001:db8:site::/96 permit

...and on IPv6 day we published a AAAA record for one of our IPv4-only 
webservers of the form:

wwwX.imperial.ac.uk. IN AAAA 2001:db8:site::155.198.x.y

...and inbound IPv6 traffic to that address came via the NAT64 function 
on the M7i.

This *mostly* worked, but we did have a few problems.

Firstly, we had problems with users whose IPv6 MTU was <1500. The NAT64 
function on the M7i (at least for JunOS 10.4R1.9, the version we ran at 
the time) does NOT appear to correctly translate the ICMPv6 messages 
into ICMPv4 required for PMTUd to work. The reverse direction (ICMPv4 
must-frag -> ICMPv6 must-frag) did appear to work, but we didn't test 
extensively.

We worked around this with TCP MSS clamping further into the network, 
but it was not very satisfactory.

Secondly, we saw problems where source IP/port combinations were re-used 
from the pool very quickly, and these re-used ports were then refused by 
the webserver with a TCP RST.

We ameliorated this by moving to:

services {
     nat {
         pool POOL {
             address y.y.y.0/24;
             port {
                 automatic {
                     random-allocation;
                 }
             }
         }
     }
}

...which seemed to reduce the problem to a very, very tiny level.

I am assuming this 2nd issue would not show up with a "real" NAT64 
deployment where IPv6 clients talk to a very diverse set of IPv4 hosts 
through the NAT64. But if your IPv6 clients are talking to a small 
number of (or singular!) host(s) via the NAT64, the fact that ports are 
recycled quicker than TIME_WAIT seems to cause problems for some IP stacks.

I did intend to investigate this further at some point, but it's looking 
unlikely I'll find the time now.

Finally, a note on throughput - the built-in services thingy on the M7i 
can do about 800Mbit/sec which is documented, but it's worth noting that 
I benchmarked >2000 sessions/sec ramp rate, and IIRC at this point the 
ASM CPU was not fully loaded. I also benchmarked >20k sessions, and 
again RAM was not fully loaded. So it's a respectable (if not huge) 
performance from such an old platform.

Hopefully this information is useful to someone.

Cheers,
Phil