[j-nsp] Data transfers break after approximately 70Mb through SRX VPNs

[Harry Lewins]

Hello everyone,

I have a strange issue where traffic that is routed over a site to site VPN fails after approximately 70Mb of transfer. There are no lifesizes associated with the VPN, and the lifetimes are not close to being exceeded each time. There is nothing in the logs to indicate anything wrong, the VPN does not rekey etc.

I've tried various file transfer types and they all fail - SFTP, FTP, etc. They fail with the remote side closing the session - it looks like it has received a TCP RST but the sender still has the socket open - eventually it times out of course.

I've run the transfer to the same end points bypassing the VPN, and it works fine every time (this traffic does not traverse the SRXs). From packet captures from the sending side it looks like there are a ton of retransmissions and duplicate ACKs. Unfortunately I've not been able to perform a capture on the receiving side, though I hope to do that tomorrow.

I've got a few thoughts - one is all the interfaces are 1GB but the WAN link is 10M. Possibly this is something to do with the buffers overflowing somewhere? I've tried tweaking the MSS on both SRXs to below 1300 in case its some MTU issue on the WAN, but again, it still happens.

Anyone got any ideas? I performed a debug to see if either Juniper was initiating the TCP RST and it doesn't look like they are. Just to fill you in a bit more, the topology looks a bit like this

Server --> ASA --> Core switch --> SRX --> WAN Firewall (ASA) --> WAN --> WAN Firewall (ASA) --> SRX --> Core Switch --> ASA --> Server

I  bypassed the SRXs for a test - they are only there for encryption, as there is a requirement that traffic over the WAN is encrypted.

I've tried disabling tcp-session-checking, and also wanted to try rate limiting the traffic but as this is an SRX240 cluster, you can't do this on clustered interfaces apparently.

Any ideas would be appreciated!

Thanks for your time

Harry