IP, IPv4, IPv6, Subnets, NAT & PortsAdvanced

Over the VPN, small responses work and anything large hangs forever

Symptoms

  • Through the new site-to-site VPN, curl https://api.internal/health (a 40-byte body) returns instantly.
  • curl https://api.internal/users (a 30 kB body) connects, completes TLS, sends the request — and then hangs until the read timeout.
  • SSH works, but cat of a large file over SSH freezes after the first screen.
  • The same requests work perfectly from inside the remote site.
$ curl -v https://api.internal/users -o /dev/null
* Connected to api.internal (10.50.0.8) port 443
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
> GET /users HTTP/1.1
< HTTP/1.1 200 OK
< Content-Length: 30412
* … hangs; 0 of 30412 bytes received

$ ping -M do -s 1372 10.50.0.8 -c 2      # 1372 + 28 = 1400-byte packet
2 packets transmitted, 2 received, 0% packet loss

$ ping -M do -s 1472 10.50.0.8 -c 2      # 1472 + 28 = 1500-byte packet
2 packets transmitted, 0 received, 100% packet loss   # no "Frag needed" reply either

$ tcpdump -ni wg0 host 10.50.0.8 and port 443
… 10.50.0.8.443 > 10.10.0.7.51002: Flags [P.], seq 1:1449, length 1448  ← never seen here
… 10.10.0.7.51002 > 10.50.0.8.443: Flags [.], ack 1, win 501            ← client keeps ACKing byte 1

$ ip link show wg0 | grep mtu
wg0: <POINTOPOINT,NOARP,UP> mtu 1420
# the remote firewall drops all ICMP "for security"

Investigate

Inspect areas in any order (0/7 inspected). When you think you know the root cause, commit to it.

Small vs large
The ping probes
The tunnel MTU
ICMP filtering
The TCP stream in `tcpdump`
TLS
The application