Dropping SSH connections over the internet

[Smoot Carl-Mitchell]

On Fri, 2008-01-11 at 12:37 -0500, Hal Burgiss wrote:

> I have had this happen while I am at the keyboard and it has almost
> always been during bad weather, like storms anywhere in the area. Not
> necessarily bad storms either, just some lightning here or there.
> Keepalive settings don't seem to help at all. Not all the disconnects
> are storm related, those are just the ones I have witnessed. My theory
> is some kind of packet corruption that "scares" the sshd end into not
> responding, and the connection dies because of tcp timeout. But just
> a wild theory. I'd love to know answer myself.

Packet corruption will not cause this.  The offending packet will be
dropped by the receiving TCP. (the packet data is checksummed). If you
have an intervening network device which also maintains connection state
which goes south, then the connection will drop.  For example a NATing
router which loses its connection state will cause the end to end TCP
connection to drop. This is true for routers which do port NATing where
the incoming connection port gets mapped to another port on the outbound
side to preserve port uniqueness on the outbound side of the connection.
The router has to maintain a list of used ports as each connection gets
NATed to the outbound IP address.  If it loses this port mapping, you
will lose the state of the end-to-end TCP connection.

Suppose you start an SSH connection from an internal IP 192.168.1.1 to
external site 11.1.1.1.  The connection address will be:

<TCP, 192.168.1.1, 50000, 11.1.1.1, 22)

where 50000 is an arbitrary source port. In practice it may be different
depending on the algorithm used to select unused source ports.

When the packet hits the NATing router it will translate the connection
to, say:

<TCP< 100.1.1.1, 28000, 11.1.1.1, 22)

where 100.1.1.1 is the address of the outbound interface on the router
and 2800 is the mapped port address.  Inbound packets reverse this
translation.  The router maintains a port map table which says inbound
packet addresses from 192.168.1.1,5000 get translated to 100.1.1.1,
28000 outbound.

If the router reboots, the port mapping is lost.  When the client TCP
sends another packet to the router from 192.168.1.1, 50000, there is no
mapping table and the router will send back a connection reset to the
client. The same thing will happen when the server trys to send a packet
back to the client.
-- 
Smoot Carl-Mitchell
System/Network Architect
email: smoot at tic.com
cell: +1 602 421 9005
home: +1 480 922 7313