Restore "Fix connections not timing out if clients haven't joined yet" from !2631

Restores !2631 (closed) which fixes a bug where a client doesn't close correctly when the client closes connection or times out in the middle of file downloading so the server will keep sending packets to that node even if no longer responds so it would clog the server's bandwith with ghost clients.