Don't skip frames when connecting or paused

Remake/backport of a change that's already been applied to Kart. This is netsafe and can go into master, but I was working with the next branch so I wrote the change on it.