I had been noticing that SSH and rsync were unusually slow to connect on my Raspberry Pi running Raspbian. I had initially assumed it was due to the small CPU, but after doing some searching around, I had found that adding an extra setting in SSHd’s config fixed the problem.
in /etc/ssh/sshd_config, add:
UseDNS no
Then restart sshd with “service ssh restart”.