A week ago we were experiencing ssh attacks that were preventing login and slowing our server dramatically. Most likely the attack was from compromised machines running software bots looking for weak passwords.
This is not normally an issue for us as none of the FTP users have shell access (they can not login via SSH or terminal) and the root user account can not SSH in either. In fact there is only one account that has ssh login permission. Yet the machine was running dog slow and no users could login via FTP and we couldn’t login via ssh at all. Most worrying for a while.
I logged in via remote desktop and a quick check of the logs showed constant reports of failed login attempts – ah ha.
The problem was that when the ssh daemon was confronted with an incorrect password, it was checking against the password database multiple times, which was overwhelming the password services. A better explanation from [1] is “every time the attacking machine tried another key/password, it would spawn a new sshd process, which had to communicate with the password services (com.apple.SecurityServer) in order to validate the password. Eventually what ended up happening is that there were so many requests to the password services that they basically ended up just hanging, and anything that required a password: ssh, ftp, etc, just stopped working.”
Firstly I changed the default /etc/sshd_config file, un-commenting this line and changing yes to no:
ChallengeResponseAuthentication no
This edit does not stop attackers from trying but it does protect the password services from the attack.
Next I installed the most excellent sshdfilter [2] – a perl daemon that actively monitors ssh logins and detects signs of intrusion attempts and then blocks the attacking IP addresses. Blocked addresses are saved in between reboots and the he startup script is here:
/Library/LaunchDaemons/net.jonbell.sshdfilter.plist
(this runs /etc/sshdfilterLoad.sh which runs /etc/firewallrules)
To receive email notifications for each block, edit the /etc/sshdfilterrc file (the mail= and mail policy sections – there are comments in the policy).
“The default setting is to block most failed logins after 5 attempts, some common invalid logins after 0 attempts, incorrect root logins after 2 attempts, and logins to non-existent accounts after 3 attempts. Counters are reset upon a valid login. These thresholds can be modified in the sshdfilterrc file if desired. It is not currently setup to expire the blocks (even if you set it here in the configuration file, they will remain in the persisting file).”
[1] http://www.slicksurface.com/blog/2008-06/ssh-attack-and-password-problems-on-os-x
[2] Mac OS X sshdfilter Installer:
http://projects.seas.columbia.edu/sshdfilter/sshdfilter_mac.zip
sshdfilter Project Home:
http://www.csc.liv.ac.uk/~greg/sshdfilter/