(I am working on this email as a general send out on policy on securing systems but i guess it is worth sticking on the blog)
1) I guess the thing to start with is mentioning that "public key cryptography is very secure indeed".
You only distribute the public part of the key, you know.. post it on
the internet, attach it to your emails, print it on a tee-shirt. It
doesn't matter what you do with it in terms of confidentiality. 
However instead of the password, it is the private key that becomes the
weak link in the security process. Protect that with your life, well
maybe not, but it is very important. Keeps it on your computer and don't
copy it to remote machines. Ever!
2) the next thing is that "You are going to get hacked eventually"
Given enough time any non-trivial enterprise is going to have a whole
bunch of people, systems, software and screw-ups, eventually one of
which is going to let some naughty person have access to something they
should not have access to.
Hence be aware of what the effect of the compromise of this system will have on the integrity of the whole system.
If you are writing some tool that requires administrator rights, and
runs on the internet, and has passwords for the database and users
systems, think twice, think 3 times. Go to the pub get drunk, then think
a fourth time. Because you are going to look mighty silly when its your
3) There are 2 golden rules for mitigating the extent of an intrusion
i) The principle of least privilege
Run services as a user with the least possible rights to complete the
task. JDBC database connection users should be database specific, if
your tomcat is hacked, then your potential losses are limited to that
particular database, and not (if you are connecting with root, or dba)
the whole 27 databases on that mysqld.
ii) The principle of separation of privileges
If your script needs to install a user, or needs to
install a package as well as some other stuff.
Then write 2 scripts. one to
install the package or the user, and one to do the normal day-to-day
stuff. And protect the admin script as a core part of the system.
4) There are always going to be exceptions. You should know what they are, and why they are there.
Your core infrastructure is an example, your installation tools and
scripts, backup tools have to have access to stuff. However this is
where you have to be sensible.
Use tools for important tasks when they are available, rather than write your own stuff. If you find yourself chmod'ing 777 then you are an idiot, turn around and walk slowly away from the computer... only joking ;-)
 in openssh authorized_keys format the file contains a comment if
which is the same as your user-id, gives an attacker a valid account
name to work with.