Almost all webhosts/isp offer both so either way you are still open to any security holes, and as said above the email is often retained even if read via pop3/imap. I know I keep all my email, what is the point having a receipt or invoice sent that is deleted once I have read it? Unless you go to the bother of setting up your own mail server and locking it down to no webmail access and only your ip can connect, there are security issues. Also unless you set up encryption with pop3 your details are sent plainly over the internet which is less secure than using a https connection with webmail.
*edit*
Johnny - you may wish to look into thunderbird, it is a bit more secure than outlook (and looks prettier!)