I do have my email entered correctly on the Mail/To: line.
It may be that your host/ISP does not permit email from your web server. Try installing and configuring a SMTP plugin and see if that solves it for you.
This one works for my test basement server.
http://wordpress.org/plugins/wp-smtp/
There are others to try as well.