email notifications will use phpmail or sendmail.  You can add your own smtp account. If you aren't getting any it could be a cron job issue.

As for the site speed - consider about installing Nginx and/or eAccelerator on your server.

also a lot of images and plugin files can be compressed, its a long process but can shave a few seconds of your sites loading speed.

also having a VPS rather than shared hosting would make a big difference as you can install your own php/apache cache scripts.