Originally posted by Gregg Bolinger: I really like http://www.dreamhost.com. They have One click installs for everything you requested.
I can second Gregg's opinion about DreamHost. I've been running a couple of domains with them for a bit over a year and I'm quite happy with their service.
The only downside I can think of for your case is that DreamHost doesn't support Java so you're down to implementing your web applications in PHP or Ruby on Rails (or good ol' CGI scripts...).
Anyway, if you decide to sign up, do consider using some promotion code. I'm running the community website for Agile Finland with DreamHost and there's a promotion code listed in the footer, which gives you a $50 discount and contributes slightly less to the hosting bills of agilefinland.com.
I also have different websites and I am using IX web hosting. Previously I used a free web hosting provider and I was not satisfied with them. So I decided to go for a paid web host and finally I have chosen IX web hosting. All I can say is that I definitely do not regret that I have chosen them. This is the best web hosting provider for me personally.