Thanks for all your and your suggestions.
To answer some of your questions,
Will you be able to implement your application faster in PHP than in Java? What's the cost of implementing a solution in PHP vs Java?
While Java is my primary language, I was dabbling in PHP for this project, because some of web services I was using seemed to have PHP clients that were easier to integrate. So, I started taking a shot at a php solution. I was able to start the development really fast because the PHP client functions were so easy to drop into the code. But, I noticed my memory usage was high per connection with PHP (as I indicated in my original post). So, I took a little time to re-look at it, and was able to get everything working fine in Java with a bit more effort. Thrashing it with Jmeter, the Java solution is faster, and I can handle a larger number of connections on smaller server (using tomcat standalone). Once I get through a bit of the hurdles of setting up things up, I think the cost development cost with Java will not be too much higher than the PHP. So, I am going forward with the Java solution. (I do not want to make this a flame war on languages, I am just sharing my experience and how I decided to deal with it).
Although modern webservers generally boost performance using a "keep-alive" mechanism, the core HTTP protocol determines the number of connections by the number of active requests being processed and not by the number of users logged in.
I was using the term "users" rather loosely . Yes, http is a request/response protocol. However, I do not think it is terribly incorrect to think of webserver load in terms of "users", depending on how we define the term. For example, we can think of users as those who are actively clicking around, (as opposed to those who are logged in). With Keep alives on (within say apache), an entire process is held up until the keep alive is done. And often those processes are heavy with mod_php, so each keep alive process will be consuming quite a bit of memory. So, with apache keep alives and 100 maxclients... if a system has 100 users clicking around every 15 seconds, the system is pretty much maxed out, regardless of how fast an individual request/response time is. Of course we can make the keep alives shorter to make the situation better (which I think is a pretty good idea), but the defaults for apache and tomcat are 15 and 20 seconds... This is more along the lines of what I was thinking about when I spoke of "users."