I have a back end program which creates multiple threads that talks to mysql database. These threads finish their jobs in a few seconds and get closed, in the mean time new threads are created. In the end, The program uses a lot of sockets to build the connections to database. I use netstat -a to check the status, the screen shows that my program uses 100-200 sockets (most of them are in on_wait status, meaning they will wait for a while then release), however this number is no where near the default windows socket number limit of 5000. The program runs normally at the beginning. After 2/3 days however, it shows it cannot establish connection with database anymore. In the meantime my tomcat server and mysql server could not respond to anything, even service restart. When i try to stop/start Tomcat log shows
java.net.SocketException: No buffer space available (maximum connections reached?): JVM_Bind at java.net.PlainSocketImpl.socketBind(Native Method)
Mysql log shows the similar thing
080721 10:11:13 InnoDB: Started; log sequence number 0 43655 080721 10:11:13 [ERROR] Do you already have another mysqld server running on port: 3306 ? 080721 10:11:13 [ERROR] Aborting
When I stop/start these 2 servers I use netstat to check the socket connections again.
Hi! I have the same problem on Windows XP with JBoss server. From time to time it cannot establish RMI connection with the same exception. I guess it is because of low memory. Have you solved your problem? Thank you!
Using OS X, I found an issue in Java relating to SSL sockets. I don't know if your underlying connections are doing SSL or not, but the file handles for the socket are not being released always. As a result, the max open file limit is reached, which blocks sockets from being created too. (A socket is still a file handle to the OS.)
This is a bug in the JVM...and it seems that the issue exists at least on Linux and OS X. I found others with the same issue.
Possibly another process monitoring tool will give you more info about the running JVM process to see what open file handles it has?