I have a back end program which creates multiple threads that talks to mysql database. These threads finish their jobs in a few seconds and get closed, in the mean time new threads are created. In the end, The program uses a lot of sockets to build the connections to database. I use netstat -a to check the status, the screen shows that my program uses 100-200 sockets (most of them are in on_wait status, meaning they will wait for a while then release), however this number is no where near the default windows socket number limit of 5000.
The program runs normally at the beginning. After 2/3 days however, it shows it cannot establish connection with database anymore. In the meantime my tomcat server and mysql server could not respond to anything, even service restart. When i try to stop/start Tomcat log shows
java.net.SocketException: No buffer space available (maximum connections reached?): JVM_Bind
at java.net.PlainSocketImpl.socketBind(Native Method)
Mysql log shows the similar thing
080721 10:11:13 InnoDB: Started; log sequence number 0 43655
080721 10:11:13 [ERROR] Do you already have another mysqld server running on port: 3306 ?
080721 10:11:13 [ERROR] Aborting
When I stop/start these 2 servers I use netstat to check the socket connections again.
Proto Local Address Foreign Address State
TCP asus-xp:smtp asus-xp:0 LISTENING
TCP asus-xp
op3 asus-xp:0 LISTENING
TCP asus-xp:epmap asus-xp:0 LISTENING
TCP asus-xp:microsoft-ds asus-xp:0 LISTENING
TCP asus-xp:3389 asus-xp:0 LISTENING
TCP asus-xp:42510 asus-xp:0 LISTENING
TCP asus-xp:1028 asus-xp:0 LISTENING
TCP asus-xp:netbios-ssn asus-xp:0 LISTENING
TCP asus-xp:3389 203.116.53.114:4438 ESTABLISHED
UDP asus-xp:microsoft-ds *:*
UDP asus-xp:isakmp *:*
UDP asus-xp:4500 *:*
UDP asus-xp:4593 *:*
UDP asus-xp:ntp *:*
UDP asus-xp:1900 *:*
UDP asus-xp:ntp *:*
UDP asus-xp:netbios-ns *:*
UDP asus-xp:netbios-dgm *:*
UDP asus-xp:1900 *:*
UDP asus-xp:42508 *:*
So obviously nothing is using those ports. Does anyone have any idea what is happening?