I am running Weblogic 10.3.2 with a J2EE EAR deployed against it. I am using JDBC data sources to connect to an Oracle 11g database. I am running on Linux Red Hat 5. The machine has 16 GB RAM and 16 cores of an Intel Xeon processor.
I have recorded some Grinder scripts to simulate the HTTP requests going back and forth from the web browser to the app server in an attempt to play them back in volume to do some load testing. I am finding that even under relatively moderate load, after some time (say 30-60 minutes) when trying to access the application in my web browser (and in Grinder), I am getting 404 errors.
The strange thing is that the app server is still running fine and there are no errors thrown in any Weblogic log file. There is nothing wrong in the verbosegc log file and the CPU usage via "sar" is very low. The number of JDBC connections is well below the threshold for concern. Sometimes, if I wait a while (hours or more), it sometimes comes back up and works fine again, until the next worjload hits it. I have studied thread dumps and see nothing unusual.
I've been very puzzled with this for days. Does anyone else know what might be wrong or something else I could check to get more information?