A java listener process is running on unix server. This java listener process listens to a MQueue. When a message is received by the queue, it is passed on to the java listener process. The java process stores the message to a file in the input directory and calls a db procedure by passing the file name. The db procedure then processes it creates an acknowledgement file in the output directory and passes the ack file name back to java process. Then the java process reads the contents of the file and sends it to another MQueue. If any of the steps fail in between, the process calls the another db procedure with the error message as parameter and the procedure generates a fail acknowledgement file in the output directory and passes the fail ack file name back to java process. Then the java process reads the contents of the file and sends it to another MQueue.
The process also writes each and every step in the logger.
This listener process has been running successfully for more than two months with no issues. But for the past two days, for some messages from MQueue (say 4 out of 20), the listener fails to store the message to an input file (java.io.FileNotFoundException) and hence calls the db procedure to generate fail acknowledge message. Then, the java process is not able to read the fail acknowledgement file (java.io.FileNotFoundException). Above all this, it is not able to write the steps to the log file during this time. There is nothing in the log file for these 4 failed messages.
We checked the unix permissions for all these folders, everything is alright.