Files will be generated in a folder during run time and using Threads, I need to insert each file in the DB .
Using threads I have to do this.Without missing any files, I have to insert all the files into DB.
Can anyone kindly suggest me how I can do this using Threads.
In your case, a producer-consumer design pattern might be used (google it up if you haven't encountered that yet). You'd have one producer thread, which would be responsible for locating new files and putting them into queue to be picked up by consumers. Since you have only once consumer, you can keep records of which files have been processed without need for any synchronization. Consumers would pick the file name from the queue and process them. You could use a synchronized queue (such as ArrayBlockingQueue, but there are others) to do all the hard synchronization work.
I can see two potential problems:
1) I assume that the files are being generated by an external process. You need to make sure you don't start processing a file before the external process is done with it. It might take some time to write the file; if you start processing it too early, you might get exceptions (if the file is locked, for example), or - worse - you might end up uploading incomplete file into the database.
2) Is your infrastructure capable of uploading several files at once? Database and network could present a bottleneck to your solution, by uploading more than one file at once you might actually decrease your overall performance. In the producer-consumer architecture this can actually be easily adjusted by creating a limited number of consumers (eg. only one).
Joined: Apr 24, 2012
Thank you very much for your suggestion .I am going through the producer-consumer design pattern .
After reading the file that is generated by the external process , I want to write to the DB and later want to delete the file from the folder .
At any point of time, the program should read only one file and only after writing to the DB, the program should read the second file.
If you possible can you please give me a small example of this as it would help me greatly.