This week's book giveaways are in the Java EE and JavaScript forums.
We're giving away four copies each of The Java EE 7 Tutorial Volume 1 or Volume 2(winners choice) and jQuery UI in Action and have the authors on-line!
See this thread and this one for details.
The moose likes I/O and Streams and the fly likes Batch processing Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of The Java EE 7 Tutorial Volume 1 or Volume 2 this week in the Java EE forum
or jQuery UI in Action in the JavaScript forum!
JavaRanch » Java Forums » Java » I/O and Streams
Bookmark "Batch processing" Watch "Batch processing" New topic
Author

Batch processing

Chris Montgomery
Ranch Hand

Joined: Jan 14, 2004
Posts: 141
I need to a load a file which consists of transactional data into a database every night. This file may contain 100s of 1000s of records.

Is there an existing framework which has been written to do just this? If not, is there an "accepted" approach to batch process �large� files in java?

I�ve never written a batch job before. My initial thought is to write some sort of multi-threaded job where each thread takes a section of the file and goes back for more until the entire file is done. It seems simple enough, but if a framework/approach is already in place, which plans for things I haven�t thought of yet, I�d rather go that route 

Thanks!
Joe Ess
Bartender

Joined: Oct 29, 2001
Posts: 8876
    
    8

Batch processing by definition is non-interactive, meaing there's no user waiting for something to happen. Since there's nobody watching, there's no need to get fancy with the processing. Read a record out of the file, set the values in the DB, do it until the file's done. If you want to get fancy, have the process send an email upon completion or error.


"blabbing like a narcissistic fool with a superiority complex" ~ N.A.
[How To Ask Questions On JavaRanch]
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Batch processing