Currently I have a pure java application using core java which interact with backend database using an OR mapping layer(hibernate). This is synchronous invocation. Now I want to run this application in a batch. i.e. the input for the application will be in a file so instead of 1 record processing invoked by an external client I will have a file which may consist 1000 records. I have to pick 1 record each from the file and invoke the application. I want to know your ideas on the same and best practices if any related to this. Like some of the issues to tackle is not allowing more than 1 batch to run at the same time. We may have to change the file based approach mentioned above to say do a database call and fetch records to be feeded into the java app. Can we use some of the batch utilities in the java space like Springbatch or quartz.
------------- You could write a wrapper than calls your app for each line of the file. This is likely to be significantly slower than doing batching within the program. -------------- Can you explain what do you mean by "than doing batching within the program".
Originally posted by manish ahuja: Can you explain what do you mean by "than doing batching within the program".
Sure. If your Java program read in the file of input values, it could read in X rows at a time (let's say 100) and do a batch update to the database. This cuts down on the number of database round trips.
Joined: Oct 23, 2003
My original intent of the question was can I use something like sub-batches within the parent batch job so as to expedite the whole flow run. Say If get an input of 100 records (file read or database read), I will typically do a for or while loop iterate every input record and run the application for each such input record. This would really take long as the volume of input increases. Each input record is independent of the other. With this regard I was wondering if there is anything where I could start multiple such jobs in parallel and accelerate the batch run time. I am not sure multi-threading can greatly help in this situation.