This week's giveaway is in the Android forum.
We're giving away four copies of Android Security Essentials Live Lessons and have Godfrey Nolan on-line!
See this thread for details.
The moose likes Performance and the fly likes Enhancing the performace of a batch job Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Android Security Essentials Live Lessons this week in the Android forum!
JavaRanch » Java Forums » Java » Performance
Bookmark "Enhancing the performace of a batch job" Watch "Enhancing the performace of a batch job" New topic
Author

Enhancing the performace of a batch job

Shruti Sharma
Ranch Hand

Joined: Mar 25, 2010
Posts: 55
Hi,

I have a question on enhancing the performance of the batch job. My batch job is supposed to process 500,000 records. Currently, it crashes after 70,000 records. Here are the steps we are performing:

1. Read the records from the file and dump into the Staging (temporary) table (this step takes just about few seconds since we are using DB2 loader).

2. Read the data from the staging table and populate the main application tables. This is where the batch code is giving performance issues. Any tips to enhance the performance?

Martin Vajsar
Sheriff

Joined: Aug 22, 2010
Posts: 3606
    
  60

I wouldn't classify "it crashes" as a performance problem. What is the error message you get? I believe you need to start from there.

Without much more details of the processing, the best you can get is some very vague general advice. I'm not sure this would be true specifically in DB2, but generally I'd say you'll get the best performance by using SQL statements to manipulate the data in bulk. Does DB2 have some documentation regarding performance?
 
It is sorta covered in the JavaRanch Style Guide.
 
subject: Enhancing the performace of a batch job
 
Similar Threads
can i use hibernate for bulk data migration
URLyBird Locking
Performance problem in jdbc
reading and writing large data(200000) from oracle
oracle stored proc for tree