Shruti Sharma wrote:Thank you so much! I have another question on enhancing the batch performance:
My batch job is supposed to process 500,000 records. Currently, it crashes after 70,000 records. Here are the steps we are performing:
1. Read the records from the file and dump into the Staging (temporary) table (this step takes just about few seconds since we are using DB2 loader).
2. Read the data from the staging table and populate the main application tables. This is where the batch code is giving performance issues. Any tips to enhance the performance?
Any inputs would be greatly appreciated.
We are also using the same approach but this seems to be working fine with us.We are pre processing our file before dumping data in staging table.Along with it we are also doing some processing while transferring data from staging to main table using some stored proc.
But I guess better approach could be using Spring Batch.Try to explore it too.
I’ve looked at a lot of different solutions, and in my humble opinion Aspose is the way to go. Here’s the link: http://aspose.com