aspose file tools*
The moose likes Spring and the fly likes Spring Batch to process 1500000 records in cache memory for each event Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Spring in Action this week in the Spring forum!
JavaRanch » Java Forums » Frameworks » Spring
Bookmark "Spring Batch to process 1500000 records in cache memory for each event" Watch "Spring Batch to process 1500000 records in cache memory for each event" New topic
Author

Spring Batch to process 1500000 records in cache memory for each event

Ninad Kulkarni
Ranch Hand

Joined: Aug 31, 2007
Posts: 797

Hi All,

I am using Spring Batch for following requirement

I have requirement for our Project Asset Liability Mamagement (ALM)

1. One user initiated event will have 1500000 records to process.
2. For every seconds there will be around 1000 events for processing data.
3. We will get input data for processing each record from application.
4. We will read records from cache memory and then will process records and again will write into cache memory. This process is for single record, in same way we have to process 1500000 records.
3. I am using Spring Batch to do above job.

Can anybody give suggestion or comments to achive above job processing in less time with optimum performance.


SCJP 5.0 - JavaRanch FAQ - Java Beginners FAQ - SCJP FAQ - SCJP Mock Tests - Tutorial - JavaSE7 - JavaEE6 -Generics FAQ - JLS - JVM Spec - Java FAQs - Smart Questions
Ninad Kulkarni
Ranch Hand

Joined: Aug 31, 2007
Posts: 797

Hi All,

Please give comments and views on above.
Mark Spritzler
ranger
Sheriff

Joined: Feb 05, 2001
Posts: 17257
    
    6

Not really much to go on for anyone to answer. But I have some suggestions.

Or better yet, the docs of Spring Batch has those suggestions

http://static.springsource.org/spring-batch/reference/html/scalability.html

Remote Chunking and Partitioning is the basic answer.

Mark


Perfect World Programming, LLC - Two Laptop Bag - Tube Organizer
How to Ask Questions the Smart Way FAQ
Ninad Kulkarni
Ranch Hand

Joined: Aug 31, 2007
Posts: 797

Thanks Mark for your views and suggestion we have used same
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Spring Batch to process 1500000 records in cache memory for each event