• Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

Spring Batch to process 1500000 records in cache memory for each event

 
Ninad Kulkarni
Ranch Hand
Posts: 802
Hibernate Linux Spring
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi All,

I am using Spring Batch for following requirement

I have requirement for our Project Asset Liability Mamagement (ALM)

1. One user initiated event will have 1500000 records to process.
2. For every seconds there will be around 1000 events for processing data.
3. We will get input data for processing each record from application.
4. We will read records from cache memory and then will process records and again will write into cache memory. This process is for single record, in same way we have to process 1500000 records.
3. I am using Spring Batch to do above job.

Can anybody give suggestion or comments to achive above job processing in less time with optimum performance.

 
Ninad Kulkarni
Ranch Hand
Posts: 802
Hibernate Linux Spring
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi All,

Please give comments and views on above.
 
Mark Spritzler
ranger
Sheriff
Posts: 17278
6
IntelliJ IDE Mac Spring
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Not really much to go on for anyone to answer. But I have some suggestions.

Or better yet, the docs of Spring Batch has those suggestions

http://static.springsource.org/spring-batch/reference/html/scalability.html

Remote Chunking and Partitioning is the basic answer.

Mark
 
Ninad Kulkarni
Ranch Hand
Posts: 802
Hibernate Linux Spring
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks Mark for your views and suggestion we have used same
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic