Win a copy of Java Challengers this week in the Java in General forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Paul Clapham
  • Ron McLeod
  • paul wheaton
  • Devaka Cooray
Sheriffs:
  • Jeanne Boyarsky
  • Tim Cooke
  • Liutauras Vilda
Saloon Keepers:
  • Tim Moores
  • Tim Holloway
  • Stephan van Hulst
  • Carey Brown
  • Piet Souris
Bartenders:
  • salvin francis
  • Mikalai Zaikin
  • Himai Minh

Spring Batch to process 1500000 records in cache memory for each event

 
Ranch Hand
Posts: 814
Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi All,

I am using Spring Batch for following requirement

I have requirement for our Project Asset Liability Mamagement (ALM)

1. One user initiated event will have 1500000 records to process.
2. For every seconds there will be around 1000 events for processing data.
3. We will get input data for processing each record from application.
4. We will read records from cache memory and then will process records and again will write into cache memory. This process is for single record, in same way we have to process 1500000 records.
3. I am using Spring Batch to do above job.

Can anybody give suggestion or comments to achive above job processing in less time with optimum performance.

 
Ninad Kulkarni
Ranch Hand
Posts: 814
Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi All,

Please give comments and views on above.
 
ranger
Posts: 17346
11
Mac IntelliJ IDE Spring
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Not really much to go on for anyone to answer. But I have some suggestions.

Or better yet, the docs of Spring Batch has those suggestions

http://static.springsource.org/spring-batch/reference/html/scalability.html

Remote Chunking and Partitioning is the basic answer.

Mark
 
Ninad Kulkarni
Ranch Hand
Posts: 814
Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks Mark for your views and suggestion we have used same
 
Don't get me started about those stupid light bulbs.
reply
    Bookmark Topic Watch Topic
  • New Topic