This week's book giveaway is in the OCMJEA forum.
We're giving away four copies of OCM Java EE 6 Enterprise Architect Exam Guide and have Paul Allen & Joseph Bambara on-line!
See this thread for details.
The moose likes Spring and the fly likes Processing millions of facts and rules with Drools and Spring Batch Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of OCM Java EE 6 Enterprise Architect Exam Guide this week in the OCMJEA forum!
JavaRanch » Java Forums » Frameworks » Spring
Bookmark "Processing millions of facts and rules with Drools and Spring Batch" Watch "Processing millions of facts and rules with Drools and Spring Batch" New topic
Author

Processing millions of facts and rules with Drools and Spring Batch

Gautam Karnataki
Greenhorn

Joined: Nov 07, 2011
Posts: 1
Has anyone done any research on combining Drools and Spring Batch? My requirement is to process more than 1.5 million records periodicallys with about 75-100 rules. I cannot do it in a single JVM. I can break the number of facts in chunks of 100000 each but then i have rules which need to iterate through all the facts.
Would spring batch somehow help me in solving this problem?
Saifuddin Merchant
Ranch Hand

Joined: Feb 08, 2009
Posts: 605

Spring batch has some good support to work in parallel (multi-thread/multi-process). You might want to have a look at remote chunking

I (think) your particular problem could also benefit from the partitioning technique

Have a look the documentation isn't too much to overwhelm, but covers everything that's needed.

You might also want to look at the samples provided.


Cheers - Sam.
Twisters - The new age Java Quiz || My Blog
 
wood burning stoves
 
subject: Processing millions of facts and rules with Drools and Spring Batch