File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
http://aspose.com/file-tools
The moose likes Spring and the fly likes Processing millions of facts and rules with Drools and Spring Batch Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Frameworks » Spring
Bookmark "Processing millions of facts and rules with Drools and Spring Batch" Watch "Processing millions of facts and rules with Drools and Spring Batch" New topic
Author

Processing millions of facts and rules with Drools and Spring Batch

Gautam Karnataki
Greenhorn

Joined: Nov 07, 2011
Posts: 1
Has anyone done any research on combining Drools and Spring Batch? My requirement is to process more than 1.5 million records periodicallys with about 75-100 rules. I cannot do it in a single JVM. I can break the number of facts in chunks of 100000 each but then i have rules which need to iterate through all the facts.
Would spring batch somehow help me in solving this problem?
Saifuddin Merchant
Ranch Hand

Joined: Feb 08, 2009
Posts: 605

Spring batch has some good support to work in parallel (multi-thread/multi-process). You might want to have a look at remote chunking

I (think) your particular problem could also benefit from the partitioning technique

Have a look the documentation isn't too much to overwhelm, but covers everything that's needed.

You might also want to look at the samples provided.


Cheers - Sam.
Twisters - The new age Java Quiz || My Blog
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Processing millions of facts and rules with Drools and Spring Batch