• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Ron McLeod
  • Paul Clapham
  • Bear Bibeault
  • Junilu Lacar
Sheriffs:
  • Jeanne Boyarsky
  • Tim Cooke
  • Henry Wong
Saloon Keepers:
  • Tim Moores
  • Stephan van Hulst
  • Tim Holloway
  • salvin francis
  • Frits Walraven
Bartenders:
  • Scott Selikoff
  • Piet Souris
  • Carey Brown

Processing millions of facts and rules with Drools and Spring Batch

 
Greenhorn
Posts: 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Has anyone done any research on combining Drools and Spring Batch? My requirement is to process more than 1.5 million records periodicallys with about 75-100 rules. I cannot do it in a single JVM. I can break the number of facts in chunks of 100000 each but then i have rules which need to iterate through all the facts.
Would spring batch somehow help me in solving this problem?
 
Ranch Hand
Posts: 607
Firefox Browser Spring Java
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Spring batch has some good support to work in parallel (multi-thread/multi-process). You might want to have a look at remote chunking

I (think) your particular problem could also benefit from the partitioning technique

Have a look the documentation isn't too much to overwhelm, but covers everything that's needed.

You might also want to look at the samples provided.
 
    Bookmark Topic Watch Topic
  • New Topic