This week's book giveaway is in the Mac OS forum.
We're giving away four copies of a choice of "Take Control of Upgrading to Yosemite" or "Take Control of Automating Your Mac" and have Joe Kissell on-line!
See this thread for details.
The moose likes Java in General and the fly likes Rule based systems Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


JavaRanch » Java Forums » Java » Java in General
Bookmark "Rule based systems" Watch "Rule based systems" New topic
Author

Rule based systems

Ernest Friedman-Hill
author and iconoclast
Marshal

Joined: Jul 08, 2003
Posts: 24187
    
  34

Hi Folks,
I'm the developer of Jess, a Java rule engine. Michael was so kind as to ask if I'd start a thread here on rule-based systems (my particular interest.)
In Java (and in many other programming languages) the focus is on telling a computer precisely how to do something; the steps to follow, in what order. In a rule-based language like Jess, you tell the computer what you want done, and the run-time system (the rule engine) figures out how. This makes rule-based systems ideal for solving non-algorithmic problems like classification, control, workflow management, and for making subjective decisions.
Programming a rule-based system generally means writing rules in a special rule language. Jess's rule language is a sort of odd hybrid of LISP and Java; it's a very powerful and succinct way to describe conditions and logical relationships. Each rule is sort of like an if-then statement, although it's up to the rule engine to decide when to apply each rule. The Jess rule language gives you full access to all Java APIs from your rules; both the "if" and "then" parts of a rule can take arbitrary action on Java objects.
Rule-based systems are actually intimately related to relational databases; they share a lot of technology and concepts. Many rules engines are tuned for working with very large data sets, and because of the way they store and index data are vastly more efficient than hand-coded Java code would be at sifting through large volumes of information.
All that said: I'll open the floor to questions, and try to answer any that come up.
[ July 14, 2003: Message edited by: Ernest Friedman-Hill ]

[Jess in Action][AskingGoodQuestions]
Bear Bibeault
Author and ninkuma
Marshal

Joined: Jan 10, 2002
Posts: 61315
    
  66

Hi Ernest, why not start off with a short explanation of what exactly a rule-based system is, and why it's useful?
thanks,
bear


[Asking smart questions] [Bear's FrontMan] [About Bear] [Books by Bear]
Thomas Paul
mister krabs
Ranch Hand

Joined: May 05, 2000
Posts: 13974
I can tell you what we did with a rules engine. It was written in C#. I didn't write it but I can tell you a little bot about it.
The problem was that we were getting files from customers that contained mailing lists for customers. We would build a database out of the data and then using business rules we would extract the data (select everyone who didn't receive a mailing in the last 2 months who live in these zip codes and have requested information on ulcers). Prior to the rules engine we would write a lot of custom code to extract the info we needed for each run. Since the customer was contstantly changing their needs this was painful. Also different customers had different fields in their input files that we might have to select on.
The rules engine allowed us to eliminate programming to specify business rules and just build rules that we used to run against the files. The rules engine read the rules and translated them into instructions. So you could specify columns to select based on any criteria you wanted even criteria from other columns. You could then run an action gainst that selected row. For example, an action could be to update a row, delete a row, mark a row as selected, write selected data to another table or file, or anything else we could think of that we might need.
The rules themselves looked sort of like a cross between C# and SQL. We wrote a GUI that could actually be used by a user (a well trained user) to generate, test, and save rules.
The whole idea of the rules engine was that it was completely generic. It knew nothing about tables, columns, user data or anything else except what you told it with the rules.
Rules could build on each other. You could create a rule and save it in the database. Then you could group a bunch of rules together into a rule set. Then you could run a rule set against a set of user tables. A rule could be a member of multiple rule sets.
Once it was completed, we could process changes in minutes instead of the hours or days required to code and test changes and port the changes to production.


Associate Instructor - Hofstra University
Amazon Top 750 reviewer - Blog - Unresolved References - Book Review Blog
Jessica Sant
Sheriff

Joined: Oct 17, 2001
Posts: 4313

So... I don't have a rules question yet (although I did take a class a few years ago about the rules language "CLIPS"), but I just wanted to show off my new signature to Ernest


- Jess
Blog:KnitClimbJava | Twitter: jsant | Ravelry: wingedsheep
Michael Morris
Ranch Hand

Joined: Jan 30, 2002
Posts: 3451
Thank you Dr. for starting this thread. In the Programming Diversions forum the problem Einstein's riddle is a fairly complex logic problem of the type seen in puzzle magazines. I have been coding a generic class to solve such problems but, as you can imagine, it gets fairly complex with large sets of data and when the total number of data types won't fit into a square truth table as in that example. Would a rule based system be a good candidate to solve these sorts of problems? Also there is a Domino game played extensively in the Southern US called 42 which is essentially a Domino version of contract bridge. I would like to create rules for non-player (computer controlled) bidding and control the bid quality. Could that be done with a rule based system?


Any intelligent fool can make things bigger, more complex, and more violent. It takes a touch of genius - and a lot of courage - to move in the opposite direction. - Ernst F. Schumacher
Ernest Friedman-Hill
author and iconoclast
Marshal

Joined: Jul 08, 2003
Posts: 24187
    
  34

Yes, Jess is actually good at solving this kind of problem. One of the example programs that comes with Jess is a solution to such a problem, and (believe it or not) chapter 1 of my new book shows exactly how to solve them, step by step.
To paraphrase here, the technique is to
  • Represent all the possible sub-solutions as data in the system
  • Encode the whole problem as a single rule

  • Jess's pattern-matcher then quickly finds the particular combination that matches all the constraints while trying to satisfy this rule.
    Step 2 in the list above is really interesting, actually; what it says is that you can directly translate the problem statement into code. It's a very nice way to program!
    Ernest Friedman-Hill
    author and iconoclast
    Marshal

    Joined: Jul 08, 2003
    Posts: 24187
        
      34

    Jessica Sant:
    Thanks for the tribute
    A Google search for "Jess in Action" primarily turns up two things: references to this book, and also discussions by Jessica Alba fans cheering some recent reappearance in the media (don't recall the details.)
    Michael Morris
    Ranch Hand

    Joined: Jan 30, 2002
    Posts: 3451
    Another problem programmer's seem to run into quite often, similar to Thomas' example above, is the processing of input data with wildly varying styles and formatting I would guess that this scenario could be made easier using rules instead of hard coding all the possibilities of which you will always miss some.
    Thomas Paul
    mister krabs
    Ranch Hand

    Joined: May 05, 2000
    Posts: 13974
    Originally posted by Michael Morris:
    Another problem programmer's seem to run into quite often, similar to Thomas' example above, is the processing of input data with wildly varying styles and formatting I would guess that this scenario could be made easier using rules instead of hard coding all the possibilities of which you will always miss some.

    Funny you should mention that!
    We built a rules based ETL (extract-transform-load) engine to suck in user input files and apply them to a standard database format. Custom fields that weren't in our standard format were applied to a table which worked basically like a Map.
    By the way Ernest, we stole a lot of your ideas when we discovered Jess on the Sun website last year. I don't think our result was as flexible as Jess but it fit in nicely with our business requirements.
    Maulin Vasavada
    Ranch Hand

    Joined: Nov 04, 2001
    Posts: 1871
    hi all,
    i've been in touch with some rule engines and i had to do some research on my part a while back...
    i guess this thread will help me understand "why ruleengine" question in my mind which i have already fought with lot of time and try to get an answer...
    btw, i'm sure u've seen Yahoo Email's filters which are based on "rules". tho i hate the fact that they only allow "only if all the following are true" scenario and doesn't have "atleast one of the following are true/false" things...may b i can offer them something and earn myself money u will
    i am amazed by the "Rete algo" used by all rule engines...tho its complex enough for me to understand w/o spending a month on it
    regards
    maulin
    Ernest Friedman-Hill
    author and iconoclast
    Marshal

    Joined: Jul 08, 2003
    Posts: 24187
        
      34

    I think one of the things about rule engines that makes it hard for people to understand the "why" is that there's such a wide range of capabilities, all lumped into this one category of tool. One the one hand, you have things like (as you mention) email filters, where a "rule" is a regular expression and a choice from a short list of possible actions to take. There's really no "rule engine" behind that at all -- just a loop that applies each rule to each message as it comes in, in order. Oh the other hand, you have complex software in the "expert systems" category (Jess can be used to write expert systems.) To some extent it's like lumping Notepad and Quark together and calling them "publishing tools."
    vasu maj
    Ranch Hand

    Joined: Jul 12, 2001
    Posts: 395
    Hi,
    I got an email from manning publishers about the book on jess and was trying to search on google to see what it is and ended up with the results as mentioned above by Dr.Friedman. I came here to post a question on it and am really thrilled to see a discussion already in progress.To be frank, even after reading these posts I am not too clear about the concept.
    Can someone with examples briefly explain why using the concept makes an application better and also scenarios where it is the best thing to implement....

    Thanks,
    Vasu
    [ July 16, 2003: Message edited by: vasu maj ]

    What a wonderful world!
    Ernest Friedman-Hill
    author and iconoclast
    Marshal

    Joined: Jul 08, 2003
    Posts: 24187
        
      34

    Well, there all sorts of arguments about separating out business logic, about the power of declarative programming, etc. But let me give you a practical programmer's example:
    Imagine you are writing a program which receives customer orders, formats them, sends them to a warehouse to be filled. Each order is a list of items with quantities, a customer, and a date. You write the program, everything is great.
    Boss comes in, says, "Give a 10% discount to everybody who spends more than $100." Great. You add a single if-then statement, recompile, boom. You're back in business.
    Boss comes in, says, "Give a 25% discount on items the customer hasn't bought before." OK, well, you've got to add some database queries, but soon enough, you're back in business.
    Boss comes in, says "Revoke 10% discount per-order, make it 10% if you spend $250 a month or more. Also, if somebody buys a CD writer, send them a free sample of CD-RW disks; but only if they're a repeat customer. Oh, and by the way, we need to price shipping based on geographic zone, except for people with multiple locations..."
    After a few weeks of this, if you've been using traditional programming techniques, your code is a gnarled mess -- and it's slow too, as it has to do all sorts of database queries for each order that comes in.
    If you're using a rule engine, though, you've got nice clean code, with one rule for each pricing policy. If somebody need to know where the "Wacky Wednesday" pricing policy is implemented, it's easy to find it.
    If you're using a Jess-like system, it's not slow, either; the rule engine itself indexes all the orders and no lookups are required; the rule engine just "knows" what it needs to know about past orders.
    OK?
    Michael Morris
    Ranch Hand

    Joined: Jan 30, 2002
    Posts: 3451
    Wow. That scenario sounds like an application I have here that I havedeveloped over the past 15 years. I am curretly porting to Java from a variety of languages. It sounds like Jess could make my life easier. Customers are constantly changing connection types and styles and I now have a patchwork of slow hacked-up code trying to keep everyone satisfied.
    vasu maj
    Ranch Hand

    Joined: Jul 12, 2001
    Posts: 395
    That was a real good explanation. Thanks. So we are talking about a set of rules that we store external to the code base and apply them as required.
    Is it in any way like aspect oriented prgramming( Having point cuts and applying business rules at those places? We wanted to change the way we aproach exception handling in our application and using AOP helped us as we didn't need to go to all those places individually but used the point cuts to implement the new procedure. Also it helped us to log all System.out.println statements as errors so as to warn the owners of the code...
    Are rules any similar...
    Thanks,
    Vasu
    muthu muruges
    Ranch Hand

    Joined: Feb 28, 2003
    Posts: 70
    Whether the Indian edition of the "Jess In Action" is available now? 'cos the US edition is too costly for me. It cost around 1/4 of my monthly salary.
    Ernest Friedman-Hill
    author and iconoclast
    Marshal

    Joined: Jul 08, 2003
    Posts: 24187
        
      34

    vasu maj -
    There's actually a whole chapter in Ramnivas Laddad's upcoming new book on AspectJ ("AspectJ in Action," from Manning Publications) on writing aspects which incorporate a rule engine. His examples use Jess, as a matter of fact. The aspects inject the operation of the rule engine into various points in the application. Very interesting synergy!
    Ernest Friedman-Hill
    author and iconoclast
    Marshal

    Joined: Jul 08, 2003
    Posts: 24187
        
      34

    m muruges -
    I asked my publisher about this, and he said that you can expect it soon (realize that the book isn't widely available in the U.S. yet, either; the first printing is just now being shipped out.) Manning has a standard deal with an Indian publisher to produce Indian editions of all their books.
    [ July 17, 2003: Message edited by: Ernest Friedman-Hill ]
    Thomas Paul
    mister krabs
    Ranch Hand

    Joined: May 05, 2000
    Posts: 13974
    In our rules engine we stored the rules in a database and told the rules engine, "grab these rules and apply them to this data".
    Joel McNary
    Bartender

    Joined: Aug 20, 2001
    Posts: 1817

    And now, in this anecdote:
    Horror Story With a Happy Ending
    ---------------- or -----------------------
    how not to write a rules engine
    I, tool have used a database-driven rules engine to accomplish taks like those outlines by Ernest; one of our clients was a small-electronics insurer (cell phones, PDAs, pagers, etc.) with several major national clients. Their main client would have policy changes on a bi-weekly basis.
    We had a developer write a rule engine that was a mess. It consisted of returning a 1 or a 0 and then executign or not executing an action based on that return. However, there was only an if - then - else structure for returning 1 or 0--and there was no support for AND or OR, so we had to chain conditions in the then or else portions. And the parentheses were worse than LISP...
    If(<condition>, <then-return>, <else-return> )
    If(State="PA", 1, If(State="WV", 1, If(State="OH", If(ZIP="44827", 1, 0), 0)))
    Would be how to write: "If the State is PA, WV, or ZIP 44827 in OH..." The If had to be capitalized and there could be no space between the If and the open paren; that is, "If(" was treated as a single token.
    Anyway, the point of this rules-engine was that it was supposed to be simple enough that the managers at the client site could go in and change the rules based on their client's new demands - taking the onus off of our developers. The problem was that with a syntax that convoluded, the managers couldn't figure it out. And then the developer left, leaving no documentation as to the syntax of this engine. I was left to figure it out (which I did), and then write my own (which I did -- a much more complete and flexible language).
    This was all done in Objective-C, though, and since I've moved on to Java I've not had the change to look at rules engines in Java. (Actually, we looked at one about three years ago and decided against it; I have thought about moving mine over to Java but I've never had the time/inclination/need to do it yet.)
    Moral of the story: I'm going to check out JESS and see if it suits my needs!
    [ July 17, 2003: Message edited by: Joel McNary ]

    Piscis Babelis est parvus, flavus, et hiridicus, et est probabiliter insolitissima raritas in toto mundo.
    Kathy Sierra
    Cowgirl and Author
    Ranch Hand

    Joined: Oct 10, 2002
    Posts: 1572
    Oh My God. This makes me very happy... Bert and I both came from AI backgrounds (long time ago in the days of experTelligence and the TI explorer, etc.) writing expert systems and doing knowledge engineering. We were so sad to see expert systems go underground when it became unfashionable to talk about AI in public. (Is it still?)
    Bert spent most of his adult programming life building rule-based systems, most of them incomprehensively large and complex.
    I spent about 10 years in expert systems, but mine were *much* wimpier than what he did, but more fun
    OK, all this talk here makes me wanna crank up some rules again just for fun!
    Yay for Jess! (and you too, Jessica)


    Co-Author of <a href="http://www.amazon.com/exec/obidos/ASIN/0596007124/ref=jranch-20" target="_blank" rel="nofollow">"Head First Design Patterns"</a><br /> <br />Just a Jini girl living in a J2EE world.
    Ernest Friedman-Hill
    author and iconoclast
    Marshal

    Joined: Jul 08, 2003
    Posts: 24187
        
      34

    Neat, Kathy, I didn't know about your background! You're certainly invited to give Jess a spin.
    It is indeed unfashionable to talk explicitly about AI these days,but that doesn't mean it isn't going on. It is, but people just don't call it that. It's become an old saw now that as soon as something you used to call "AI" actually works, it isn't AI anymore; it's just software. Therefore people always think that AI is a failure. It's not, it just doesn't get credit where credit is due!
    Anyway -- Jess, which is indeed nominally what used to be called an "expert system shell" is now called a rule engine; everybody else with a product in this niche calls it that, too.
    Anupam Sinha
    Ranch Hand

    Joined: Apr 13, 2003
    Posts: 1088
    Hi all
    I don't know much about Jess or rule based engine. I am happy to know about it. But what exaxctly is it. Is it a seprate software which works in conjunction with java or is a set of rules on how to develop a rule beased engine. Secondly if its a software what is it's cost. Is there any specific reason why is it called Jess.
    Ernest Friedman-Hill
    author and iconoclast
    Marshal

    Joined: Jul 08, 2003
    Posts: 24187
        
      34

    Jess is a software library that you can use in Java programs. You can program Jess via its Java API and/or in a special "rule language." There are a few posts above that talk about what rule engines are good for. You can also go to the Jess home page to learn more.
    Jess is a commercial product from Sandia National Laboratories. It's not "free software" but it is available at no cost for academic use. There's also a 30-day trial version. If you buy a licensed version of Jess, you do in fact get the full source code.
    Why is it called Jess? It's named after Jessica Sant. Actually, a long time ago, it was called "JESS" whic was an acronym for "Java Expert System Shell." Sun doesn't like you to use their trademarks in this way, so we eventually got a letter suggesting we change the name. We did. Now it's called "Jess" and it doesn't stand for anything! ("Expert system shell" is a kind of old-fashioned word for "Rule engine.")
    [ July 19, 2003: Message edited by: Ernest Friedman-Hill ]
    Michael Morris
    Ranch Hand

    Joined: Jan 30, 2002
    Posts: 3451
    Why is it called Jess? It's named after Jessica Sant.
    Now she's really gonna have a big head. Geez!
    Sri Rangan
    Ranch Hand

    Joined: Dec 08, 2001
    Posts: 160
    How are the licensing prices for Jess?
    Ernest Friedman-Hill
    author and iconoclast
    Marshal

    Joined: Jul 08, 2003
    Posts: 24187
        
      34

    Sri Langan --
    Commercial licensing fees for Jess are based on intended use (in-house software vs. shrink-wrap, number of users, etc.) You can also elect to get a royalty-based license, or pay a single fee up-front. They're really pretty flexible about it. As a result, it's impossible to quote one number here.
    Craig Smith, casmith@sandia.gov, handles Jess licensing; he'd be glad to provide you with more information based on your unique situation .
    [ July 24, 2003: Message edited by: Ernest Friedman-Hill ]
    Maulin Vasavada
    Ranch Hand

    Joined: Nov 04, 2001
    Posts: 1871
    Hi Ernest
    I am getting confused in differences between workflow engine and rule engine.
    Here is what I am upto,
    1. I have to design workflow engine
    2. I have to make it work with a rule engine
    Now, here is what I understand about both of those terms,
    1. Workflow engine should decide the flow of the process
    2. Rule engine should be used to configure 'rules' that could apply to specify some promotions sort of things like 'give discount of x% to users who are purchasing above 4000 dollars' etc you know
    (I could be wrong in both of them above or fairly inaccurate)
    Now, the reason I am getting confused is- if rule should determine what needs to be done then it becomes the workflow, doesnt it? e.g. if we have a rule that says if 'user is having email addess on yahoo then invoke process X but if user is having email address on hotmail then invoke process Y', then should not this be defined by the workflow definition? If it is specified by the workflow definition then what rule engine does?
    May be I am inaccurate in depicting my befuddled state but I hope you (or anybody else) can guide me little bit...
    Regards
    Maulin
    Ernest Friedman-Hill
    author and iconoclast
    Marshal

    Joined: Jul 08, 2003
    Posts: 24187
        
      34

    Remember that all computer software does the same thing: it makes the transistors in a computer switch on and off. There are similarities among different kinds of software all the way from the lowest level to the highest level of abstraction. For example, there are fairly deep and important similarities betwen databases and rule engines, as well.
    A workflow engine is, more or less, software that sends email to people telling them what to do, based on things that other people have done, and on deadlines, sensor inputs, etc, etc.
    Now, at the core, that workflow engine needs some representation for what things should be done, how they should be done, and when they should be done. Very often, this representation will be hardcoded: someone creates a flowchart, and that is the "workflow" and the workflow engine just reminds people to follow the flowchart.
    Some fancier workflow engines, though, can use more sophisticated, more dynamic workflows, with many decision points. If there are enough of those decision points, and they're complex enough decisions, then it makes sense for there to be a rule engine inside the workflow engine making those decisions. The workflow would then be represented as a set of rules.
    OK?
    Maulin Vasavada
    Ranch Hand

    Joined: Nov 04, 2001
    Posts: 1871
    Hi Ernest
    I see your point. Thanks for the help.
    Regards
    Maulin
    Naveen Kuppili
    Greenhorn

    Joined: Aug 18, 2003
    Posts: 7
    It is good that there is some discussion going on about rules engines here. I have not personally used Jess - so cannot comment on it too much. But, in general, I have found rules engines very helpful especially when the business rules are more dynamic. In fact, we have a rules engine which is XML based and based on Rete algorithm, and had realy good experience it with so far. The feature that I particularly like is that it supports transactions and also the IDE provides debugging utilities.
    Originally posted by Ernest Friedman-Hill:
    Hi Folks,
    I'm the developer of Jess, a Java rule engine. Michael was so kind as to ask if I'd start a thread here on rule-based systems (my particular interest.)
    In Java (and in many other programming languages) the focus is on telling a computer precisely how to do something; the steps to follow, in what order. In a rule-based language like Jess, you tell the computer what you want done, and the run-time system (the rule engine) figures out how. This makes rule-based systems ideal for solving non-algorithmic problems like classification, control, workflow management, and for making subjective decisions.
    Programming a rule-based system generally means writing rules in a special rule language. Jess's rule language is a sort of odd hybrid of LISP and Java; it's a very powerful and succinct way to describe conditions and logical relationships. Each rule is sort of like an if-then statement, although it's up to the rule engine to decide when to apply each rule. The Jess rule language gives you full access to all Java APIs from your rules; both the "if" and "then" parts of a rule can take arbitrary action on Java objects.
    Rule-based systems are actually intimately related to relational databases; they share a lot of technology and concepts. Many rules engines are tuned for working with very large data sets, and because of the way they store and index data are vastly more efficient than hand-coded Java code would be at sifting through large volumes of information.
    All that said: I'll open the floor to questions, and try to answer any that come up.
    [ July 14, 2003: Message edited by: Ernest Friedman-Hill ]


    Naveen<br />Flexetech Solutions, Inc.<br /><a href="http://www.flexetech.com" target="_blank" rel="nofollow">http://www.flexetech.com</a>
     
    GeeCON Prague 2014
     
    subject: Rule based systems