TreeSet sorting dropping elements

[Mark Herschberg]

I have a really bizzare problem. We are writing a trading application. We're using WebLogic 7.0 working against Oracle 9i, both running on Win2000. (We're writing using JBuilder 8 Enterprise--although this shouldn't matter for this problem.)
At the heart of the system is what's known as a limit order book. This is a listing of limit orders (e.g. "buy 1000 shares at $50," "sell 500 shares at $80"). These are placed into the database table for limit orders. When someone comes in with a market order (e.g. "buy 600 shares") it crosses with the sell orders (sorted by price) until it fills the number of shares.
I tested this by hand, meaning I manually placed orders using a gui I created and everything worked fine. All orders got placed, and I can see them printed out on the screen.
Now we've added computer traders, who rapdily place limit orders. Thanks to screen print outs, I can see the orders being placed. All the info is correct. In fact, I can even see the orders in the data base (again, they are entity beans). However when I do a findBy on the entity bean, it does not returnt he complete listing. That is, I might have 10 orders in the database, but the findBy method only returns 2 orders.
Parts of the code is below. I include 4 methods. The method to add the order, the method to print all the orders, a sorting method the latter relies on, and a method to print a collection.

   /**
    *  This method adds a limit order to the system.
    *
    *  @param order
    */
   private void addLimitOrder(LimitOrder order) {
      System.err.println("addLimitOrder");
      System.err.print("Adding order... " + order.toString());
      order.util_printLimitOrder();
      try {
         liveOrderHome.create( order.getOrderID(),
                               order.getSecurityID(),
                               order.getQuantity(),
                               order.getAction(),
                               order.getOrderType(),
                               order.getMarketName(),
                               order.getTimestamp(),
                               order.getPrice(),
                               order.getTimeout(),
                               order.getGranularity(),
                               order.getUserID(),
                               order.getParentOrderID());
      } catch ( Exception ex ) {
         ex.printStackTrace();
      }
      // test
      System.err.println();
      System.err.println("just added order");
      util_printAllOrders(order.getSecurityID(), order.getMarketName());
      OrderUpdateBroadcast update = new OrderUpdateBroadcast(order.getOrderID(), order.getSecurityID(), order.getQuantity(), order.getPrice(), OrderUpdateBroadcast.NEW_ORDER);
      orderUpdateSender.broadcastOrderUpdate(update);
   }

   /**
    *  This method prints the Limit Order Book to the screen.
    */
   private void util_printAllOrders(long securityID, String marketName) {
      try {
         Collection halfOrders;
         halfOrders = liveOrderHome.findCrossingOrders( securityID, marketName, Order.ACTION_SELL );
         Collection allAskOrdersSorted = new_sortLimitOrders(halfOrders);
         halfOrders = liveOrderHome.findCrossingOrders( securityID, marketName, Order.ACTION_BUY );
         Collection allBidOrdersSorted = new_sortLimitOrders(halfOrders);
         System.err.println();
         System.err.println();
         System.err.println("Limit Order Book: " + securityID + " " + marketName);
         System.err.println("----------------");
         util_PrintCollection(allAskOrdersSorted);
         util_PrintCollection(allBidOrdersSorted);
      }
      catch ( FinderException ex ) {
         ex.printStackTrace();
      }
   }
   /**
    *  This method takes a collection and returns the collection after it
    *  has been sorted.
    *
    *  @param col  the Collection to be sorted
    *  @return     a Collections sorted in ascending "natural" order
    */
   private TreeSet new_sortLimitOrders(Collection col) {
      TreeSet set = new TreeSet(col);
      return set;
   }
   private void util_PrintCollection(Collection col) {
      LiveOrder liveOrder;
      Iterator iter = col.iterator();
      while ( iter.hasNext() ) {
         liveOrder = ( LiveOrder ) iter.next();
         System.err.println( liveOrder.getOrderAsString() );
      }
      System.err.println();
   }

The code in the addLimitOrder seems to work without error. No exceptions are thrown, I see all the printouts, and the order is definately in the database.
The liveOrderHome.findCrossingOrders method is a finder method defined on an entity bean and works as follows:

SELECT Object (o) FROM LiveOrder o WHERE o.securityID = ?1 AND o.marketName = ?2 AND o.action = ?3

Again, this works fine when I place the orders myself, by hand.
I haven't a clue what's wrong. It almost feels like the database read is not working (because the code works and the data goes into the database, I just can't find it when doing a find). Could it be caching old reads or something like that? Are there any Oracle or WebLogic settins which might cause this?
I'm lost. Help is greatly appreciated!

--Mark
[ February 05, 2003: Message edited by: Mark Herschberg ]
[ February 05, 2003: Message edited by: Mark Herschberg ]

[Mark Herschberg]

OK, upon further inspection, my sorting method (new_sortLimitOrders) is killing it. The print statements in the modified method below will print out 20 before hand, and 4 afterwards.

   private TreeSet new_sortLimitOrders(Collection col) {
      System.err.println("SORTING: collection size = " + col.size());
      TreeSet set = new TreeSet(col);
      System.err.println("SORTING: set size = " + set.size());
      return set;
   }

My compareTo method, in the Entity Bean, is

   public int compareTo(Object obj) {
      LiveOrder liveOrder = (LiveOrder) obj;
      int relation;
      System.out.println();
      System.out.println();
      System.out.println("Comparing: A) " + "id=" + getOrderID() + ", $" + getPrice() + ", " + getTimestamp());
      System.out.println("           B) " + "id=" + getOrderID() + ", $" + getPrice() + ", " + getTimestamp());
      // order them by the difference in price
      double difference = getPrice() - liveOrder.getPrice();
      System.out.println("...price = " + difference);
      // if the prices are the same order them by time
      if (difference == 0) {
         difference = getTimestamp() - liveOrder.getTimestamp();
         System.out.println("...time = " + difference);
      } else {
         // when ordered by price (not time), we need to flip
         // the order for buy orders
         if (liveOrder.getAction() == Order.ACTION_BUY) {
            difference = 0 - difference;
            System.out.println("...flip = " + difference);
         }
      }
      return (int) difference;
   }

I checked and there is never a case of two equal timestamps, so it's not an issue of duplicate elements.
Does sorting not work well on EJBs? What defines a duplicate value in a TreeSet, is it the same object, or the two objects which return 0 on comparison?
--Mark
[ February 05, 2003: Message edited by: Mark Herschberg ]

[Michael Morris]

Hi Mark,
Is it possible that you have a truncation problem with your return [return (int) difference;] ? For instance, 62.50 and 62 would be the same here and hence equal by your compareTo() method. Just a possibility. I doubt that it has anything to do with EJB's. I know that int truncation has bitten me many times over the years.
Hope this helps,
Michael Morris
[ February 05, 2003: Message edited by: Michael Morris ]

[Mark Herschberg]

Oh son of a...
Yep, I just replaced my return with...

      if (difference > 0) {
         return 1;
      } if (difference < 0) {
         return -1;
      } else {
         return 0;
      }
      //return (int) difference;

and lo and behold it worked! Michael, you are a genius! Genius I say! I owe you one. If you are every stuck on something you can send me an email and I'll see what I can do to help.
This problem was killing me... because all my tests where run with whole numbers (e.g. 40, 42) whereas the computer traders are using values differing far to the right of the decimal point.
Lessons I've learned/been reinforced.
1) Check every damn value. Check it again. it doesn't matter if you print it in one spot, print it again after the next statemtn. Print it exactly as it is used in the equation.
2) JavaRanch rocks as do the people who use it. Go internet community!
3) I'm amazed at how many problems on this project disappear with time. I've often been stuck, and simply walk away, and the solution comes to me (usually I can think it up). I was so frustrated that for the last hour I was playing Warcraft, and viola, a solution came forth! Clearly I need to play more warcraft.
Thanks again Michael!!!
--Mark