client locking for URlyBird 1.3.2

[Lee Sunter]

Hi All,

I am close to admitting defeat with this locking issue! so hopefully you guys can help! My URlyBird only allows the passing of the record number in the interface. So we pass a reference of the dataRemote to the locking class, which we then make a new weak reference, this is used to store the record number that is locked (this is the id) and then the weak reference to the client that has made the lock is stored in a hashmap. The locking and subsequent booking of the record is fine, BUT (why is there always a but), the unlock method again only takes the record number as a parameter and then we pass a reference of the dataremote to the locking class, which we then make a new weak reference this is then used to compare with the weak reference that is stored in the hashmap for the record id (ie the row number), these two client weak references NEVER match Why? I compared this solution with a book and they do exactly the same and when I try to run their example 'solution' code it too fails in the same way. So I am either doing something wrong in the locking code or I am missing something silly.
Any help is appreciated.

[Lara McCarver]

So we pass a reference of the dataRemote to the locking class, which we then make a new weak reference, this is used to store the record number that is locked (this is the id)

What does this mean, "pass a reference of the data Remote"... is that a "this" object? And what is a "weak Reference"? What book is it that does this and fails?

And why can't you figure out what is going on? If you use Eclipse, it has a built-in graphical debugger. (And the latest version, 3.1, which I just downloaded last night, supports J2SE 5.0.)

[Lee Sunter]

Sorry yes it is 'this' object that I am passing. I run a server and I run 2 clients all on the same machine, with the localhost and port 60000. The remote reference is the same and its only when I create a weak reference that the values are different for the clients (in fact they are always different).
A weak reference (I think is a refernce to an object that when the object dies it is garbage collected and the weak reference dies with it).

The book is Java 2 Developer Exam Cram 2 (Exam CX-310-252a and CX-310-027)

web page

The book is useful as it seams to cover most parts of the exam but this part (the bit I need help on) does not work.

[Lara McCarver]

I never bought that book, so I can't comment on their code, but from your description, you are passing the Data object to the lock() method, and then the WeakReference is created inside the lock() method. Similarly, I imagine that you pass a Data object to the unlock() method, and create a WeakReference there too.

The problem could be that the 2 WeakReferences that you are creating from this Data method are not "equals". I just looked up WeakReference in the JavaDoc, and there is no equals() method. I also looked up its superclass, and it does not have an equals() method either. That means comparisons between 2 WeakReferences are going to use the exact address of the reference, and since there are 2 of them, they might not have the same reference.

Keep in mind, I have never used a WeakReference, I probably am woefully ignorant about it.

Anyway, I have a similar interface, but what I do is have a simple static int in my Data class, and in the constructor, each Data class uses this to create a unique locking token (and then increments the static int). So I end up testing the equality of 2 Integers, which always works perfectly.

You could even argue that this is approach is more likely to comply with future requirements, because the client could pass in a unique clientID and you could use that as your locking token instead... which would allow you to keep an audit trail of the activities of each client.

[Andrew Monkhouse]

Hi Lee,

I recommend that you stop using the WeakReference for the moment - get the code working using a normal collection of Data objects, then once you are convinced it is working then you can try experimenting with the WeakReference class (if you want to).

Unfortunately I do not have my copy of this book around anymore, however when I reviewed it almost 2 years ago, I noticed that the book had serious problems with the locking in general and with how the server developed in the book used the data classes. I had hoped that Alain Trottier would list these in an erratta page (I passed the details to him), however he doesn't even list this book in his published works anymore . So I cannot find an erratta page for it.

I discussed some of the problems in this post. Max's comment is still very valid, and may save you a lot of grief .

Regards, Andrew

[Lee Sunter]

Thanks, I have now removed the weakreferences. But I have another issue all my clients have the same reference and I cant understand how to have unique values for each client. my server calls the DataRemote. My clients call the database factiry and look up the server but both have the same reference to the server data base ie
lookupString = "rmi://" + host + ":" + port + "/" + RMI_SERVICE_NAME;
System.out.println("Database is " + (DBMain)Naming.lookup(lookupString));
return (DBMain)Naming.lookup(lookupString);
so both cleints have the same value printed out.
the clients call the data and the interface only allows the client to pass the row id (sun assignment specified). In the DataRemote it calls the lock class (which is fine) but I dont have a different DataRemote for each client (should I so that I can tell the difference between the clients, and if so what am I doing wrong ?)

the server class calls the
createRegistry(port) then creats a new DataRemote
DBMain databaseServer = new DataRemote(db);
then does the rebind for that DataRemote, but this only occurs once?

[Andrew Monkhouse]

Hi Lee,

What Alain needed for his program to work was to either use cookies, or to use a Connection Factory on the server side.

That is, you should have two Remote objects on the server: one is the Factory, and one is the Data Adapter (that adapts the Data class so that the Remote requirements can be met). Only the Factory gets bound to the registry. The clients get their own instance of the Data Adapter by calling the appropriate Factory method.

Usage of this in psuedo-code would be something like:

[B]server side[/B]
    Factory.getDataRemoteAdatper() {
        return new DataRemoteAdapterImplementation();
    }
 
[B]client side[/B]
    Factory f = Naming.lookup("MyDatabaseFactory");
    DataRemoteAdapter database = f.getDataRemoteAdapter();
    database.find() // or whatever

If you do this, when your client is calling the remote method, they are calling a method on a unique instance of the Remote implementation of your Adapater. If this Adapter has a unique instance of the Data class, then within the Data class you can use the this keyword to identify the particular client.

I demonstrate a basic connection factory in this post.

Regards, Andrew
[ August 09, 2005: Message edited by: Andrew Monkhouse ]

[Lee Sunter]

Thanks Andrew, the waters are starting to clear, but the course and book I have used dont have 2 remote objects, i have the Remote data which is the remote implementation of the DBMain interface for the data access rtequirments, as you called Data Adapter. This second remote class that I need, this is somethiong new that I havent got, when does this get called
what I am trying to do is from the server call the AdapterConnection class
Naming.rebind("//:60000/DatabaseService", new AdapterConnection(db))
This then in the AdapterConnection do this
Registry registry = LocateRegistry.createRegistry(60000);
registry.rebind("DatabaseService", new DataRemote(databaseFileName));
Then we exit the server.

In the Client call the database factory it works out from the params that we are running in remote mode,
I think I should be looking up the
Naming.rebind("//:60000/DatabaseService", new AdapterConnection(db))
value set by that line, ie so I am getting an AdapterConnection object

hence the line (excuse the poor name for the look up)
AdapterConnection connection =(AdapterConnection) Naming.lookup("//:60000///:60000/DatabaseService");
then I return the DBMain interface type fro the dataRemote using the get
getMyServer() method in the AdapterConnection

but I am getting classCastExecptions (one of my favorite )

Any ideas on where I am going wroing and in case I am on the wrong track?

[Andrew Monkhouse]

Hi Lee

...but the course and book I have used dont have 2 remote objects

The problem is that Alain's code uses the instance of DatabaseRMIImpl to identify who owns the lock, however there is only ever one instance of DatabaseRMIImpl. So although Alain doesn't go into details, he needs a connection Factory to create multiple instances of DatabaseRMIImpl. And both the DatabaseRMIImpl and the Factory would have to be Remote objects.

Which course did you do? Did they also have a lock method that did not use cookies?

what I am trying to do is from the server call the AdapterConnection class
Naming.rebind("//:60000/DatabaseService", new AdapterConnection(db))
This then in the AdapterConnection do this
Registry registry = LocateRegistry.createRegistry(60000);
registry.rebind("DatabaseService", new DataRemote(databaseFileName));
Then we exit the server.

So at the end of all this, you will only have the DataRemote instance bound - the rebind will have replaced the AdapterConnector instance.

hence the line (excuse the poor name for the look up)
AdapterConnection connection =(AdapterConnection) Naming.lookup("//:60000///:60000/DatabaseService");
then I return the DBMain interface type fro the dataRemote using the get
getMyServer() method in the AdapterConnection

but I am getting classCastExecptions (one of my favorite )

Which means you are trying to cast the DataRemote instance to an AdapterConnector instance.

Where you are going wrong is that "Only the Factory gets bound to the registry". Take another look at my code in this post to see an example.

Regards, Andrew
[ August 09, 2005: Message edited by: Andrew Monkhouse ]

[Lee Sunter]

Apologies Andrew I seamed to go off on one yesterday.
I have looked again an yes the classCastException has been solved, but I am unsure on a few points (sorry if I am being really dim here).

server class creates a registry and binds the dataRemote class (only run once when the server starts)
Registry registry = LocateRegistry.createRegistry(60000);
registry.rebind(RMI_SERVICE, new DataRemote(db));

The client calls the databasefactory (this works out depending on input args if its remote or local).
For the case of remote I create a new registry for the AdapterConnection
Registry registry = LocateRegistry.createRegistry(60001);
registry.rebind("Server", new AdapterConnection(DBPATH))
In the AdapterConnection the constructor does nothing.
I then do a look up for this
AdapterConnection rmctx = (AdapterConnection)registry.lookup("Server");
and then from the rmctx i call the getmyServerData method which looks up the dataRemote and returns the dataRemote interface which is then returned to the client.

Registry registry = LocateRegistry.getRegistry(60000);
return (DBMain)registry.lookup("DatabaseService");

The this object in the dataRemote is still thye same for both cleints because its the same object, what am I doing wrong, I think I am close (hope so)

Thanks
Lee

[Lee Sunter]

Andrew I have changed my mind,
the server calls a bind on the the AdapterConnection class which implemnts the DB interface
registry.rebind("Adapter", new AdapterConnection(db));
The AdapterConnection creates a new dataRemote class.

The client calls a lookup on the Adapter and this is then used by the clients but these are not unique so I am still stuck with the same problem


I think I am going round in circles!!

[Andrew Monkhouse]

Hi Lee,

OK, let's look at what a server is in my factory example:

     1  import java.rmi.registry.*;
     2  import java.rmi.server.*;
     3
     4  interface MyServer extends java.rmi.Remote {
     5      public int getServerNumber() throws java.rmi.RemoteException;
     6  }
     7
     8  public class MyServerImpl extends UnicastRemoteObject implements MyServer {
     9      private static int numberOfServersCreated = 0;
    10      private int serverNumber = numberOfServersCreated++;
    11
    12      public MyServerImpl() throws java.rmi.RemoteException {}
    13
    14      public static void main(String[] args) throws Exception {
    15          LocateRegistry.createRegistry(1099);
    16          java.rmi.Naming.rebind("factory", new ConnectionFactoryImpl());
    17          System.out.println("Server started");
    18      }
    19
    20      public int getServerNumber() throws java.rmi.RemoteException {
    21          System.out.println(this + " running");
    22          System.out.println("returning server number " + serverNumber);
    23          return serverNumber;
    24      }
    25  }

Lines 4 - 6 are a simple RMI interface for my server method(s).

Lines 9 & 10 are not really needed - they just provide a simple integer identifier so I can easily see whether or not I have a unique instance of a server.

Skipping line 16 for the moment, line 21 is printing "this" to standard output. This is possibly the most dangerous section of the code - depending on which version of the JDK you are running, the toString method of the UnicastRemoteObject might not provide you with enough information to see whether we have an unique instance of MyServerImpl or not. This is why we have line 10, which we use in line 22 - we can use our variable to confirm that we do have an unique instance of MyServerImpl.

Line 16 is the interesting line - I have not bound the ServerImpl to the registry, but rather I have bound an instance of the ConnectionFactoryImpl. So let's look at the connection factory next:

     1  public interface ConnectionFactory extends java.rmi.Remote {
     2      public MyServer getMyServer() throws java.rmi.RemoteException;
     3  }
     4
     5  class ConnectionFactoryImpl extends java.rmi.server.UnicastRemoteObject
     6          implements ConnectionFactory {
     7      public ConnectionFactoryImpl() throws java.rmi.RemoteException {}
     8
     9      public MyServer getMyServer() throws java.rmi.RemoteException {
    10          return new MyServerImpl();
    11      }
    12  }

The only line of interest is line 10 - whenever a client calls this, we will return a new instance of MyServerImpl. So all we need is a client to test this:

     1  import java.rmi.*;
     2
     3  public class Client {
     4      public static void main(String[] args) throws Exception {
     5          ConnectionFactory connections =
     6              (ConnectionFactory) Naming.lookup("factory");
     7          MyServer server = connections.getMyServer();
     8          System.out.println("using server # " + server.getServerNumber());
     9      }
    10  }

In this, lines 5 & 6 are looking up the factory, which only provides the one useful method - the getMyServer method. We use this method in line 7 to get a unique instance of the server, and then we print out the instance number in line 8.

The following two screen shots show me compiling the classes, running rmic, running the server, then running two clients:





It is important to realise that the output from printing this (shown in my example as MyServerImpl[UnicastServerRef [liveRef: [endpoint:[192.168.0.2:2529](local),objID:[1]]]] running) can differ depending on what machine you are running on and what JVM you are using. In particular, the bit that identifies the unique objID (objID:[1]) may not appear in all versions of Java. I am currently running JDK 5 (which means I did not need to run rmic, but I left it in as a convenience).

Because the earlier JVMs (and JVMs from manufacturers other than Sun (e.g. IBM or Apple) may not display a unique identifer, I have also printed my own server identifer (returning server number ...).

Does this help clear it up? If not, can you modify my code in some way to demonstrate where you would like extra clarification?

Regards, Andrew

[Lee Sunter]

Thanks for that but I am not understanding how the two sets (yours and mine) map together, we run the server once and we run several clients
these clients each call their own connection class which looks up the shared dataRemote class.

yours                mine
-------------        ------------------------------------------------------
client               databaseFactory (chooses which type of db we want
                     depending on the client this returns the data class
myserver             UrlyBirdServer (this is only run once)
connectionFactory    Adapterconnection
?                    RemoteData (the one databasde file)

I understand that the client wants one each of the connection factory but they share the data factory.

so your client code matches my databaseFactory (which is called by the client to choose which database) this looks up the connectionFactory (factory), and this returns a new instance of MyServer (whats this is this not meant to be the RemoteData class ie the shared file, but from your example this is the server)?

[Andrew: Put table inside ubb [code] tags]
[ August 11, 2005: Message edited by: Andrew Monkhouse ]

[Lee Sunter]

Ok,

I now have the server putting the remoteData in th registry.

a client runs and just gets a new ConnectionFactory (conf1)
a client runs and just gets a new ConnectionFactory (conf2)

the connectionFactory class has a look up for the remoteData and returns that. therefore both clients have access the the shared dataRemote though an individual connectionFactory. ConnectionFactory also implements the Database interface as specified by the sun requirments, so all the methods just call the method on the shared dataRemote class, all this works fine for all methods (is this a valid solution) BUT the locking is still an issue , so if i call the lock method on the shared dataremote class then I get the old error of the this being same inside the dataremote class which calls the locking method. The other option is that i call the locking method direct then each client has its own lock hashmap so that doesnt work as both clients can lock a record and its only though additional checking before the update that the client is refused an update. So yet again help!

(I so wished i had the assignment with the cookies option in the interface)

[Andrew Monkhouse]

Hi Lee,

yours                mine
-------------        ------------------------------------------------------
myserver             UrlyBirdServer (this is only run once)

Hmmm. If you look at my code above, at line 10 of the ConnectionFactoryImpl I create a new instance of MyServer for every client who connects. You can verify this in the outputs of both the clients and server.

yours                mine
-------------        ------------------------------------------------------
?                    RemoteData (the one databasde file)

I suspect that this is where the problem may be. First I would like to confirm that each connected client does have it's own instance of MyServer (or UrlyBirdServer in your case).

If we are agreed on that, then the issue may be that you only have a single instance of RemoteData. If each client has their own instance of RemoteData (and hence, their own instance of Data), then within the Data class, you can use the instance of the Data class itself as the identifier.

This may mean that you may have to rethink your Data class or your RemoteData class - perhaps make it a Facade or have some static members - but I am getting ahead of myself here. First we have to confirm that each client does have it's own instance of UrlyBirdServer on the server.

Regards, Andrew

[Lee Sunter]

Thanks Andrew, if thats what i am supposed to do, then I shall try that, as that will help make sense for the rest of the problems. I was under the impression that there would be just one server which the user specifies the port number and the database but these could also be passed into the server instance when the client calls for their copy of the server.
Is this ok?

[Lee Sunter]

Sorry just going back to the book I menmtioned, doesnt he just have one server or maybe I miss understood that and hence my confusion?

[Lee Sunter]

Hi I still think i am going to get the same problem. the DataRemote class which is shared between all the clients (ie I only want one) so thats ok on the whole for the methods but for the lock this method takes in just an int (ie the record number) see suns interface definition below.

// Locks a record so that it can only be updated or deleted by this client.
// If the specified record is already locked, the current thread gives up
// the CPU and consumes no CPU cycles until the record is unlocked.
public void lock(int recNo) throws RecordNotFoundException;

then in the data remote I was passing to the locker lock method the recNo and 'this' to represent the client thats locking the row. I will always have this problem! as the data class is shared.

I have a new cunning plan but I am unsure if its ok by Sun. If I pass for the recNo an int that represents the recNo and the client (number) ie
say record int is 18 and we are allowing 1000 clients and we are client number 2
we pass the int 18002 to the data
that strips by 18001/1000 = 18 so we have the record number
and we do a mod(1000) and the remainder is 2 which is passed as the client id to the locker method

Is this allowed keeping in mind the interface specified by sun. If this is then I am ok and I dont have to creat any more remote objects other than the one remoteData file and we only need one server, this reduces a lot of classes and complexity and I make a not that I would have changed the interface to allow a client id. So what do you think?

How have other people solved this under version 1.3.2? there must be others out there?

[Lee Sunter]

Hi can any one help on this, especially anyone who has version 1.3.2 as I feel so close yet so far
[ August 15, 2005: Message edited by: Lee Sunter ]

[Lara McCarver]

I am doing basically what Andrew is suggesting... creating a new RMI server object for every client. Each RMI server object contains a Data object (well not directly, but through a layer or two of indirection, but that is the general idea). Once you get to the point where each client is getting its own Data object, then its easy to attach a unique id to the Data object. The approach of having a separate RMI server for each client is discussed in the SCJD Exam book (and Andrew's updated version is coming out in just a couple of weeks according to Amazon.com) and also in Java RMI, which provides a discussion of why you might want to use multiple RMI servers like this even if you weren't led into it based on the Data interface you are given.

[Lee Sunter]

Laura, hos does the data stay in sync with the rest of the other users, do we not need to have only 1 data class and so that when updates have been made then other users will see the changes, ie a booking.

What i do is in the constructor of the data class i read in from the data file into a local 2d array and this is then used for the database in the application. Changes to the data are just shown in this 2d array. If i have 2 data remotes which basically call the data class then I will get 2 separate 2d arrays which will not show the updates. (Possible way is to re-read from the data file all the time but, this is not ideal, plus i would have to update the actual data file, which I dont do for the application in such, (but i have added a method to do this but the application doesnt use this)). I take it that from your solution you work with the datafile making changes to the datafile as they happen is that correct?

[Andrew Monkhouse]

Hi Lee,

Originally posted by Lee Sunter:
Laura, hos does the data stay in sync with the rest of the other users, do we not need to have only 1 data class and so that when updates have been made then other users will see the changes, ie a booking.

You can have static objects for items such as your cache (your 2D array), your file descriptors, your collection of locks, and so on.

Perhaps the following code sample will help (deliberately obfuscated from the problem in the SCJD assignment, but not by much):

public class CheckingIDs { // just a class to create a couple of clients
    public static void main(String[] args) {
        for (int i = 0; i < 2; i++) {
            new Client().start();
        }
    }
}
 
// Equivalent to AdapterConnection? (Remember RMI gives you threads automagically)
class Client extends Thread {
    static int numberOfClientsCreated = 0;
    int clientId = numberOfClientsCreated++; // An ID to show who is doing what
 
    public void run() {
        Data data = new Data();  // important: each client has their own Data
 
        System.out.println("Client " + clientId + " doing something");
        data.doSomething();
 
        // wait a random amount of time so we no longer know whether [B]this[/B]
        // particular client was the last caller
        try {
            Thread.sleep((int) (Math.random() * 500)); // sleep up to 1/2 second
        } catch (InterruptedException ignored) { /* dont do this */ }
 
        System.out.println("Client " + clientId + " doing something else");
        data.doSomethingElse();
    }
}
 
class Data {
    static Data lastCaller = null;  // some objects should be common across all
                                    // instances of Data, otherwise our checks
                                    // wont work.
 
    public void doSomething() {     // just track who the last caller was
        lastCaller = this;
    }
 
    // report whether this caller is the same as the last caller
    // since that is all we want to prove, just exit abruptly.
    public void doSomethingElse() {
        if (lastCaller == this) {
            System.out.println("methods called in sequence");
        } else {
            System.out.println("methods called out of sequence");
        }
        System.exit(0);
    }
}

This code is deliberately not thread safe (it is already longer than code samples I like to post), but it does show the basic concepts that I am trying to get across: namely using the instance of Data to identify the client, and using static object to store information that all instances must use.

A sample run:



Regards, Andrew

[Lara McCarver]

Lee, I initially was planning on creating a DatabaseCache. It was going to be a class, but conceptually similar to your 2D array I think. I initially went with that design because when you have been programming for a long time, it sort of gets drilled into your head that File I/O is very inefficient compared to reading from memory. The problem with using a Database Cache is that when the database changes, due to users calling delete(), update() or create() in your Data class, you are eventually going to have to update your DatabaseCache. But this gets into some complex threading issues. Of course, the problems are all solvable. You can keep track of all the threads which are doing read() methods, and when you know you need to do an update(), start locking them out of the Cache object, then wait until all the reads are finished, then update the Cache, then release them all... it is all possible, and a very interesting threading problem, but it also introduces unnecessary complications. Because apparently, people pass the exam with full marks with or without a cache.

And honestly, pretend this was a RL application. Is it really a good idea to create a DatabaseCache class? It would probably be a lot smarter to buy an inexpensive database server and use it to do your caching! That is why I don't think it is worthwhile to write a cache class. Yes, if this was a company that was trying to write a new Database server to sell on the market, then it would be worth writing a DatabaseCache class, but this is not a Database Server company, it is a contractor company, so it's better to focus your effort on the real business problems this company is going to face.
[ August 16, 2005: Message edited by: Lara McCarver ]

[Lee Sunter]

Thanks Lara, this clears lots of issues but this means I am going to have to change my code (and fundamental design decision) of having the database cached, before I leap into this change is there no other way, with caching?

(if I didnt implent the lock method of the data interface and just called the lockmanager with the client id and the record id then i have finished and I only need one dataremote which all the clients can share, but i think i would fail for not implementing the interface method lock etc? what does everyone think?

If i do change (like Lara did, and I think I will), for the clients to be upto date do i just call the data class for all the reads instead of passing the cached data around and reading from that?

[Lee Sunter]

Thanks guys for all your help I now have the caching issue solved.