File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
http://aspose.com/file-tools
The moose likes Developer Certification (SCJD/OCMJD) and the fly likes URLyBird 1.3.1 DB Access layer design, pls comment! Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Soft Skills this week in the Jobs Discussion forum!
JavaRanch » Java Forums » Certification » Developer Certification (SCJD/OCMJD)
Bookmark "URLyBird 1.3.1 DB Access layer design, pls comment!" Watch "URLyBird 1.3.1 DB Access layer design, pls comment!" New topic
Author

URLyBird 1.3.1 DB Access layer design, pls comment!

Leo Tien
Ranch Hand

Joined: Sep 10, 2002
Posts: 156
Now, I'v implemented my DB Access layer, as following:
classes:
-- Data.java : This class implements the DBMain interface, all the methods in it is synchronized except three methods -- lock(), unlock(), islocked(). I will select multiple data instance design, so every client will have it's own data instance. Like others, my Data class have a private, static WeakHashMap attribute, named lockedRecords, is used for locked records container. The Data class have other two static attribute too, one is DataSchema instance, the other is RandomAccessFile instance which is created by DataSchma's method getRAF(), these two attributes is private.
-- DataSchema.java : This class is singleton, to store some information: field names, field length, and so on. This class have a static RandomAccessFile attribute, in my DB Access layer only have this one RandomAccessFile instance, the DataSchema have a static method getRAF() return this instance.
-- DataHelper.java : This class supports several static methods, all these method is public and synchronized. This class's constructor is private, cann't create any instance of this class. The method in it to read or write the data to the db file.
locking :
In my DB Access layer, there is two level lock. One is DB file level, the other is record level. In the DB file level lock implementation, I use DataHelper. Because all methods in it is synchronized and all DB Access operation is in these methods. So, I think it would only allow one client to access db file at one time. In record level lock, I use WeakHashMap. In the map, I use Data.this as the key to identify who lock the record, and use the record's number as the value.
questions :
(1) In my assignment, there is this statement "The character encoding is 8 bit US ASCII. " I use these translation to ensure this :
When read --
byte[] temp = new byte[fieldLength];
raf.readFully(temp);
return (new String(temp, "US-ASCII")).trim();
When write --
byte[] valueToBytes = null;
....
valueToBytes = value.getBytes("US-ASCII");
raf.write(valueToBytes);
Whether this is enough?
(2) In my assignment, also have this statement "1 byte "deleted" flag. 0 implies valid record, 1 implies deleted record", so I use raf.writeByte(0x00) to set one record valid in create method and use raf.writeByte(0x01) to delete one record in delete method. Whether this is exactly?
(3) To test the lock mechanism, I create four thread lock and unlock five record again and again, the recNo is random. Lock and unlock method all call isValidRecord() method to exam one record whether valid or not. In my isValidRecord method, to use raf.readByte() to read the deleteFlag(this flag is one byte as statement above). But when my program reach this, always throws a EOFException. The time throw this exception is different, but at last always throw. Later, I use "byte[] data = new byte[1]; raf.read(data); return data[0];" instead of "return raf.readByte();", this Exception don't throw any more. Whether this take place?
(4) When a record is booked, what I can do on it? I read some thread in this forum, some people allow this record is booked again, I don't know whether this against the assignment, because the assignment isn't talk about it exactly. What do you think?
I hope your reply, thanks a lot.
George Marinkovich
Ranch Hand

Joined: Apr 15, 2003
Posts: 619
Hi Leo,
1) Yes, I think you've got it right. I wish I had remembered to use the character encoding on the write as well as the read.
2) Yes, I think that's right.
3) Not sure what to say about this. I guess I would be concerned about simultaneous access to the raf. The method is not sychronized but you seem to be taking care of the synchronization in DataHelper. I would look in the code for something that could allow two threads to simultaneously access the raf. I'm not sure why the substitution you made caused the exception not to be thrown again. I would still be concerned. Apparently fixing a problem without understanding how is sort of a dangerous thing to my mind. Getting errors are good things if they lead us to the flaws in our implementation.
4) Re-book an already booked record. Why not? It's not prohibited by the assignment instructions. It makes sense under some circumstances. It's easier to implement.
Hope this helps,
George


Regards, George
SCJP, SCJD, SCWCD, SCBCD
Leo Tien
Ranch Hand

Joined: Sep 10, 2002
Posts: 156
Hi George :
(3) Thanks your reply

isLocked can call isValidRecord. this is isValidRecord :
This is readDeleteFlag :
I find the reason cause this exception is two thread call getByte() method at the same time.
But today, I test this program in window XP, the exception is never thrown, I test it in window 2000 before, why this take place?
[ January 20, 2004: Message edited by: Leo Tien ]
[Andrew: removed unlock() method - please don't post complete solutions to your entire locking code.]
[ January 27, 2004: Message edited by: Andrew Monkhouse ]
George Marinkovich
Ranch Hand

Joined: Apr 15, 2003
Posts: 619
Hi Leo,
Thanks for posting the relevant code. It looks like the isValidRecord was responsible for the problem. Lock is not a synchronized method (for good reason), and it calls isValidRecord (which is also not synchronized). So, for instance, the "raf.seek(pointer);" in isValidRecord is occuring in a non-synchronized context. As such it could potentially interfere with an raf access from another method because it's not prevented from doing that by synchronization. One way of protecting it would be be to make isValidRecord synchronized.
I don't know why you're seeing a difference in Windows XP versus Windows 2000. But since this is case of multiple threads interfering with each other over a shared variable it is timing related and the problem could appear (or not appear) randomly, not only on different operating systems, but also between different runs of the executable on the same machine. Regardless, it's a very good thing to get an exception in this case because it led to analysis and fixing a latent bug in your implementation. It's a much worse situation for you not to get the exception and therefore not find out about the problem before making your submission. Better for you to find it than the grader.
Hope this helps,
George
Leo Tien
Ranch Hand

Joined: Sep 10, 2002
Posts: 156
Hi, George:
Thank you first !
Whether I can understand your reply as this : When one thread call raf.seek method, and now the position is 255. At the same time, the other thread catch the CPU, and call the raf.seek(100) method, now the position is 255+100. But when the db file's size is 300, it will cause the EOFException ?
One way of protecting it would be be to make isValidRecord synchronized.

isValidRecord method isn't only be called from lock and unlock method. It be called in other methods synchronized too, such as read. I don't know the shortcoming on one synchronized method call the other synchronized method, but I feel this isn't perfect.
George Marinkovich
Ranch Hand

Joined: Apr 15, 2003
Posts: 619
Hi Leo,
I think its possible to get an EOFException when you don't synchronize access to the raf and you have several threads calling methods that access the raf. The example you give would not seem to reasonably cause an EOFException. But since the problem is repeatable, I would suggest you put logging or System.out.println around each raf access and print out the file pointer's position before and after. Maybe you'll see an example that makes the EOFException being thrown seem reasonable.
On the second question, I think the following are equivalent:

I also believe that if your thread receives and holds the lock on "this" in a particular class then the next time you try to call a synchronized method in that same class it will not cause any problem. In other words, once you hold the lock on something, the next time you enter a code block that requires a lock on the same thing, then it's not a problem (you already have the lock so there's no need to wait to acquire it again). So, I think it's OK for one sychnronized method to call another synchronized method in the same class.
[WARNING: the following comments are based on a possibly controversial interpretation of the meaning of the RecordNotFoundException with regard to the lock and unlock methods. So if you find the argument made above convincing there's no need to read any further.]
There's a totally different approach to handling the lock and unlock with respect to the RecordNotFoundException. With a method like read it's pretty clear that the RecordNotFoundException should be thrown if the requested record number doesn't exist in the database. Consider the lock method. Maybe the RecordNotFoundException should be thrown if the requested record number doesn't exist in the database (in which case you have to access the database and you need to protect the raf access because the lock method is unsynchronized), or you can take the position that there are no circumstances under which RecordNotFoundException can be thrown from the lock method. So for example, if the client wants to book recNo = -23875, then you go ahead and lock that recNo. Your lock method never really looks for that record, it just adds it to the variable keeping track of the locked records, so it's not possible for it to be "not found" because you never really look for it in the database. It's not really going to cause any problems because that record number doesn't exist anyway. The question is this: Is it reasonable to allow the user to lock a record number that doesn't exist in the database?
In the unlock method maybe the RNFE should be thrown if the recNo doesn't exist in the database, or maybe the RNFE should only be thrown if the recNo doesn't exist in the variable you're using to keep track of locked records. You can make the argument that the latter interpretation violates the letter of the specification, but maybe it doesn't violate the spirit of the specification. Or maybe it does violate the spirit; it's a matter of interpretation. The advantage of not looking for the record number in the database is that you wouldn't need to access the raf from within an unsynchronized context.
Hope this helps,
George
[ January 22, 2004: Message edited by: George Marinkovich ]
Leo Tien
Ranch Hand

Joined: Sep 10, 2002
Posts: 156
Hi George, thank you very much, thanks.
In my design, there are two reasons throws RNFE: 1. the record is invalid. 2. the record is deleted already. In the interface which sun give me named DBMain.java, the lock and unlock method must throw RNFE, so I have to access the database and I need to protect the raf access if it in unsynchronized context.
Protect the raf access, this is a big problem. At first, I use DataHelper deal with it. The design on DataHelper is shown on above, the first post. When one thread call the lock method in Data.java, it catch the Data's lock first, then call isValidRecord catch the DataHelper's lock to access the db file. Before it finish the db access, other thread cann't catch the DataHelper's lock any more, this isn't enough protect the raf access ???
Wait your reply.
Thank you again!
George Marinkovich
Ranch Hand

Joined: Apr 15, 2003
Posts: 619
Hi Leo,
Originally posted by Leo Tien:
In my design, there are two reasons throws RNFE: 1. the record is invalid. 2. the record is deleted already. In the interface which sun give me named DBMain.java, the lock and unlock method must throw RNFE, so I have to access the database and I need to protect the raf access if it in unsynchronized context.

Agreed.
Protect the raf access, this is a big problem. At first, I use DataHelper deal with it. The design on DataHelper is shown on above, the first post. When one thread call the lock method in Data.java, it catch the Data's lock first, then call isValidRecord catch the DataHelper's lock to access the db file. Before it finish the db access, other thread cann't catch the DataHelper's lock any more, this isn't enough protect the raf access ???

1) The first thing that lock does is call isValidRecord to determine whether recNo is a valid record.
2) isValidRecord (which is not synchronized so it seems that it's possible for more than one thread to be in this method at the same time) does several raf-related things: A) accesses raf.length, B) modifies the raf pointer by seeking to the pointer position, and C) calls DataHelper.readDeleteFlag which calls raf.readByte.
3) After validating the record the lock method next obtains a lock for the recNo. At this point no other method (that is, one that checks the lock status of a record) can operate on this recNo. So at this point the recNo in question is protected from being accessed by any other lock-aware database operation.
I think there might be a problem with 2B since it is unprotected from concurrent access by multiple threads. I don't see anything else at the moment. Did you every try adding the logging (or print statements) to find out the state of the raf pointer when the exception occurs?
Hope this helps,
George
Jonathan Liu
Greenhorn

Joined: Aug 21, 2003
Posts: 26
If you don't mind, I would like to add something.
I think use FileChannel is simpler than using synchronized RAF if you are not sure your solution is perfect.
use methods:

you are guaranteed that file pointer will not be moved and also you get safe atomic file operation.
I have a question about your isValidRecord method. In this method, you check 1. the record is invalid. 2. the record is deleted already. However, look at the code in Lock method:

after isValidRecord is called and before synchronized(lockedRecords), there is a possibility that another thread using cpu cycle and delete this record and set the deleteflag to 1. In my design, I just check the record number is in the right range in the Lock method, even if this record is already deleted.
[ January 23, 2004: Message edited by: Jonathan Liu ]
George Marinkovich
Ranch Hand

Joined: Apr 15, 2003
Posts: 619
Hi Jonathan,
I think use FileChannel is simpler than using synchronized RAF...

Well, I didn't investigate FileChannel at all so I can't say if it would have been simpler. If it is simpler then I'm sorry I didn't use it. Thanks for sharing your opinion.
-George
Jonathan Liu
Greenhorn

Joined: Aug 21, 2003
Posts: 26
Using FileChannel, I don't need to synchronized anything.
in my readRecord method, I use the codes:

you see, to read a record, I just read once without moving the file pointer, plus read methos is atomic, so I think this is enough
Leo Tien
Ranch Hand

Joined: Sep 10, 2002
Posts: 156
Hi George, you are right. I bieleve in my program above the 2B can take place. Because I select multiple Data instance design, so the synchronized method in Data.java don't make much senses. So when I synchronize isValidRecord method, the problem appear yet.
Hi Jonathan, thanks your reply, it give me a new idea.
after isValidRecord is called and before synchronized(lockedRecords), there is a possibility that another thread using cpu cycle and delete this record and set the deleteflag to 1.

Good, I agree. I think this is the key of my problem. But I'v said, my unlock and isLock method must throw RNFE, so the validation must do in these two methods. So I modify my these method as following :

The problem don't rise any more !!
Hi George, if you want to see the printStackTrace, pls go on:
This is the segment in isValidRecord:

This is the problem:
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: URLyBird 1.3.1 DB Access layer design, pls comment!