File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
http://aspose.com/file-tools
The moose likes Developer Certification (SCJD/OCMJD) and the fly likes NX Problem with Threads and Concurrent Access to Database Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Soft Skills this week in the Jobs Discussion forum!
JavaRanch » Java Forums » Certification » Developer Certification (SCJD/OCMJD)
Bookmark "NX Problem with Threads and Concurrent Access to Database" Watch "NX Problem with Threads and Concurrent Access to Database" New topic
Author

NX Problem with Threads and Concurrent Access to Database

Mehmet Ali Atlihan
Greenhorn

Joined: Jan 14, 2004
Posts: 5
Hi guys
Thanks to everyone who posts here.It turns out here is an invaluable resource for SCJD takers
I have a problem when testing the concurrent access to database. I have created several threads to simulate the concurrency. The problem occurs when two threads try to update the same record. Here is what is supposed to happen
Thread A locks record 1
Thread B calls lock method on record 1 but as its locked it gives up CPU and goes to wait() state.
Thread A obtains a write lock (to provide mutual exclusion in RAF)
Thread A updates record 1
Thread A releases write lock
Thread A unlocks record 1
Thread A notifyAll()
Thread B resumes its execution..
What I experience is when Thread B gives up CPU the execution of the program stucks. I am expecting Thread A to obtain the CPU control and resume its execution but it just doesnt. I dont see any dead-lock indication in the code and it doesnt make sense.Here is the output I get
READTHREAD1 obtains a read lock
READTHREAD1 reads the record 0
READTHREAD1 releases its lock
READTHREAD1 Publishing Record: 0
Palace, Smallville, 2, Y, $150.00, 2005/07/27
UPDATETHREAD1 locked the record 0
record 0 is already locked UPDATETHREAD2 gives UP CPU
<-- stucks here UPDATETHREAD1 does not resume its execution -->
What am I doing wrong ? Any ideas will be appreciated
Thanks
Ali
George Marinkovich
Ranch Hand

Joined: Apr 15, 2003
Posts: 619
Hi Mehmet,
Originally posted by Mehmet Ali Atlihan:
Thread A locks record 1
Thread B calls lock method on record 1 but as its locked it gives up CPU and goes to wait() state.
Thread A obtains a write lock (to provide mutual exclusion in RAF)
Thread A updates record 1
Thread A releases write lock
Thread A unlocks record 1
Thread A notifyAll()
Thread B resumes its execution..

My first thought is that one should try to avoid a locking scheme that requires obtaining multiple locks. I guess there are some problems that require a locking scheme that requires obtaining multiple locks, but is this really one of them?
I would need to know more about how these two locks work in order to make a useful comment about what may be going wrong.
So I'll ask these questions:
1) When Thread A locks record 1, what really happens? Is record 1 added to some variable that keeps track of the locked records? Or does something else happen?
2) What does it mean to say that Thread A has a lock on record 1? Is this a read-only lock?
3) When Thread A gets a write lock on record 1, what really happens? Is record 1 added to some variable that keeps track of the records that hold an active write lock? Or does something else happen?
4) What does it mean to say that Thread A has a write lock on record 1? Does this mean that no other thread can update the record?
5) Do you really need a read lock and a write lock, or is it possible to satisfy the requirements using only a single lock?
After answering these questions you might come to the conclusion that you don't need to obtain multiple locks.
If not, it's possible to figure out how to successfully prevent deadlock in a locking scheme that requires multiple locks. However, I think it's more complicated than having a locking scheme that requires only a single lock (that is a lock that's used for both reading and writing).
You raised a concern about exclusive access to the RAF. This is indeed a legitimate concern but it may be possible to enforce exclusive access in ways other than using a write lock. For instance, if you knew that you could only have a single RAF, and you synchronized on this RAF whenever you accessed it, then wouldn't you be enforcing exclusive access?
Hope this helps,
George


Regards, George
SCJP, SCJD, SCWCD, SCBCD
Mehmet Ali Atlihan
Greenhorn

Joined: Jan 14, 2004
Posts: 5
Originally posted by George Marinkovich:
Hi Mehmet,

My first thought is that one should try to avoid a locking scheme that requires obtaining multiple locks. I guess there are some problems that require a locking scheme that requires obtaining multiple locks, but is this really one of them?
I would need to know more about how these two locks work in order to make a useful comment about what may be going wrong.
So I'll ask these questions:
1) When Thread A locks record 1, what really happens? Is record 1 added to some variable that keeps track of the locked records? Or does something else happen?
2) What does it mean to say that Thread A has a lock on record 1? Is this a read-only lock?
3) When Thread A gets a write lock on record 1, what really happens? Is record 1 added to some variable that keeps track of the records that hold an active write lock? Or does something else happen?
4) What does it mean to say that Thread A has a write lock on record 1? Does this mean that no other thread can update the record?
5) Do you really need a read lock and a write lock, or is it possible to satisfy the requirements using only a single lock?
After answering these questions you might come to the conclusion that you don't need to obtain multiple locks.
If not, it's possible to figure out how to successfully prevent deadlock in a locking scheme that requires multiple locks. However, I think it's more complicated than having a locking scheme that requires only a single lock (that is a lock that's used for both reading and writing).
You raised a concern about exclusive access to the RAF. This is indeed a legitimate concern but it may be possible to enforce exclusive access in ways other than using a write lock. For instance, if you knew that you could only have a single RAF, and you synchronized on this RAF whenever you accessed it, then wouldn't you be enforcing exclusive access?
Hope this helps,
George

Hi George thanks for giving your time
1) When Thread A locks record 1, what really happens? Is record 1 added to some variable that keeps track of the locked records? Or does something else happen?


Here is the lock() method i coded.Hope it clears out your question. As you may guess lockedRecords is a HashMap

2) What does it mean to say that Thread A has a lock on record 1? Is this a read-only lock?

No it is the lock mechanism itself that I pasted in the previous question
Any other thread to lock the same record will be hung
3) When Thread A gets a write lock on record 1, what really happens? Is record 1 added to some variable that keeps track of the records that hold an active write lock? Or does something else happen?

Actually that part might be the reason for this mess. My initial idea for providing the mutex in RAF was to synchronize the RAF itself as you mentioned .But after reading some messages in the forum some argue that it wont be an ideal choice to block the entire file and even when a record is locked, any read thread should read the record if its not updated yet. And someone introduced a LockManager mechanism that provides readLock and writeLock. It is a traditional reader/writer solution: at any given point of time only one write thread can execute whereas multiple readers should read at the same time.Needless to say write operation is mutually exclusive with both write and read operations and read operation is mutually exclusive with write operation. That made sense to me so i implemented a LockManager similar to this approach and delegate the concurrency issues with RAF to this class.
Back to your question I guess it is better to paste the code again. Note that this is the same code of the person who introduced this LockManager


4) What does it mean to say that Thread A has a write lock on record 1? Does this mean that no other thread can update the record?

To update a record , the thread should first obtain the lock on it. This is the logical synchronization. When it grabs the lock (ie: no other thread has already locked it), it asks for a writeLock(). If it is Granted then it can update the record. According to the implementation of LockManager having a write lock on a record is similar to blocking the entire file. No other thread can read/write regardless of the record they want to access during that writeLock holds.
5) Do you really need a read lock and a write lock, or is it possible to satisfy the requirements using only a single lock?

Probably I dont need read/write lock at all. As you also mentioned synchronizing the static singleton RAF would do just fine . Perhaps I have to stick with this simple solution. Or maybe just using FileChannel which I assume it provides thread safe access to the file
Thanks again
Mehmet
George Marinkovich
Ranch Hand

Joined: Apr 15, 2003
Posts: 619
Hi Mehmet,

Actually that part might be the reason for this mess. My initial idea for providing the mutex in RAF was to synchronize the RAF itself as you mentioned .But after reading some messages in the forum some argue that it wont be an ideal choice to block the entire file and even when a record is locked, any read thread should read the record if its not updated yet.

I was willing to have a single database operation interact with the database file at time, that is, I was willing to sequentialize the concurrent access to the database file in this way.
A LockManager with read and write locks will probably increase the level of concurrency depending on how it is implemented. However, one would need to be vigilant in routing out any possibility of deadlock. I guess I opted for simplicity over greater concurrency. You have to decide where you come down on this debate.
I didn't investigate FileChannel's so I don't really have anything intelligent to say about them. In general I would say they're not required, it's likely that they are at least a little more complicated to implement correctly (compared to RAF), and I'm not sure you'll be rewarded with a higher score because you implemented a more concurrent solution. The reward for using FileChannel's is the increase in your knowledge and experience, not necessarily in a higher score on the exam. I'm not against using FileChannel's just pointing out that it's possible to successfully complete the project without using them.
Hope this helps,
George
Mehmet Ali Atlihan
Greenhorn

Joined: Jan 14, 2004
Posts: 5
Hi George Thanks for the reply again
I have removed LockManager and go with the easier way that is synchronizing the RAF. Yet , the problem still occurs in case two seperate threads try to update/lock the same record
Thread A locks record 1
Thread B tries to lock record 1 but gives up CPU
Thread A is supposed to resume its execution but it doesnt. The program stucks
I am out of ideas. May be its my lack of knowledge about Threads. I am almost hundered percent sure that no two blocked threads are waiting on each other that is no dead-lock indiciation. What am I missing? Isnt a non blocked Thread in this instance Thrad A supposed to grab the CPU again?
What can make Thread A dead? Why can it be lost? Any debugging techniques you might suggest? Or any other way to test concurrency issue ? I am stucked..
Thanks
Mehmet
George Marinkovich
Ranch Hand

Joined: Apr 15, 2003
Posts: 619
Hi Mehmet,
My guess is that your locking scheme and raf synchronization scheme may be interfering with each other. But that's really just a guess.

1) Thread A locks record 1
2) Thread B tries to lock record 1 but gives up CPU
3) Thread A is supposed to resume its execution but it doesnt. The program stucks

So, after step 1 thread A has the lock on record 1.
Some time after 1 and before 2, Thread A is sliced out and Thread B becomes the active thread.
In step 2, Thread B attempts to lock record 1, but as it's unavailable, Thread B enters the wait state (waiting for the lock on record 1 to become available). Furthermore, Thread B isn't going to leave the wait state until it receives the notification it's waiting for.
At some time after step 3, Thread A is picked to be the actively running thread (but this doesn't seem to happen!)
Let me ask you this: If you run Thread A by itself (without Thread B running) does your application work the way you want?
If it doesn't that's good news because it will probably be easier to find the bug in that case. If it works in isolation then you do have some sort of interference problem and it's going to be harder to nail down.
I think you're going to need to use a debugger or logging. I found it more useful to use logging when I was trying to debug my locking scheme. I would basically log everything that happens in the lock and unlock methods. I would do this to a seemingly ridiculous extent. For example, before the call to the wait in the lock method I would log (Thread.currentThread() + " about to wait for lock on " + recNo)
After the wait I would log (Thread.currentThread() + " received notification that lock available for " + recNo). This might seem silly, but bugs involving threads are subtle and difficult (at least for me) to catch so I like to have a lot of information so that I feel I understand exactly what's happening at any moment in time.
I would run a single test thread and then analyze the resulting log until I understood exactly what was going on. Then I would introduce the second test thread and analyze the resulting log. This isn't easy or really a lot of fun, but after you do this not only will you probably discover your bug, but you'll have a deeper understanding of locking and threading which is very useful to have, not least because you'll have more confidence explaining your locking scheme when you take the essay part of the exam.
Hope this helps,
George
Mehmet Ali Atlihan
Greenhorn

Joined: Jan 14, 2004
Posts: 5
Hi George thanks for giving your time
[Let me ask you this: If you run Thread A by itself (without Thread B running) does your application work the way you want?

Unfortunately Thread A standalone executes perfectly as it is supposed to do. Moreover a Read Thread and a Write Thread(update/delete/create/) works OK concurrently too.
I agree that debugging a multi-threaded application is a pain in the butt. Although The IDE I am using (JBuilder) has a powerfull debugging mechanism when it comes to thread debugging it doesnt much help. I am doing some primitive logging (System.out) in lock and unlock already but I will extend it with respect to your advice and let you know what happens
Thanks
Mehmet
Mehmet Ali Atlihan
Greenhorn

Joined: Jan 14, 2004
Posts: 5
Hi George
After logging extensively I realized that the program stucks in synchronized axuliary methods. Removing the synchronization over these methods enables two write threads work correctly concurrently.
But here comes my question. Normally mutex is required for a shared data
For instance a hashmap of lockedRecords must be mutually exclusive. But since Data class is singleton my thaught was as each Thread is working on the same object its methods should be invoked in a synchronized way.
Suppose I have a private method of Data class called "validateRecord" that is invoked from each of the public methods of Data clas(read,update,create,delete).

I thought validateRecord should be synchronized too for the given reasons. But after debugging I realized that the compiler creates a shallow copy of Data Class (or somehow its methods) so each thread is able to keep track of its associated local variables and as such can act on those variables without the worry of corruption of those datas.
I am still not sure so I do appreciate your feedback
Thanks
Mehmet
George Marinkovich
Ranch Hand

Joined: Apr 15, 2003
Posts: 619
Hi Mehmet,
Congratulations on your successful logging.
About your question:
When a synchronized method calls another method in the same class, then the called method is essentially synchronized as well. So although validateRecord is declared to be unsychronized, since it's called from within another synchronized method, for the duration of that call it is itself synchronized. In fact, if validateRecord is only ever called by synchronized methods, then it may as well be declared synchronized itself. It's not a problem for a synchronized method to call another synchronized method (if they're in the same class).
I think this explains the phenomenon you've been seeing.
Hope this helps,
George
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: NX Problem with Threads and Concurrent Access to Database