This week's giveaway is in the Android forum.
We're giving away four copies of Android Security Essentials Live Lessons and have Godfrey Nolan on-line!
See this thread for details.
The moose likes Developer Certification (SCJD/OCMJD) and the fly likes [locking] Hand over hand, remove() ? Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Android Security Essentials Live Lessons this week in the Android forum!
JavaRanch » Java Forums » Certification » Developer Certification (SCJD/OCMJD)
Bookmark "[locking] Hand over hand, remove() ?" Watch "[locking] Hand over hand, remove() ?" New topic
Author

[locking] Hand over hand, remove() ?

Sebastian Puzon
Greenhorn

Joined: Sep 04, 2008
Posts: 13
Hi!

I'm wondering if anyone decided to implement 'hand over hand' locking mechanism and what was the final score?

Of course there is one issue with this approach, where is remove()? Map with locks may grow and grow to the size equal to number of records in db. That's the scalability problem. Did anyone solved that?

I'm only concerned with this locking mechanism, I know that other exist

Regards
Sebastian.
[ October 10, 2008: Message edited by: Sebastian Puzon ]
Alecsandru Cocarla
Ranch Hand

Joined: Feb 29, 2008
Posts: 158
I chose to implement something somewhat similar, with a growing map of locks. But my code does not use hand-over-hand, because I consider it is not necessary to keep the master lock until acquiring recordLock. I do recordLock.lock() after releasing the master lock. I don't see any problem in this. If another thread comes and tries to acquire the same recordLock, then the first thread will have to wait, so what's the big deal? Why should it be "necessary" to do this hand-over-hand?

You could remove the the locks from the map at, for example, unlock(). It should be something like:



As I said in my comments, I'm not sure that this will work as expected (especially if you set the cookie to null when removing it from the map), but, theoretically, I think it will... I chose to ignore the memory consumption issue and keep the code simpler. I documented it in the choices file. I hope this will be enough...


SCJP 1.4 100%
SCJD 99.5%
Liviu Carausu
Ranch Hand

Joined: Oct 07, 2004
Posts: 157
Hi Sebastian,
Of course there is one issue with this approach, where is remove()? Map with locks may grow and grow to the size equal to number of records in db. That's the scalability problem. Did anyone solved that?


I think you are allowed to remove the lock from the map in two situations:
1) when the corresponding record is removed from the database and after the
waiting threads are notified
2) after unlocking the record if there are no waiting threads.(You have the possibility to check if there is any thread waiting for a condition)

Removing the lock when there are more threads waiting to lock the record will fail because when the first thread will be notified it will create a new lock in the map and the other waiting threads will never be notified because they are waiting at the removed lock.

I'm also using the new concurrency Api to implement the locking but i remove the locks from the map only when the records are deleted from the database. I keep this part away because I do not want to complicate the code too much.

Hope this helps,
Liviu


Oracle Certified Master Java SE6 Developer(SCJD),
OCE JEE 6 JSP and Servlets Developer.
Sebastian Puzon
Greenhorn

Joined: Sep 04, 2008
Posts: 13
Hi Liviu!

Yes, I saw this thread Multiple notification Point I think this approach with Condition per record and one Reentrant lock to map with cookies is very good. I was considering that implementation but now I'm near to submit (few hours) so I'll leave the simplest solution with synchronized methods (hope it will enough) Thanks for update!

Regards.
Anton Golovin
Ranch Hand

Joined: Jul 02, 2004
Posts: 476
Locking perpetually puzzles SCJD takers. All you need is to know how to implement locking without wasting CPU cycles.

You do this by waiting/notifying a dedicated object (per thread). You can store these objects in a Queue to ensure fairness and completeness of notification. This is the physical locking mechanism.

Then, also, you need to provide for a logical locking mechanism, and you can do that with a HashMap, but the actual locking flag must be an atomic one, a boolean or an integer.
[ November 01, 2008: Message edited by: Anton Golovin ]

Anton Golovin (anton.golovin@gmail.com) SCJP, SCJD, SCBCD, SCWCD, OCEJWSD, SCEA/OCMJEA [JEE certs from Sun/Oracle]
Karen Smrha
Greenhorn

Joined: Nov 13, 2008
Posts: 11
Please correct me if I am wrong, but I think if you use record level locks you do need hand-over-hand locking. The master lock is what locks the collection containing your row locks.

If the collection was read-only, we would not need to lock it, but lock() add elements and unlock (if you implement lock removal for deleted records) removes elements. We need a safe way to modify the collection in a multi-threaded environment.


SCJD working on B&S
Anton Golovin
Ranch Hand

Joined: Jul 02, 2004
Posts: 476
Don't lock rows in the database file.
Alecsandru Cocarla
Ranch Hand

Joined: Feb 29, 2008
Posts: 158
Originally posted by Karen Smrha:
Please correct me if I am wrong, but I think if you use record level locks you do need hand-over-hand locking. The master lock is what locks the collection containing your row locks.


As I said earlier, I don't understand why this hand-over-hand is really necessary. What is the difference between:

and



In the second case, the worst thing that might happen is that another thread can lock the recordLock() in between unlocking the masterLock and locking the record lock. This is not a problem, two threads compete for the same lock, that's it. But in the first case, you just keep the masterLock unnecessarily (blocking all other threads, not only the ones that try to get the specific recordLock) until locking the recordLock.

Originally posted by Anton Golovin:
Don't lock rows in the database file.

Why not?
[ November 19, 2008: Message edited by: Alecsandru Cocarla ]
Karen Smrha
Greenhorn

Joined: Nov 13, 2008
Posts: 11
The difference is:






I am not sure if there is any real difference if your system never removes record locks from the collection.

If you do release the master before getting the record-level lock, I suggest adding an implementation comment explaining this. Otherwise, someone else working on the code might decide to add code that removes entries from the collection and breaks your locking.


Of course there is one issue with this approach, where is remove()? Map with locks may grow and grow to the size equal to number of records in db. That's the scalability problem. Did anyone solved that?


I added lock removal to my unlock method. If the record doesn't exist in the DBMS and there are no waiting threads, I remove it.

You need to hold the master lock to check the number of waiting threads and remove the entry. Otherwise, a new waiting thread may be added while you are working.

So far, my code seems ok, but I think I need run a test with many threads working on a single record with some of them deleting it.

- Karen Smrha
Alecsandru Cocarla
Ranch Hand

Joined: Feb 29, 2008
Posts: 158
Originally posted by Karen Smrha:
I am not sure if there is any real difference if your system never removes record locks from the collection.
- Karen Smrha

Hmm, yes, it's true, I only thought from my perspective (I chose not to remove the locks from the map). This issue can be solved by not re-setting the cookie to null, and waiting until a timeout occurs (see my first post on this topic). Of course, this is not really elegant, and your approach sounds better for someone who wants to remove those locks. Probably, the performance hit is also quite small.

Originally posted by Karen Smrha:
If you do release the master before getting the record-level lock, I suggest adding an implementation comment explaining this. Otherwise, someone else working on the code might decide to add code that removes entries from the collection and breaks your locking.
- Karen Smrha

Not really. The code is in the same class, and, anyway, such decision is not taken over night. Everything involving multithreading should be carefully thought/tested before doing so. For example, even with your approach, if the junior forgets to lock on the masterLock, then it's still possible to break the application. If, doing so, he also sets the cookie to null or if you don't have a timeout, then again, it's a very big problem.

Also, the remove-lock code can be implemented in many different ways, not only as you proposed. For example, a thread which runs from time to time and checks the map for unused locks. Or by using weak references, and so on.

Originally posted by Karen Smrha:

So far, my code seems ok, but I think I need run a test with many threads working on a single record with some of them deleting it.
- Karen Smrha

I might suggest using this one :
http://www.coderanch.com/t/418342/java-developer-SCJD/certification/Data-class-multithreaded-load-test
Karen Smrha
Greenhorn

Joined: Nov 13, 2008
Posts: 11
Alecsandru,

Thanks for posting your multi-thread test. I will run it; the more testing the better. I also have a 3-tier design. You are right, SUN will hit our database tier as hard as they can and will do things our application cannot.

In case you haven't seen it, here's another test:
http://www.coderanch.com/t/189353/java-developer-SCJD/certification/JUnit-Test-Cases


Originally posted by: Alecsandru Cocarla
Not really. The code is in the same class, and, anyway, such decision is not taken over night. Everything involving multithreading should be carefully thought/tested before doing so. For example, even with your approach, if the junior forgets to lock on the masterLock, then it's still possible to break the application. If, doing so, he also sets the cookie to null or if you don't have a timeout, then again, it's a very big problem.


Yes, junior programmers can mess up complicated code quite easily.

I added implementation comments defining locking requirements: locking order (master/record), must hold master to lock record, modify map, check waiting threads... That plus cleaning up code/design so its easier to understand is how I hope to satisfy SUN's junior programmer requirement.

- Karen Smrha
 
Don't get me started about those stupid light bulbs.
 
subject: [locking] Hand over hand, remove() ?
 
Similar Threads
Multiple Notification Points
Passed java developer certification
Locking, about sentence: "consuming no CPU cycles"
Test your business service
Hand-over-hand Locking.