It's not a secret anymore!*
The moose likes Developer Certification (SCJD/OCMJD) and the fly likes DataClassTest: magic cookie not working afterwards + RNFE Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


JavaRanch » Java Forums » Certification » Developer Certification (SCJD/OCMJD)
Bookmark "DataClassTest: magic cookie not working afterwards + RNFE " Watch "DataClassTest: magic cookie not working afterwards + RNFE " New topic
Author

DataClassTest: magic cookie not working afterwards + RNFE

Norbert Lebenthal
Ranch Hand

Joined: Sep 23, 2010
Posts: 74
hi

I'm using the DataClassTest available here http://www.coderanch.com/t/427863/java-developer-SCJD/certification/Tests-Data-class-locking-mechanism (Thanks Roberto !)

It runs in less than one second but a few points puzzled me:
- the console doesn't stop => I've to stop it myself
- when I try to run it a second time on the same file, it fails due to the magic cookie not being the one expected
- some RecordNotFoundException are thrown: it looks like the test class is trying to update records which aren't present yet...

Are these stuff normal ?

I spent quite a while tonight trying to figure out what was wrong, writing some more tests on the way, but I didn't manage to boil it down...

thanks in advance
best
norbert
Norbert Lebenthal
Ranch Hand

Joined: Sep 23, 2010
Posts: 74
I tried to upload my trace log but I didn't succeed... As it's quite large, I kind of hesitate to put it here

do you think you need it ? if so, how could I provide it ?
Roel De Nijs
Bartender

Joined: Jul 19, 2004
Posts: 5300
    
  13

Hi Norbert,

If you have to stop the console yourself, you have a deadlock and that's not something you want So you certainly have to handle that issue.

That the magic cookie is wrong the second time, is also quiet weird and unexpected.

That some RecordNotFoundExceptions are thrown is expected behavior, because the program is nothing more than some deletes/updates of record 1. So if the update is invoked after record 1 is deleted a RNFE is what you get ;)

Kind regards,
Roel


SCJA, SCJP (1.4 | 5.0 | 6.0), SCJD
http://www.javaroe.be/
Piotr Nowicki
Ranch Hand

Joined: Jul 13, 2010
Posts: 610

Howdy Norbert,

Just thinking - is it possible that after the deadlock (during some create or update operations) your file is unapropriately changed? Did you test the file operations with your unit tests? Maybe you should try viewing the database file (after the first run) in some hexadecimal editor, to check the actual value of the magic cookie (and overall file structure) if it remained unchanged.

BTW: when I was trying to find the deadlock, some of the debugger's features (like conditional breakpoints or break when exception occurs) was quite useful.

Cheers!


OCP Java SE 6 Programmer, OCM Java SE 6 Developer, OCE Java EE 6 JSPSD, OCE Java EE 6 EJBD, OCE Java EE 6 JPAD, Spring 3.0 Core Professional.
Roberto Perillo
Bartender

Joined: Dec 28, 2007
Posts: 2265
    
    3

Howdy y'all.

Pedro Kowalski wrote:Just thinking - is it possible that after the deadlock (during some create or update operations) your file is unapropriately changed?


Well, that's what I suspect too. Maybe it is because the tests also use the create() method of the Data class being tested. Maybe it is either writing directly to the file wrongly or, if it uses a record cache, then the records are being written wrongly (what doesn't make much sense, beucase it never exits correctly). And I also think that Roel's conclusion is correct: if you have to stop the process manually (in an IDE, pressing stop, or hitting Ctrl + C on the console), then you have a deadlock.


Cheers, Bob "John Lennon" Perillo
SCJP, SCWCD, SCJD, SCBCD - Daileon: A Tool for Enabling Domain Annotations
Norbert Lebenthal
Ranch Hand

Joined: Sep 23, 2010
Posts: 74
hi

thanks all for your help: it helped me much in knowing in which direction to go !

I found the error regarding the magic cookie/file no more readable issue: I've a freeLocations ConcurrentLinkedQueue and instead of putting the location in it I was putting the record number...

Regarding the deadlock, well, it's still there :$

In fact my lockManager has the following structure:


with RecordLock being:


as you might guess, I go first at the masterLock, grab (creating if needed) the record lock, lock on it, release the master, do my stuff, release the lock on the record lock.

the issue is the following: I've an Exception being thrown when trying to update some record. This leads rightly to the corresponding unlock BUT the threads waiting on the condition for this record aren't fired...

My code for the waiting bit is the following:

Edit: unfortunately, we can't post the code of the locking mechanism here.
Roberto Perillo
Bartender

Joined: Dec 28, 2007
Posts: 2265
    
    3

Howdy, Norbert.

Here's something that might help you: you are not adding the locked record to the map of locked records after exiting the while loop. Unfortunately, I had to edit your post as we can't post the code of the locking mechanism here.
Norbert Lebenthal
Ranch Hand

Joined: Sep 23, 2010
Posts: 74
hi Roberto

thanks for the hint. However, in the end, the issue was different and with 2 aspects:
- the DataClassTest: when it grabs the lock and then try to do something (like delete/update), this action isn't in a try{}finally{} block to ensure unlock happens. => this is one of the issue I faced.
Shouldn't this DataClassTest be updated to apply this good pratice (and avoid risks of thinking of having deadlock when in fact the test class is simply not releasing its lock) ?
- I forgot to, hum, signal to my waiters that they could check again for the condition... So they didn't feel like having to check again and then were waiting forever.

It's all fixed now.

Thanks again for this test, it really helped me spot hidden bugs (which my own unit tests haven't spotted, shame on me)

++
Roel De Nijs
Bartender

Joined: Jul 19, 2004
Posts: 5300
    
  13

Norbert Lebenthal wrote:- the DataClassTest: when it grabs the lock and then try to do something (like delete/update), this action isn't in a try{}finally{} block to ensure unlock happens. => this is one of the issue I faced.

It all depends on how your methods behave. My test code is similar to the one in Roberto's test class and my test runs without deadlocking. Don't forget it's just a tool (freely shared to the java community) and you are supposed to adjust the code to fit your application.

3 possible scenarios:
1/ The lock method throws a RecordNotFoundException, because the record does not exist and it can't be locked, so a call to unlock is not needed.
2/ The lock method can successfully lock record because it exists and is not locked by another thread, which means the record can be updated/deleted successfully (and no RNFE should be thrown here because it was successfully locked).
3/ The lock method is called, the thread is put to waiting state because the record is already locked. When the thread is notified and back running, it has to check first if the record still exists (because it could be deleted by the other thread) before trying to lock the record. And then it's back to scenario 1 or 2
Norbert Lebenthal
Ranch Hand

Joined: Sep 23, 2010
Posts: 74
hi Roel

thanks a lot for your answer !

in fact, due to my IDE code generation tools, I had completely missed in the class implementing the lock method that it should throw RecordNotFoundException...

I'm now well aware of this, and I went for a simpler implementation of the lock mechanism: a simple map with guarding reentrantlock and condition. As I cache my records, checking for existence in the lockManager is fast. on top of that, it should be a mostly read intensive application. And even if not, I guess there'll be other issues before that this simple lock strategy causes trouble lol

BTW, when releasing the lock, I do a signalAll and not a simple signal. Indeed, when using only signal I think the awaken thread could be waiting on some other lock, and then return to sleep immediately, potentially stopping a thread waiting on the same lock to start. Am I right here ?

thanks again !
nono
Roel De Nijs
Bartender

Joined: Jul 19, 2004
Posts: 5300
    
  13

Norbert Lebenthal wrote:Am I right here ?

Yes!
 
GeeCON Prague 2014
 
subject: DataClassTest: magic cookie not working afterwards + RNFE