Meaningless Drivel is fun!*
The moose likes Developer Certification (SCJD/OCMJD) and the fly likes Locking policies and general architecture doubts. Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Java 8 in Action this week in the Java 8 forum!
JavaRanch » Java Forums » Certification » Developer Certification (SCJD/OCMJD)
Bookmark "Locking policies and general architecture doubts." Watch "Locking policies and general architecture doubts." New topic
Author

Locking policies and general architecture doubts.

Marcin Mroczkowski
Greenhorn

Joined: Apr 15, 2013
Posts: 17

I have URLyBird 1.3.3, some informations about it:
-three-tiered architecture with Data.class layer, business service layer, and gui layer
-Data.class implements provided Sun's interface, service class implements service interface, which is used to communicate with GUI
-used RMI for communication, buisness layer rest on server, and only business service interface is exported on RMI
-i've avoided whole RMI-threading, and client cookie issues because all database trasactions are done completelly server-side in service layer
-Map of ReetrantLocks is used to keep track of record locks, RMI thread executing business action is holder of locks
-used optimistic locking pattern in GUI, so user don't care about any locking in gui, if overlapping edit occurs, client will be promted to refresh data and try edit again
-i've used most recent Java7 jdk and lot of Java7 only goodies (including NIO:SeekableByteChannel for file operations, autoclosing try-catch,diamond syntax etc.)

I'm not sure if my architecture meets specification completely especially:
-optimistic lock pattern in gui
-java7 tricks
-DBMain not exposed by RMI, usage of other, business-centered interface, which cuts tons of implementation problems

Also some doubts around locking arised in my mind:

1. Do i need to create deadlock-safe Data class even when interface is abused ?
From project specification, Data class should be deadlock-proof, and it will be tested for this. When multiple clients try to lock multiple locks, they will create deadlock very quickly. In my opinion we can classify situation like this as wrong interface usage and reward any client who tried to lock second record with RuntimeException (i think IllegalMonitorStateException).

2. Can i make record locking a must for operations on records ?
Specification do not state that client MUST get lock prior edit but i think it's pretty obvious. Editing without lock could create indeterminate results, without atomicity client can't determine state of record prior edit. So, we can do two things about it:
-automatically get lock for record inside Data class implementation and let client do blind edit
-send RuntimeException to client informig, that lock must be acquired
I've decided to send Exception because it's creating very nice fast-fail mechanism for future developing on this interface.

I've tried to find answers here and in other places with no success. Some topics were coverted on forum but were not completelly accurate.

BTW. Sorry for my english : )

OCJP 6, OCMJD 6
Roel De Nijs
Bartender

Joined: Jul 19, 2004
Posts: 4925
    
  10

I don't see any problems at all. Everything sounds just fine and I don't see any violation of specification.

Regarding your questions:
1/ No, that's not needed. You can assume correct API usage and/or force correct API usage by throwing exception (but that's not possible in every assignment, it depends on your API)

2/ I wouldn't do that at all, because think about the design principle "high cohesion": each method should have 1 purpose. So the update method should only be responsible for updating, not for locking.


SCJA, SCJP (1.4 | 5.0 | 6.0), SCJD
http://www.javaroe.be/
Marcin Mroczkowski
Greenhorn

Joined: Apr 15, 2013
Posts: 17

Big thanks Maybe i'm just too meticolous about it.
Marcin Mroczkowski
Greenhorn

Joined: Apr 15, 2013
Posts: 17

Ok, i've just fully finished my project. Now it's time for paranoidal hunt for possible flaws. Im not sure about:

- i've created very strong abstraction (service locator pattern) between GUI and database/RMI code, this way GUI is absolutely clueless about underlying service implementation, it's sweet from OOD point of view but has downsides, i wonder if GUI should inform user which mode is used and/or require only mode-specific settings on settings page, i think standalone-only user will not be happy when he see unusefull remote address setting...

- i've decided to totally drop "Book" button for starting book process, in my opinion it is completelly redundant and only increase gui clutter, "double-click" or "arrow select and enter" combination, are both more natural, faster and satisfies both mouse and keyboard users, i wonder if it's acceptable (of course is throughtly documented in userguide and tooltips)

- i've created naive domain model implementation, which utilizes separate package for domain object and it's factory, package constructor and package fields are used to allow factory to create object at will without setters, this way domain object is hermetic for all other classes than factory and itself, i'm not sure if it's nice from OOD perspective

- i'm not sure about DBMain.isLocked() implementation, my Data.class uses ReetrantLock to deal with record locks, method ReetrantLock.isLocked() should do the job but it isn't guarantee to be fully synchronized with lock, so it can violate DBMain contract, because it's theoritically possible that it will return bad status just after locking/unlocking, to solve this i've created pretty complex custom implementation using ReetrantLock.tryLock() method, im not sure if i should trade code simplicity for this

- userguide.txt verbosity is not specified, so i wonder what should be discussed there, especially should i explain overall technological standards, for example how focus tabbing works and how to use it on certain components, or how to use drop-down list using keyboard

- should i allow user to clear/rebook reservation or only allow booking of free reservation

- did someone lost points for not implementing OR logic for "and/or" dillema included in specification("It must allow the user to search the data for all records, or for records where the name and/or location fields exactly match values specified by the user."), im interpreted and discussed this sentence in choices.txt and decided to implement only AND logic

Sorry if any of this questions is somewhere answered i tried to find concrete answers before posting.
Roel De Nijs
Bartender

Joined: Jul 19, 2004
Posts: 4925
    
  10

1/ That's contradictory with your next point (totally drop "book" button). You drop a button, because it increases gui clutter, but you do show config settings which are useless to the user. My config settings dialog only shows the mode-specific settings.

2/ I don't know why it would not be acceptable. It's a decision you make. My gui contains a book-button, double-clicking on a record in the table will display its details.

3/ That's encapsulation on class & package level, so that's a good thing in OOD and decreases the chance of misusing the api (classes & packages). Another good thing.

4/ I would suggest KISS. I always tried to keep simplicity in mind. Maybe you can go for the simple option and make a note about it in your choices document.

5/ The user guide is a document for someone to work with your application (how can i search records?, how can I book?,...). It's not a manual for learning to work with a computer.

6/ Is it a must requirement? I guess rebook/clear is not (unless instructions has changed), so why bother? You won't get extra credits for implementing, but might lose some points because of a bug.

7/ My application supports 4 possible scenarios: (a) all records (b) name (c) location (d) name and location. I didn't implement "name or location"


Hope it helps!
Marcin Mroczkowski
Greenhorn

Joined: Apr 15, 2013
Posts: 17

Thank you, you're helping a LOT as always, especially about "and/or" ambiguity, now i'm fully comfortable with my desing choice.

And about:

1.) You're right, it's contradictory because im aiming to simplify the gui, but as i said abstraction beteen GUI and Service Locator is so strong, that gui just don't know if service is remote or standalone. To show mode-specific settings i must relax this abstraction a bit to smuggle information about running mode. I must trade, gui simplicity vs pure service abstaction.

4.) Im not sure, i tested this behavior and if i use ReetrantLock.isLocked() i can get false just after locking. This is bad, i know that simplicity is a king, bur correctness is even bigger. I just don't want to be caught by stupid software test, which prove that my isLocked implementation don't work as specified in DBMain.

6.) It's not specified, so i added business context to determine this dillema. And from user point of view ability to correct mistakes is really usefull, but if you're implemented only booking of free rooms and do not lost any points, i'm conviced
Roel De Nijs
Bartender

Joined: Jul 19, 2004
Posts: 4925
    
  10

1/ Having loose coupling between gui and service (locator) is of course a good thing from an OOD perspective. But my controller also uses an interface to perform the actions (unaware of the connection type). And the initialisation of this interface happens in the controller constructor (which makes it less loosely coupled). But I could easily extract this code to a service locator class which sole purpose is to return the appropriate interface implementation based on the connection type (direct or network). But both scenarios will have the same config settings dialog (with mode-specific settings).

4/ My implementation of isLocked is fairly simple: just check if the record number is in the map with locked records. I didn't use new concurrency api, but the old-school synchronized, wait and notify(All).

6/ For the assignment 2 things are key: simplicity rules and just stick to the requirements (extra features can only make you lose points due to bugs/mistakes). Excellent decision!
Marcin Mroczkowski
Greenhorn

Joined: Apr 15, 2013
Posts: 17

1.) One way or another, it's needed to relax this abstraction. One simple method on service locator interface (isNetworked()) should do the job, gui will now know if service uses networking to operate.

4.) isLocked() on ReentrantLock is just complete ********, it's usage in DBMain could lead to contract violation and ill not take this risk. Complex solution should be avoided, but ill not trade it for correctness.

6.) Whats more, even if extra functionality is done correct there is a room for significant point loss. Exam assessor can point that this extra things make application bigger and harder to use and cut some points anyway. Because of this i mercilessly stripped lot of things from my initial version.

Ok now i know what i got to do. Thanks again, i hope my next post will be on new thread and inform about exam pass
Roel De Nijs
Bartender

Joined: Jul 19, 2004
Posts: 4925
    
  10

Marcin Mroczkowski wrote:Ok now i know what i got to do. Thanks again, i hope my next post will be on new thread and inform about exam pass

Good luck! I'm confident you'll be a OCMJD soon...
Marcin Mroczkowski
Greenhorn

Joined: Apr 15, 2013
Posts: 17

Yesterday i've detected subtle unlocking bug, which could cost me a lot of points, so i've decided to delay submission once more and do some very scrupulous testing of both Data.class and business interface. I did multi-variant tests for all methods, and got stuck on lock durability test. This test should get multiple locks, and then create group of threads, which try to lock the same locks again. Test should check if all of those threads got blocked properly, and then release locks and wait for them to die. To make things simpler let's analyze scenario with only single lock.



It's simple and does work, but it's not completely correct. First of all it depends on sleep, so it is theoreticaly possibile, that second thread will not run for sleep period. Second problem is execution time, 100ms is rather long so this test is very, very slow when runs in a loop. I'm not satisfied with this implementation and i've tried to make it better, with no success. Any suggesions, how it can be synchronized in more correct way ?
Roel De Nijs
Bartender

Joined: Jul 19, 2004
Posts: 4925
    
  10

For my Data class I used 2 tests: one I created myself which is a test case to verify business logic and all my assumptions. For the 2nd one I just used Roberto's Data class test for verifying it's thread-safe. Exactly the same for the business service: again 1 test case for business logic and assumptions, another one for thread-safety. Both these tests I created myself and the 2nd one (thread-safety) I shared in ScjdFaq. Maybe you could try that one too.
Marcin Mroczkowski
Greenhorn

Joined: Apr 15, 2013
Posts: 17

Roberto's test is great to maximize concurrency and provoke hidden deadlock to appear, but it's unable to detect, if records are properly blocked. Non-multithreaded unit tests, are also unable to detect possible flawed lock method.
One of the major requirments for Data.class is implementation of lock method, which will block any other attempts to lock the same record again, so i think this one should be tested hard.

I have new idea to use thread states to synchronize locking threads. Main thread can take lock, start new thread which should block on same record, and then probe new thread and wait for it to get to blocked status. This implementation will not be ideal but it's surely better than using sleep.
Roel De Nijs
Bartender

Joined: Jul 19, 2004
Posts: 4925
    
  10

Marcin Mroczkowski wrote:One of the major requirments for Data.class is implementation of lock method, which will block any other attempts to lock the same record again, so i think this one should be tested hard.

Do I understand correctly if I state you want to create a test case which verifies that if a record is locked by a threadA it can't be locked by any other thread?

If that's the case I would try something like this test case: create a list with e.g. 10 threads. All these threads do the same thing: lock record 1, do some assertions and finally unlock record 1. If your lock-method works as expected 1 of these threads should own the lock (it's state should be RUNNABLE), all other threads should be in BLOCKED state. That's easily to verify through Thread's getState()-method. When unlock of record 1 is called by the thread that has locked record 1, all other threads will compete again to lock record 1. And then you verify the assumptions again. This process will repeat itself until all records have locked and unlocked record 1
Marcin Mroczkowski
Greenhorn

Joined: Apr 15, 2013
Posts: 17

Roel De Nijs wrote:
Marcin Mroczkowski wrote:One of the major requirments for Data.class is implementation of lock method, which will block any other attempts to lock the same record again, so i think this one should be tested hard.

Do I understand correctly if I state you want to create a test case which verifies that if a record is locked by a threadA it can't be locked by any other thread?


Yes that's exacly what i want to test. This problem can be very tricky because any of normal tests will not detect it (including Roberto's test case, unit tests, business interface tests, user-side apllication tests), it's MUST requirement, and if threre is even slight chance for two threads to get same lock, the project can be failed on the spot.

Roel De Nijs wrote:If that's the case I would try something like this test case: create a list with e.g. 10 threads. All these threads do the same thing: lock record 1, do some assertions and finally unlock record 1. If your lock-method works as expected 1 of these threads should own the lock (it's state should be RUNNABLE), all other threads should be in BLOCKED state. That's easily to verify through Thread's getState()-method. When unlock of record 1 is called by the thread that has locked record 1, all other threads will compete again to lock record 1. And then you verify the assumptions again. This process will repeat itself until all records have locked and unlocked record 1


Your test algorithm seems good, but after carefull reading of JAVA API about method Thread.getState():

"This method is designed for use in monitoring of the system state, not for synchronization control."

This means, that thread states are not exact in any moment, they can be little off during transitions. To constrast, synchronized blocks, wait, notify always produce 100% exact results, Thread.getState() does not. Because of this it's better to avoid depending on it and create safety window for threads transintions.

To overcome this problem i created another algorithm:
- main thread locks random records, and saves their numbers in common synchronized set
- main thread starts threads for each record in file, exacly one per record
- started threads tries to lock their record, and then check if set contain their record number, if yes, test is failed because, this means that main thread also have this lock, if no, thread just dies
- main thread operate in infinite loop and checks threads which should block on lock, if it spot thread in "WAITING" state (ReetrantLock.lock() produce waiting threads) then it's removes it's number from set, and unlocks this record
- main thread breaks loop if it remove last number from set and unlock it's last locked record, and then waits for all remaining threads (.join on all threads)

By using infinite loop, the getState() problem is bypassed, because thread transition delay will not affect main thread behavior. It's worth nothing that this test also checks if it's possible for Data.lock method to block if lock is free. If this is the case, this test deadlock on one of started threads...

Now this test do what it should do Thanks again for you help. If You have any further comments about this case feel free to post, any suggestions are very appreciated.
Roel De Nijs
Bartender

Joined: Jul 19, 2004
Posts: 4925
    
  10

Marcin Mroczkowski wrote:Yes that's exacly what i want to test. This problem can be very tricky because any of normal tests will not detect it (including Roberto's test case, unit tests, business interface tests, user-side apllication tests), it's MUST requirement, and if threre is even slight chance for two threads to get same lock, the project can be failed on the spot.

It can be tricky to test some multi-threaded issues. Because of my simple design, implementation and straightforward business logic I did not needed such a test. I could easily determine by using a bit of logic thinking that it's impossible for 2 threads to get same lock. With a more complex situation I would have created a similar test.


Marcin Mroczkowski wrote:Now this test do what it should do

If you are satisfied with your test (and it's not closely tied with your Data class and assumptions you made), you could maybe (just like Roberto and me) share your test with the community (and add a link to ScjdFaq), so others could run this test to verify their lock-method is spot-on.
Marcin Mroczkowski
Greenhorn

Joined: Apr 15, 2013
Posts: 17

Roel De Nijs wrote:It can be tricky to test some multi-threaded issues. Because of my simple design, implementation and straightforward business logic I did not needed such a test. I could easily determine by using a bit of logic thinking that it's impossible for 2 threads to get same lock. With a more complex situation I would have created a similar test.


I was thinking in exacly same manner, did only simple unit tests, Roberto's deadlock test, and was very happy and confident about my implementation. And then, completely by accident (when beautifying javadoc) i've spotted very ugly unlock bug, which could cost me lot of points. From now on i do tests to cover anything that can possibly break, covering all aspects of interface contract, and then repeat it 5000 times on high load. Better safe than sorry

Roel De Nijs wrote:If you are satisfied with your test (and it's not closely tied with your Data class and assumptions you made), you could maybe (just like Roberto and me) share your test with the community (and add a link to ScjdFaq), so others could run this test to verify their lock-method is spot-on.


Good idea, ill try to reformat, comment and post it, but before i need to make sure that it's working as desired (in simple words i must test this test ).
Marcin Mroczkowski
Greenhorn

Joined: Apr 15, 2013
Posts: 17

I encountered very annoying issue during JavaDoc creation (jdk1.7.0_17). I was using {@inheritDoc} happily and reused some java API comments. Works like a charm in IDE, but javadoc tool did not copied those comments, leaving them blank without any warning. After some goggling i tried to modify my sourcepath (i'm using roel's ant script) and enable verbose mode:
- when i added jdk libraries, javadoc scanned them but completely ignored comments and left them blanks as always
- when i added openjdk sources, javadoc detects lot of errors in sources and terminates
- corrected "-link" attribute to valid java 7 API, no results on comments

I was really surprised that such detail can waste so much time and leave me completely clueless. I can't hard-copy this comments, because it will be sign of wrong javadoc usage, I must write them because it's one of MUSTS, so how you guys got it done?
Roel De Nijs
Bartender

Joined: Jul 19, 2004
Posts: 4925
    
  10

It should not be really surprising, but I used my ant script without any of the problems you described. I had a look at the javadoc tool documentation and it clearly states supporting {@inheritDoc} when you want to inherit comments. Very strange!
Marcin Mroczkowski
Greenhorn

Joined: Apr 15, 2013
Posts: 17

Fix found in: http://stackoverflow.com/questions/12334460/javadoc-not-generating-comments-for-inherited-methods

And the key sentence form javadoc documentatnion:"The source file for the inherited method need only be on the path specified by -sourcepath for the doc comment to actually be available to copy".

I think Java core source comments should be exception(and be copied regardless of sourcepath configuration), but i don't know why sometimes it doesn't work. When i unpacked src.zip from jdk and added it to sourcepath, javadoc at last copied comments properly.
Roel De Nijs
Bartender

Joined: Jul 19, 2004
Posts: 4925
    
  10

Glad to hear you were able to solve your problem. Well done!
Marcin Mroczkowski
Greenhorn

Joined: Apr 15, 2013
Posts: 17

It's done !

http://www.coderanch.com/t/615209/sr/certification/Passed-OCMJD
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Locking policies and general architecture doubts.
 
Similar Threads
Extending DB interface - doubts
Doubts reagarding database operations for Bodgitt and Scarper
lock method without lock cookie?
passed SCJD with 360/400
Bunch of questions - near the end...