Locking & Synchronization on Bodgitt & Scarper

[Harry Henriques]

Hi Fellow Bloggers,

I'm running on Eclipse. I modified Roberto Perillo's DataClassTest in order to test my own Data class. I don't read the database file into RAM, nor operate upon it as a collection of objects in RAM memory. I read data from the hard disk, and I write data to the hard disk. Of course, I had problems with concurrency and IOExceptions, but I think I have those ironned out.


Will you take a look at my output (below) and let me know if you see any errors? Thanks Harry

The first field in the trailing post to this entry is the Thread Id.
The second field in the trailing post to this entry is the Thread Name (which is self documenting).

I ran a 5 iteration loop that creates the following Threads sequentially.

    for (int i=0; i < 5; i++ ) {
               Thread t1 = new RandomRecordUpdateThread();
                t1.setName("RandomRecordUpdateThread");
                t1.start();               
                Thread t2 = new RecordOneDeletionThread();
                t2.setName("RecordOneDeletionThread");
                t2.start();
                Thread t3 = new RecordCreationThread();
                t3.setName("RecordOneCreationThread");
                t3.start();
                Thread t4 = new RecordOneUpdateThread();
                t4.setName("RecordOneUpdateThread");
                t4.start();
                Thread t5 = new FindingRecordsThread();
                t5.setName("FindingRecordsThread");
                t5.start();
                Thread t6 = new RecordCreationThread();
                t6.setName("RecordTailendCreationThread");
                t6.start(); 
      }

[Roel De Nijs]

Hi Harry,

If you deadlock for example your program never ends, so that's something you can spot without any output. If you can increment the loop counter to 1000 or 10000 and you don't deadlock, I think your locking is working.

I'm missing something in your output: I don't see any RecordNotFoundException being caught or do you just not log it? If threadA deletes the record and afterwards threadB tries to lock the record I would expect to see a RNFE.

Kind regards,
Roel

[Harry Henriques]

Thanks Roel,

In my simulation, the only record that is deleted is record #1. I should probably expand my testing to include random deletes. My Data class implementation will update/create a record in the slot left empty by a deleted record in the database file. I don't try to detect a deleted record in my lockRecord() method. There really isn't any reason to. The lockRecord() method is only used on create/update/delete (nowhere else). The findByCriteria() method skips over deleted records.

There is a performance hit associated with this implementation. If the lockRecord() method allows a thread to proceed with execution and the associated record is deleted, then a subsequent deleteRecord() is a waste of CPU cycles. A deleted record can't be deleted a second time (even though it can be in my implementation, it is redundant).

Best regards,

Harry

[Roel De Nijs]

Hi Harry,

I have a few remarks.

What about this scenario?:
threadA.lock(1); // gets lock
threadB.lock(1); // has to wait
threadA.delete(1); // record 1 is deleted
threadA.unlock(1); // notify waiting threads
threadB gets notified and then I would expect a RNFE being thrown.

in my interface lock/unlock methods are only used for deleting and updating a record, not for creating one.

Kind regards,
Roel

[Harry Henriques]

Hi Roel,

I don't see why it is necessary to check that a record has been deleted before adding the record number to the HashMap.

Best regards,

Harry

[Roel De Nijs]

Hi Harry,

I think there are 2 major issues with your scenario:

1) So first clientA deletes record 1 and then clientC just updates the deleted record How can you update a record that doesn't exist anymore? After record 1 is deleted that record is gone (although it still exists in your flat file), doesn't exist anymore! And that's why your lock-method should throw a RNFE, because the record has gone, doesn't exist anymore. If you would have a RDBMS (instead of the flat file) clientA would do a "delete from table_rooms where id = 1" and then clientC would do a "update table_rooms set ... where id = 1", which results in no rows being affected (because it is already deleted) and clientC should do a insert/create instead of an update.

2) because you use "cookie" in your scenario I guess you have an interface which uses a lockCookie in the method signature of delete/update/unlock and the lock method returns an int or a long. And according to your scenario you don't use it correctly. Correct (and intended) usage should be:
long cookie = clientA.lock(1);
clientA.update(1, newData, cookie);
clientA.unlock(1, cookie);

in the lock-method: here you check if the record is already locked, if not you generate a cookie. this cookie is added to the hashmap<recNo, cookie> and is used as return value
in the update-method: check to see if record is locked and if the lockCookie in your hashmap corresponds to the cookie passed as an argument. if one of these checks fail, throw a SecurityException because another client tries to update (or delete) the record. if everything is ok update the record.
in the unlock-method: check to see if record is locked and if the lockCookie in your hashmap corresponds to the cookie passed as an argument. if one of these checks fail, throw a SecurityException because another client tries to unlock the record. if everything is ok remove the entry from your hashmap and notify other waiting threads.

Kind regards,
Roel

[Harry Henriques]

Hi Roel,

You make two good points. Let me address the second point first. I can't show you the guts of my lockRecord() method (because it is a major assignment deliverable). I think that my example in the previous post is confusing, concerning my implementation. Exactly as you indicate, I do the following:

long lockCookie = data.lockRecord(recNo);
data.deleteRecord(recNo, lockCookie);
data.unlock(recNo, lockCookie);

Roel de Nijs wrote:
check to see if record is locked and if the lockCookie in your hashmap corresponds to the cookie passed as an argument. if one of these checks fail, throw a SecurityException because another client tries to update (or delete) the record.

You say above that there are two checks that need to be made in the update/delete methods. I make both of these checks on my HashMap<recNo, cookie>. First, I check to see if the record is locked by checking to see if the record is in the HashMap. Second, I use the recNo key to get the savedCookie in the key-value pair. I compare the savedCookie to the lockCookie passed into the method as an argument. If they are NOT the same, I throw a Security Exception. I think that I have the second one of your points covered.

About the first point, the specification doesn't instruct me in how to form the String[] data that is passed into the updateRecord() method. I made the assumption that all fields had to contain valid data (no NULLs), and that a complete overwrite was made when the updateRecord() method is called on a recNo. I made the assumption that if you were going to update a record that the first thing you would do is call readRecord(recNo) to get the original data. My readRecord() method throws a RNFE exception if an attempt is made to read a deleted record. After getting the original data using readRecord(), the data would be modified in the client, and the whole record would be overwritten.

You are correct. I overlooked an important consideration. A record that is still in the flatfile database that has been marked invalid has garbage data. You wouldn't want to read that record, and use the data to populate a new modified record. You would want to throw a RNFE exception, if you try to update a deleted record. Thanks

Best regards,
Harry

[Harry Henriques]

Hi Fellow Bloggers,

I just read the quote below in my SCJD design specification. I guess I have to make some changes in my implementation.

SCJD Assignment wrote:
Any methods that throw RecordNotFoundException should do so if a specified record does not exist or is marked as deleted in the database file.

Regards,

Harry

[Harry Henriques]

Hi Fellow Bloggers,

I'm not sure if this is of interest, but after correcting a few problems associated with deleted records, this is the output of 5 iterations of DataClassTest.java. Enjoy munching!

The first numeric field is the Thread Id.
The second varchar field is the Thread Name.
In most cases, the record that is being operated upon is identified.

Harry Henriques wrote:[Edited 2:45PM on 12/31/09]

I introduced a whole bushel of bugs during refactoring. I think that I've fixed them at this point.

I have overwritten the original data in this post with new data acquired after making some changes to my application. There was a problem with my application. The createRecord() method was NOT overwriting deleted record entries in the flatfile database. My createRecord() method doesn't use the lockCookie. I don't think that my application manages well when two threads are both invoking the createRecord() method. It would be better controlled if the application used the lockRecord() and unlock() methods.

Harry Henriques wrote:[Edited 9:16PM on 12/31/09]
Thank you, Roel De Nijs for keeping me honest. I think that locking is working as it is expected to, now. I have overwritten the updated test output with new test output from 9:16PM. I think this test output is clean. Please let me know if you see anything wrong.

8 RandomRecordUpdateThread locking record #29
8 RandomRecordUpdateThread updating record #29
8 RandomRecordUpdateThread unlocking record #29
9 RecordOneDeletionThread locking record #1
9 RecordOneDeletionThread deleting record #1
9 RecordOneDeletionThread unlocking record #1
10 RecordCreationThread creating record #1
11 RecordOneUpdateThread locking record #1
11 RecordOneUpdateThread updating record #1
11 RecordOneUpdateThread unlocking record #1
12 FindingRecordsThread searching for matching records.
12 4 results found. 4 to go.
12 FindingRecordsThread reading record #2
12 2 Dogs With Tools Smallville
12 4 results found. 3 to go.
12 FindingRecordsThread reading record #24
12 24 Dogs With Tools Xanadu
12 4 results found. 2 to go.
12 FindingRecordsThread reading record #30
MAIN THREAD TERMINATING.
182 RandomRecordDeletionThread locking record #14
12 30 Dogs With Tools Hobbiton
12 4 results found. 1 to go.
182 RandomRecordDeletionThread deleting record #14
181 RecordCreationThread creating record #14
180 FindingRecordsThread searching for matching records.
180 4 results found. 4 to go.
180 FindingRecordsThread reading record #2
180 2 Dogs With Tools Smallville
180 4 results found. 3 to go.
180 FindingRecordsThread reading record #24
180 24 Dogs With Tools Xanadu
180 4 results found. 2 to go.
180 FindingRecordsThread reading record #30
179 RecordOneUpdateThread locking record #1
179 RecordOneUpdateThread updating record #1
180 30 Dogs With Tools Hobbiton
180 4 results found. 1 to go.
178 RecordCreationThread creating record #34
176 RandomRecordUpdateThread locking record #18
176 RandomRecordUpdateThread updating record #18
175 RandomRecordDeletionThread locking record #2
175 RandomRecordDeletionThread deleting record #2
174 RecordCreationThread creating record #2
173 FindingRecordsThread searching for matching records.
173 4 results found. 4 to go.
173 FindingRecordsThread reading record #2
173 2 Dogs With Tools Smallville
173 4 results found. 3 to go.
173 FindingRecordsThread reading record #24
171 RecordCreationThread creating record #35
169 RandomRecordUpdateThread suncertify.db.RecordNotFoundException
isDeleted(): record #49 java.io.EOFException
168 RandomRecordDeletionThread suncertify.db.RecordNotFoundException
isDeleted(): record #50 java.io.EOFException
173 24 Dogs With Tools Xanadu
173 4 results found. 2 to go.
167 RecordCreationThread creating record #36
166 FindingRecordsThread searching for matching records.
166 4 results found. 4 to go.
166 FindingRecordsThread reading record #2
166 2 Dogs With Tools Smallville
166 4 results found. 3 to go.
166 FindingRecordsThread reading record #24
166 24 Dogs With Tools Xanadu
166 4 results found. 2 to go.
166 FindingRecordsThread reading record #30
164 RecordCreationThread creating record #37
162 RandomRecordUpdateThread locking record #22
162 RandomRecordUpdateThread updating record #22
166 30 Dogs With Tools Hobbiton
166 4 results found. 1 to go.
161 RandomRecordDeletionThread suncertify.db.RecordNotFoundException
isDeleted(): record #39 java.io.EOFException
160 RecordCreationThread creating record #38
159 FindingRecordsThread searching for matching records.
159 4 results found. 4 to go.
159 FindingRecordsThread reading record #2
159 2 Dogs With Tools Smallville
159 4 results found. 3 to go.
159 FindingRecordsThread reading record #24
157 RecordCreationThread creating record #39
155 RandomRecordUpdateThread locking record #31
154 RandomRecordDeletionThread locking record #3
154 RandomRecordDeletionThread deleting record #3
159 24 Dogs With Tools Xanadu
159 4 results found. 2 to go.
153 RecordCreationThread creating record #3
152 FindingRecordsThread searching for matching records.
152 4 results found. 4 to go.
152 FindingRecordsThread reading record #2
152 2 Dogs With Tools Smallville
152 4 results found. 3 to go.
152 FindingRecordsThread reading record #24
150 RecordCreationThread creating record #40
148 RandomRecordUpdateThread locking record #33
148 RandomRecordUpdateThread updating record #33
152 24 Dogs With Tools Xanadu
152 4 results found. 2 to go.
146 RecordCreationThread creating record #41
145 FindingRecordsThread searching for matching records.
145 4 results found. 4 to go.
145 FindingRecordsThread reading record #2
145 2 Dogs With Tools Smallville
145 4 results found. 3 to go.
145 FindingRecordsThread reading record #24
145 24 Dogs With Tools Xanadu
143 RecordCreationThread creating record #42
145 4 results found. 2 to go.
141 RandomRecordUpdateThread locking record #34
141 RandomRecordUpdateThread updating record #34
139 RecordCreationThread creating record #43
138 FindingRecordsThread searching for matching records.
138 4 results found. 4 to go.
138 FindingRecordsThread reading record #2
138 2 Dogs With Tools Smallville
138 4 results found. 3 to go.
138 FindingRecordsThread reading record #24
138 24 Dogs With Tools Xanadu
138 4 results found. 2 to go.
136 RecordCreationThread creating record #44
133 RandomRecordDeletionThread locking record #23
133 RandomRecordDeletionThread deleting record #23
132 RecordCreationThread creating record #23
131 FindingRecordsThread searching for matching records.
131 4 results found. 4 to go.
131 FindingRecordsThread reading record #2
131 2 Dogs With Tools Smallville
131 4 results found. 3 to go.
131 FindingRecordsThread reading record #24
131 24 Dogs With Tools Xanadu
131 4 results found. 2 to go.
131 FindingRecordsThread reading record #30
129 RecordCreationThread creating record #45
127 RandomRecordUpdateThread locking record #43
127 RandomRecordUpdateThread updating record #43
131 30 Dogs With Tools Hobbiton
131 4 results found. 1 to go.
126 RandomRecordDeletionThread locking record #19
126 RandomRecordDeletionThread deleting record #19
125 RecordCreationThread creating record #19
124 FindingRecordsThread searching for matching records.
124 4 results found. 4 to go.
124 FindingRecordsThread reading record #2
124 2 Dogs With Tools Smallville
124 4 results found. 3 to go.
122 RecordCreationThread creating record #46
120 RandomRecordUpdateThread locking record #16
120 RandomRecordUpdateThread updating record #16
119 RandomRecordDeletionThread locking record #30
119 RandomRecordDeletionThread deleting record #30
118 RecordCreationThread creating record #30
117 FindingRecordsThread searching for matching records.
117 4 results found. 4 to go.
117 FindingRecordsThread reading record #2
117 2 Dogs With Tools Smallville
117 4 results found. 3 to go.
115 RecordCreationThread creating record #47
113 RandomRecordUpdateThread locking record #42
113 RandomRecordUpdateThread updating record #42
112 RandomRecordDeletionThread suncertify.db.RecordNotFoundException
isDeleted(): record #49 java.io.EOFException
111 RecordCreationThread creating record #48
110 FindingRecordsThread searching for matching records.
110 4 results found. 4 to go.
110 FindingRecordsThread reading record #2
110 2 Dogs With Tools Smallville
110 4 results found. 3 to go.
110 FindingRecordsThread reading record #24
108 RecordCreationThread creating record #49
104 RecordCreationThread creating record #50
103 FindingRecordsThread searching for matching records.
103 4 results found. 4 to go.
103 FindingRecordsThread reading record #2
103 2 Dogs With Tools Smallville
103 4 results found. 3 to go.
103 FindingRecordsThread reading record #24
103 24 Dogs With Tools Xanadu
103 4 results found. 2 to go.
103 FindingRecordsThread reading record #30
110 24 Dogs With Tools Xanadu
110 4 results found. 2 to go.
101 RecordCreationThread creating record #51
103 30 Dogs With Tools Hobbiton
103 4 results found. 1 to go.
99 RandomRecordUpdateThread locking record #7
99 RandomRecordUpdateThread updating record #7
97 RecordCreationThread creating record #52
96 FindingRecordsThread searching for matching records.
96 4 results found. 4 to go.
96 FindingRecordsThread reading record #2
96 2 Dogs With Tools Smallville
96 4 results found. 3 to go.
96 FindingRecordsThread reading record #24
96 24 Dogs With Tools Xanadu
96 4 results found. 2 to go.
96 FindingRecordsThread reading record #30
94 RecordCreationThread creating record #53
92 RandomRecordUpdateThread locking record #41
92 RandomRecordUpdateThread updating record #41
90 RecordCreationThread creating record #54
89 FindingRecordsThread searching for matching records.
89 4 results found. 4 to go.
89 FindingRecordsThread reading record #2
89 2 Dogs With Tools Smallville
89 4 results found. 3 to go.
89 FindingRecordsThread reading record #24
89 24 Dogs With Tools Xanadu
89 4 results found. 2 to go.
89 FindingRecordsThread reading record #30
96 30 Dogs With Tools Hobbiton
96 4 results found. 1 to go.
87 RecordCreationThread creating record #55
85 RandomRecordUpdateThread locking record #36
85 RandomRecordUpdateThread updating record #36
89 30 Dogs With Tools Hobbiton
89 4 results found. 1 to go.
84 RandomRecordDeletionThread locking record #8
84 RandomRecordDeletionThread deleting record #8
83 RecordCreationThread creating record #8
82 FindingRecordsThread searching for matching records.
80 RecordCreationThread creating record #56
82 4 results found. 4 to go.
77 RandomRecordDeletionThread locking record #15
77 RandomRecordDeletionThread deleting record #15
76 RecordCreationThread creating record #15
75 FindingRecordsThread searching for matching records.
75 4 results found. 4 to go.
75 FindingRecordsThread reading record #2
75 2 Dogs With Tools Smallville
75 4 results found. 3 to go.
75 FindingRecordsThread reading record #24
75 24 Dogs With Tools Xanadu
75 4 results found. 2 to go.
75 FindingRecordsThread reading record #30
73 RecordCreationThread creating record #57
71 RandomRecordUpdateThread locking record #26
71 RandomRecordUpdateThread updating record #26
75 30 Dogs With Tools Hobbiton
75 4 results found. 1 to go.
70 RandomRecordDeletionThread locking record #32
70 RandomRecordDeletionThread deleting record #32
69 RecordCreationThread creating record #32
68 FindingRecordsThread searching for matching records.
68 4 results found. 4 to go.
68 FindingRecordsThread reading record #2
68 2 Dogs With Tools Smallville
68 4 results found. 3 to go.
66 RecordCreationThread creating record #58
64 RandomRecordUpdateThread locking record #9
64 RandomRecordUpdateThread updating record #9
63 RandomRecordDeletionThread locking record #40
63 RandomRecordDeletionThread deleting record #40
62 RecordCreationThread creating record #40
61 FindingRecordsThread searching for matching records.
61 4 results found. 4 to go.
61 FindingRecordsThread reading record #2
61 2 Dogs With Tools Smallville
61 4 results found. 3 to go.
61 FindingRecordsThread reading record #24
59 RecordCreationThread creating record #59
52 RecordCreationThread creating record #60
55 RecordCreationThread creating record #61
54 FindingRecordsThread searching for matching records.
54 4 results found. 4 to go.
54 FindingRecordsThread reading record #2
54 2 Dogs With Tools Smallville
54 4 results found. 3 to go.
54 FindingRecordsThread reading record #24
54 24 Dogs With Tools Xanadu
54 4 results found. 2 to go.
54 FindingRecordsThread reading record #30
61 24 Dogs With Tools Xanadu
61 4 results found. 2 to go.
48 RecordCreationThread creating record #62
54 30 Dogs With Tools Hobbiton
54 4 results found. 1 to go.
47 FindingRecordsThread searching for matching records.
47 4 results found. 4 to go.
47 FindingRecordsThread reading record #2
47 2 Dogs With Tools Smallville
47 4 results found. 3 to go.
47 FindingRecordsThread reading record #24
49 RandomRecordDeletionThread locking record #39
49 RandomRecordDeletionThread deleting record #39
47 24 Dogs With Tools Xanadu
47 4 results found. 2 to go.
45 RecordCreationThread creating record #39
42 RandomRecordDeletionThread locking record #25
42 RandomRecordDeletionThread deleting record #25
41 RecordCreationThread creating record #25
40 FindingRecordsThread searching for matching records.
40 4 results found. 4 to go.
40 FindingRecordsThread reading record #2
40 2 Dogs With Tools Smallville
40 4 results found. 3 to go.
40 FindingRecordsThread reading record #24
40 24 Dogs With Tools Xanadu
40 4 results found. 2 to go.
40 FindingRecordsThread reading record #30
38 RecordCreationThread creating record #63
34 RecordCreationThread creating record #64
33 FindingRecordsThread searching for matching records.
33 4 results found. 4 to go.
33 FindingRecordsThread reading record #2
33 2 Dogs With Tools Smallville
33 4 results found. 3 to go.
33 FindingRecordsThread reading record #24
33 24 Dogs With Tools Xanadu
33 4 results found. 2 to go.
33 FindingRecordsThread reading record #30
40 30 Dogs With Tools Hobbiton
40 4 results found. 1 to go.
31 RecordCreationThread creating record #65
29 RandomRecordUpdateThread locking record #48
29 RandomRecordUpdateThread updating record #48
33 30 Dogs With Tools Hobbiton
33 4 results found. 1 to go.
28 RandomRecordDeletionThread locking record #27
28 RandomRecordDeletionThread deleting record #27
27 RecordCreationThread creating record #27
26 FindingRecordsThread searching for matching records.
24 RecordCreationThread creating record #66
22 RandomRecordUpdateThread locking record #29
22 RandomRecordUpdateThread updating record #29
26 4 results found. 4 to go.
20 RecordCreationThread creating record #67
19 FindingRecordsThread searching for matching records.
19 4 results found. 4 to go.
19 FindingRecordsThread reading record #2
19 2 Dogs With Tools Smallville
19 4 results found. 3 to go.
19 FindingRecordsThread reading record #24
19 24 Dogs With Tools Xanadu
19 4 results found. 2 to go.
19 FindingRecordsThread reading record #30
17 RecordCreationThread creating record #68
19 30 Dogs With Tools Hobbiton
19 4 results found. 1 to go.
19 FindingRecordsThread reading record #32
19 32 Dogs With Tools Lendmarch
13 RecordCreationThread creating record #69
22 RandomRecordUpdateThread unlocking record #29
26 FindingRecordsThread reading record #2
26 2 Dogs With Tools Smallville
26 4 results found. 3 to go.
26 FindingRecordsThread reading record #24
28 RandomRecordDeletionThread unlocking record #27
29 RandomRecordUpdateThread unlocking record #48
33 FindingRecordsThread reading record #32
33 32 Dogs With Tools Lendmarch
40 FindingRecordsThread reading record #32
40 32 Dogs With Tools Lendmarch
42 RandomRecordDeletionThread unlocking record #25
26 24 Dogs With Tools Xanadu
26 4 results found. 2 to go.
26 FindingRecordsThread reading record #30
26 30 Dogs With Tools Hobbiton
26 4 results found. 1 to go.
26 FindingRecordsThread reading record #32
26 32 Dogs With Tools Lendmarch
49 RandomRecordDeletionThread unlocking record #39
47 FindingRecordsThread reading record #30
47 30 Dogs With Tools Hobbiton
47 4 results found. 1 to go.
47 FindingRecordsThread reading record #32
47 32 Dogs With Tools Lendmarch
54 FindingRecordsThread reading record #32
54 32 Dogs With Tools Lendmarch
61 FindingRecordsThread reading record #30
61 30 Dogs With Tools Hobbiton
61 4 results found. 1 to go.
61 FindingRecordsThread reading record #32
61 32 Dogs With Tools Lendmarch
63 RandomRecordDeletionThread unlocking record #40
64 RandomRecordUpdateThread unlocking record #9
68 FindingRecordsThread reading record #24
70 RandomRecordDeletionThread unlocking record #32
71 RandomRecordUpdateThread unlocking record #26
75 FindingRecordsThread reading record #32
75 32 Dogs With Tools Lendmarch
68 24 Dogs With Tools Xanadu
68 4 results found. 2 to go.
77 RandomRecordDeletionThread unlocking record #15
82 FindingRecordsThread reading record #2
82 2 Dogs With Tools Smallville
82 4 results found. 3 to go.
82 FindingRecordsThread reading record #24
82 24 Dogs With Tools Xanadu
82 4 results found. 2 to go.
82 FindingRecordsThread reading record #30
82 30 Dogs With Tools Hobbiton
82 4 results found. 1 to go.
82 FindingRecordsThread reading record #32
84 RandomRecordDeletionThread unlocking record #8
82 32 Dogs With Tools Lendmarch
85 RandomRecordUpdateThread unlocking record #36
89 FindingRecordsThread reading record #32
89 32 Dogs With Tools Lendmarch
96 FindingRecordsThread reading record #32
96 32 Dogs With Tools Lendmarch
92 RandomRecordUpdateThread unlocking record #41
99 RandomRecordUpdateThread unlocking record #7
103 FindingRecordsThread reading record #32
103 32 Dogs With Tools Lendmarch
110 FindingRecordsThread reading record #30
110 30 Dogs With Tools Hobbiton
110 4 results found. 1 to go.
110 FindingRecordsThread reading record #32
110 32 Dogs With Tools Lendmarch
113 RandomRecordUpdateThread unlocking record #42
117 FindingRecordsThread reading record #24
119 RandomRecordDeletionThread unlocking record #30
117 24 Dogs With Tools Xanadu
117 4 results found. 2 to go.
117 FindingRecordsThread reading record #30
117 30 Dogs With Tools Hobbiton
117 4 results found. 1 to go.
117 FindingRecordsThread reading record #32
117 32 Dogs With Tools Lendmarch
120 RandomRecordUpdateThread unlocking record #16
124 FindingRecordsThread reading record #24
127 RandomRecordUpdateThread unlocking record #43
126 RandomRecordDeletionThread unlocking record #19
131 FindingRecordsThread reading record #32
131 32 Dogs With Tools Lendmarch
124 24 Dogs With Tools Xanadu
124 4 results found. 2 to go.
133 RandomRecordDeletionThread unlocking record #23
138 FindingRecordsThread reading record #30
138 30 Dogs With Tools Hobbiton
138 4 results found. 1 to go.
138 FindingRecordsThread reading record #32
138 32 Dogs With Tools Lendmarch
141 RandomRecordUpdateThread unlocking record #34
145 FindingRecordsThread reading record #30
145 30 Dogs With Tools Hobbiton
145 4 results found. 1 to go.
145 FindingRecordsThread reading record #32
145 32 Dogs With Tools Lendmarch
148 RandomRecordUpdateThread unlocking record #33
152 FindingRecordsThread reading record #30
152 30 Dogs With Tools Hobbiton
152 4 results found. 1 to go.
152 FindingRecordsThread reading record #32
154 RandomRecordDeletionThread unlocking record #3
159 FindingRecordsThread reading record #30
159 30 Dogs With Tools Hobbiton
159 4 results found. 1 to go.
159 FindingRecordsThread reading record #32
159 32 Dogs With Tools Lendmarch
155 RandomRecordUpdateThread updating record #31
155 RandomRecordUpdateThread unlocking record #31
152 32 Dogs With Tools Lendmarch
162 RandomRecordUpdateThread unlocking record #22
166 FindingRecordsThread reading record #32
166 32 Dogs With Tools Lendmarch
173 FindingRecordsThread reading record #30
173 30 Dogs With Tools Hobbiton
173 4 results found. 1 to go.
173 FindingRecordsThread reading record #32
173 32 Dogs With Tools Lendmarch
175 RandomRecordDeletionThread unlocking record #2
176 RandomRecordUpdateThread unlocking record #18
179 RecordOneUpdateThread unlocking record #1
180 FindingRecordsThread reading record #32
180 32 Dogs With Tools Lendmarch
182 RandomRecordDeletionThread unlocking record #14
12 FindingRecordsThread reading record #32
12 32 Dogs With Tools Lendmarch
124 FindingRecordsThread reading record #30
124 30 Dogs With Tools Hobbiton
124 4 results found. 1 to go.
124 FindingRecordsThread reading record #32
124 32 Dogs With Tools Lendmarch
68 FindingRecordsThread reading record #30
68 30 Dogs With Tools Hobbiton
68 4 results found. 1 to go.
68 FindingRecordsThread reading record #32
68 32 Dogs With Tools Lendmarch
14 RandomRecordDeletionThread locking record #27
14 RandomRecordDeletionThread deleting record #27
15 RandomRecordUpdateThread locking record #41
15 RandomRecordUpdateThread updating record #41
16 RecordOneDeletionThread locking record #1
16 RecordOneDeletionThread deleting record #1
35 RandomRecordDeletionThread locking record #39
35 RandomRecordDeletionThread deleting record #39
36 RandomRecordUpdateThread locking record #22
36 RandomRecordUpdateThread updating record #22
50 RandomRecordUpdateThread locking record #2
56 RandomRecordDeletionThread locking record #43
56 RandomRecordDeletionThread deleting record #43
57 RandomRecordUpdateThread locking record #40
57 RandomRecordUpdateThread updating record #40
78 RandomRecordUpdateThread locking record #18
78 RandomRecordUpdateThread updating record #18
105 RandomRecordDeletionThread locking record #34
105 RandomRecordDeletionThread deleting record #34
106 RandomRecordUpdateThread locking record #30
106 RandomRecordUpdateThread updating record #30
147 RandomRecordDeletionThread locking record #3
106 RandomRecordUpdateThread unlocking record #30
105 RandomRecordDeletionThread unlocking record #34
78 RandomRecordUpdateThread unlocking record #18
57 RandomRecordUpdateThread unlocking record #40
56 RandomRecordDeletionThread unlocking record #43
50 RandomRecordUpdateThread updating record #2
50 RandomRecordUpdateThread unlocking record #2
36 RandomRecordUpdateThread unlocking record #22
35 RandomRecordDeletionThread unlocking record #39
16 RecordOneDeletionThread unlocking record #1
15 RandomRecordUpdateThread unlocking record #41
14 RandomRecordDeletionThread unlocking record #27
147 RandomRecordDeletionThread deleting record #3
147 RandomRecordDeletionThread unlocking record #3
177 RecordOneDeletionThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
170 RecordOneDeletionThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
163 RecordOneDeletionThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
156 RecordOneDeletionThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
172 RecordOneUpdateThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
165 RecordOneUpdateThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
158 RecordOneUpdateThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
151 RecordOneUpdateThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
140 RandomRecordDeletionThread locking record #18
140 RandomRecordDeletionThread deleting record #18
140 RandomRecordDeletionThread unlocking record #18
142 RecordOneDeletionThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
135 RecordOneDeletionThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
134 RandomRecordUpdateThread locking record #22
137 RecordOneUpdateThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
144 RecordOneUpdateThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
134 RandomRecordUpdateThread updating record #22
134 RandomRecordUpdateThread unlocking record #22
149 RecordOneDeletionThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
130 RecordOneUpdateThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
128 RecordOneDeletionThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
123 RecordOneUpdateThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
121 RecordOneDeletionThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
109 RecordOneUpdateThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
107 RecordOneDeletionThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
102 RecordOneUpdateThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
98 RandomRecordDeletionThread locking record #22
98 RandomRecordDeletionThread deleting record #22
98 RandomRecordDeletionThread unlocking record #22
100 RecordOneDeletionThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
114 RecordOneDeletionThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
116 RecordOneUpdateThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
95 RecordOneUpdateThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
91 RandomRecordDeletionThread locking record #2
91 RandomRecordDeletionThread deleting record #2
91 RandomRecordDeletionThread unlocking record #2
93 RecordOneDeletionThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
88 RecordOneUpdateThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
79 RecordOneDeletionThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
74 RecordOneUpdateThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
60 RecordOneUpdateThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
58 RecordOneDeletionThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
53 RecordOneUpdateThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
51 RecordOneDeletionThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
46 RecordOneUpdateThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
43 RandomRecordUpdateThread locking record #41
39 RecordOneUpdateThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
43 RandomRecordUpdateThread updating record #41
43 RandomRecordUpdateThread unlocking record #41
44 RecordOneDeletionThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
65 RecordOneDeletionThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
67 RecordOneUpdateThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
72 RecordOneDeletionThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
81 RecordOneUpdateThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
86 RecordOneDeletionThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
18 RecordOneUpdateThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
37 RecordOneDeletionThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
32 RecordOneUpdateThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
21 RandomRecordDeletionThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
23 RecordOneDeletionThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
25 RecordOneUpdateThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
30 RecordOneDeletionThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.


Harry

[Roel De Nijs]

Hi Harry,

I just made my 2nd point in my previous post, because I saw in your scenario (from a previous post) something like this

updateRecord add <recNo, cookie> to HashMap

I couldn't really quote it (just used my memory to recollect), because you deleted that entire part from your previous post. That is ! You simply don't delete an entire part from a post where someone else replied on, because other ranchers reading this post trying to learn something about locking & synchronization don't have a clue why I made that remark (and maybe think I'm under influence and seeing some pink elephants). So please don't do it in the future (or just add a small indication you edited your post as a result of a remark from another rancher at the bottom of your post, like this: "[edit] reason for editing").
But according to your explanation in one of your next posts, you use the lockCookie correctly and throw SecurityException in the appropriate situation.

Your output seems to look now.

I have just a small remark: do you reuse the record numbers of deleted records when you create a new record? or are you always adding it to the end of the file? Because you have 2 threads RecordTailendCreationThread and RecordOneCreationThread. So according to their name one will create always at the end and another will create record at position 1 (when possible) or at the end.
But according to this part of the output:

17 RecordOneUpdateThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.
16 RecordOneCreationThread before creating a record.
16 RecordOneCreationThread after creating record #42

I would expect the record being created having recNo 1 instead of 42.
But don't get me wrong: there is no requirement at all that say you must reuse deleted entries, so if you always add a record at the end of the file, then that's just fine, but then use only 1 RecordCreationThread (instead of the 2 you have now). And because you don't use a cache and just doing the file i/o every time I think reusing deleted entries is a whole lot harder than when you use a cache (like I did). So I guess your Data class is behaving nicely.

Harry Henriques wrote:the specification doesn't instruct me in how to form the String[] data that is passed into the updateRecord() method.

I think that's a decision you have to make for yourself. In my update-method null-values are allowed: the null-values will be ignored, the other values will be updated. In my create-method no null-values are allowed. And this behavior and expectations are clearly described and explained in the javadoc comments of each method.

Kind regards,
Roel

[Harry Henriques]

Hi Roel,

I have just a small remark: do you reuse the record numbers of deleted records when you create a new record?

In my DataClassTest.java, I create two threads, i.e. RecordOneCreationThread and RecordTailendCreationThread, and both call the createRecord() method behind the scenes. Since only record #1 is being deleted in my test scenario, the createRecord() method either re-uses record #1 (if deleted) or creates a new record on the end-of-file. There may be a problem, because my test output doesn't show the createRecord() method creating record #1. Thanks.

Best regards,

Harry

[Harry Henriques]

Hi Roel,

Roel De Nijs wrote:
But don't get me wrong: there is no requirement at all that say you must reuse deleted entries, so if you always add a record at the end of the file, then that's just fine, ...

My B&S assignment does discuss re-using a deleted entry during record creation with createRecord(). See below. You are correct. It is not a "must" requirement. I think that I can easily get it working with my implementation.

SCJD Assignment wrote:
// Creates a new record in the database (possibly reusing a
// deleted entry). Inserts the given data, and returns the record
// number of the new record.

Best regards,

Harry

[Harry Henriques]

Hi Fellow Bloggers,

I have updated the DataClassTest.java test output in my previous post within this blog thread.

Regards,

Harry

[Roel De Nijs]

Hi Harry,

I have updated the DataClassTest.java test output in my previous post within this blog thread.

According to this changes: using lockRecord and unlockRecord in combination with creation of a new record is a bit weird, because you have to pass a record number, but there isn't one because you are just creating a new record. So that shouldn't be the solution. If you have problems when 2 threads invoking createRecord, your createRecord is not thread-safe. So you should be sure that every shared member which is changed, is put in a synchronized block, so another thread can not interfere and corrupts your data / shared member. In my solution every method was marked synchronized. That is not the most performant solution, but it is by far the easiest one and you don't have to worry about 2 threads corrupting some data.

Another remark about the new output: I don't think you are allowed to see anything as an EOFException appear in your output, but of course that could be the problem if your random record numbers are just too big and you calculate the offset, move the filepointer to the location and do a read. Because you try updates of numbers 42 and 43, it seems to be the cause. (Other people got EOFException because something was wrong with their create-method or read-method or calculation of the offset, that's why I have mentioned it).

Enjoy the new year's eve!
Kind regards,
Roel

[Harry Henriques]

Hi Roel,

Best wishes for the coming year. Happy New Year!

You are correct about the EOFException. My lockRecord() method doesn't throw an EOFException. It throws a RecordNotFoundException that wraps an EOFException, when the random number generator in DataClassTest.java generates a random long integer that is beyond the database file boundaries.

I heavily synchronize all my methods in the Data class. I'm not sure if I have a problem with the createRecord() method. The records written to the flatfile database file on my hard disk are NOT corrupted. I don't see the problem in my last test output post [edit].

Thanks again,

Harry

[Roel De Nijs]

Hi Harry,

Thread-id 9 locks record 1, deletes it and unlocks it. And then I see following line in your output:

16 RecordOneDeletionThread deleting record #1

How can thread 16 successfully lock a record that does not exist anymore?

Kind regards,
Roel

[Harry Henriques]

Hi Roel,

Roel De Nijs wrote:
How can thread 16 successfully lock a record that does not exist anymore?

The DataTestClass.java prints the following text to the ouput stream, and then invokes the associated method.

1. print "Thread Id __ Thread Name __locking record #__recNo"
2. invoke long lockCookie = lockRecord(recNo);
3. print "Thread Id __ Thread Name __deleting record #__recNo"
3. invoke data.deleteRecord(recNo, lockCookie);
4. print "Thread Id __ Thread Name __unlocking record #__recNo"
5. invoke data.unlock(recNo, lockCookie);

Taking the aforementioned pseudocode into account, the following occurs in the test output:

9 RecordOneDeletionThread locking record #1
9 RecordOneDeletionThread deleting record #1
16 RecordOneDeletionThread locking record #1
9 RecordOneDeletionThread unlocking record #1
16 RecordOneDeletionThread deleting record #1
16 RecordOneDeletionThread unlocking record #1

Thread 16 doesn't acquire the lock; it just invokes the lock method, and enters the wait() queue. Not until Thread 9 unlocks record #1 can Thread 16 acquire the lock and delete record #1.

I think it is working, Roel.

Do you see what I mean?

Best regards,

Harry

[Roel De Nijs]

Hi Harry,

This will be my last post of 2009

Thread 9 locks the record and Thread 16 goes into the wait queue.
Thread 9 deletes the record. Thread 16 still waiting.
Thread 9 unlocks the record (and all waiting threads get a notification).
Thread 16 now gets the lock (otherwise it should never be able to invoke the delete-method, because it doesn't have an appropriate lockCookie). But how can thread 16 acquire the lock of a deleted record.

I would expect the test output to be as follows:
9 RecordOneDeletionThread locking record #1
9 RecordOneDeletionThread deleting record #1
16 RecordOneDeletionThread locking record #1
9 RecordOneDeletionThread unlocking record #1
16 RecordOneDeletionThread suncertify.db.RecordNotFoundException
lockRecord(): record 1 is deleted.

and no calls from thread 16 to delete and unlock methods, because a thread can't get the lock of a deleted record.

Kind regards,
Roel

[Harry Henriques]

Hi Roel,

You caught me again. Thanks. What you noticed in the updated test output was wrong. I have made changes to my synchronization. I think that it is working now.

I have a question for you. In the test output, there are two threads, i.e. 181 and 182. They both work on record #14. I think that my test output is correct. Do you see a problem?

Best regards,

Harry

[Roel De Nijs]

Hi Harry,

And this is my first post of 2010!

You mean threadA that deletes a record and threadB creates a new record and reuses the recNo of the deleted record before the record is unlocked by threadA. I think there is nothing wrong with your output. To be honest I don't know how my application reacted in this kind of situation, but I think there is nothing wrong with it because lock/unlock is only for update/delete, not for create.

Kind regards,
Roel