• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Data Locking with a DB File.

 
Ranch Hand
Posts: 531
1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I think the forum rules forbid going into detailed description of the assignment, so please let me know if I am describing parts of it in too much detail.

The question I have deals with the requirement to enable "record locking" on a database file. I understand it as if I had to write code that locks chunks of the file itself. It is not impossible to implement without the NIO facilities, but more difficult.

I saw on Sun website a requirement that the NIO facilities are not to be used for the Developer project. And yet the Assignment's requirement calls for something like that, although not explicitly. What would your recommendation be? Do they really want parts of the file locked down without the use of NIO, or could I be possibly mis-reading the instructions?

P.S. And if they do want parts of file locked down, how does one implement a full-file search?


My Solution:
---------------------------------------------------------------------------

I was thinking about doubly using the byte at the beginning of the record as the record's cookie value, and keeping the said bytes in a Set or maybe in a synchronized Collection-type object. But that still would not solve the problem of having to have the entire db file locked. If there are n clients having locked n records, and an n+1 client wishes to have the db searched, it will have quite a time trying to acquire the lock on the entire file. Making the find method unsynchronized is not an answer, as it could get a dirty read. So what would the solution be?
[ July 25, 2004: Message edited by: Anton Golovin ]
 
Ranch Hand
Posts: 118
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Anton,

There are two issues which need locking in the assignment and I am not too sure which you are refering to - remember that they are both seperate issues:

The first issue is to prevent concurrent writes on the database file. On my instructions (and I think it is the same with all asignments) I was told to assume that only the app I was developing would access the file. With that assumption you can use synchronisation to ensure that db writes are not corrupted.

The second issue is record locking controlled by the locking cookie. You mentioned using the byte at the begining of the record to track the cookie - this byte is just a deleted flag (1 or 0) so it is not suitable. The way the majority of people do this is using a Map (possibly using a seperate lock manager class) to relate a cookie to a record number - you can use this object as the monitor for the wait() in lock() and notifyall() in unlock(). I just used a long starting at 0 and incremented by one each time a cookie was needed as the cookie value.

Hope this clears it up a bit....

Regards,

Jon
[ July 25, 2004: Message edited by: Jon Entwistle ]
 
Anton Golovin
Ranch Hand
Posts: 531
1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Oh, no, as the value of the cookie I was going to use the offset number of bytes to the row being locked (i.e. an x number of bytes row y is into the file.)

I understand the scheme of locking records by record number, using a Set of some sort to ensure uniqueness of record, but there is a big problem here, regarding deleted records.

What if user A gets a list of all records and attempts to lock a record Y. Afterwards, user B records a record in place of a deleted one at the location of Y-1. And user C, getting the new record list, wishes to lock the record Y as user A sees it. And the system lets him, because the system thinks he is locking a different record.

Here is the problem with the locking scheme, and one cannot assume that the record numbers, as seen by all users, are going to be the same every time.

It can be cured by counting deleted records for valid record numbers, of course, but then the client must, presumably, display non-interrupted record numbers for the viewer, while keeping a real set hidden, at the GUI.

This is only possible if the offset bytes are used as the cookie because offset number of bytes and the real record number can be converted into one another easily.

Anton
[ July 25, 2004: Message edited by: Anton Golovin ]
 
Jon Entwistle
Ranch Hand
Posts: 118
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Ah - I see. The danger I see with that is if you delete one record from the database file (rather than mark it as deleted), all of your cookies will point to the wrong records - I would go for something simpler if I were you like incrementing a variable.

Regards,

Jon
 
Anton Golovin
Ranch Hand
Posts: 531
1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I completely agree. It can be a problem, and I am really thinking about incrementing a variable or making the assumption that a record marked deleted will never be wiped from the file but will only be overwritten.

Thanks for your opinion!

Also, I think the requirement that at most only one program will access the file probably means the server program is the only one to access the file, but the multiple clients have to be able to access the file concurrently...

Anton.
[ July 25, 2004: Message edited by: Anton Golovin ]
 
author and jackaroo
Posts: 12200
280
Mac IntelliJ IDE Firefox Browser Oracle C++ Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Anton,

Also, I think the requirement that at most only one program will access the file probably means the server program is the only one to access the file, but the multiple clients have to be able to access the file concurrently...



No - in that case, multiple client will have to be able to access the server concurrently. This is a pedantic point but it is important. Since only your server has to access the physical file, you do not need to worry about locking the file to prevent other clients using it.

Regards, Andrew
 
Anton Golovin
Ranch Hand
Posts: 531
1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by Andrew Monkhouse:
Hi Anton,



No - in that case, multiple client will have to be able to access the server concurrently. This is a pedantic point but it is important. Since only your server has to access the physical file, you do not need to worry about locking the file to prevent other clients using it.

Regards, Andrew



That's how I read it at first: that only one client at a time can access the file. But then I started re-reading it and probably complicated the problem beyond the intention of the assignment by conjuring up the idea that multiple clients have to be able to access the file concurrently by accessing only parts of the file, which are said to be "locked" by a client's thread. Then I read your other post, in which you wrote that there is no guarantee that one thread will handle a particular client's all requests.

So I can't write the code which prevents other threads from writing in a particular part of the file. I mean, it may be possible, but I don't know how.

I think I need to go back to the assignment and re-read it very carefully.

Thanks.

Anton.
 
Jon Entwistle
Ranch Hand
Posts: 118
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Anton,

That's how I read it at first: that only one client at a time can access the file.

Please tell me if I am wrong but I think that this is where you might be having problems conceptualy with this - the client actualy never accesses the file. The only thing any client does is to ask the server to access the file on its behalf - it is up to the server to manage these requests from any number of concurrent clients, which it must do without violating the record (i.e. cookie) locking contract and without corrupting the file or any shared variables.

Regards,

Jon
 
Anton Golovin
Ranch Hand
Posts: 531
1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by Jon Entwistle:
Hi Anton,

[qb] That's how I read it at first: that only one client at a time can access the file.

Please tell me if I am wrong but I think that this is where you might be having problems conceptualy with this - the client actualy never accesses the file. The only thing any client does is to ask the server to access the file on its behalf - it is up to the server to manage these requests from any number of concurrent clients, which it must do without violating the record (i.e. cookie) locking contract and without corrupting the file or any shared variables.

Regards,

Jon[/QB]



Sure, what I meant is that the Data class accesses the file. And I thought that each Data object has to be able to access the file concurrently, i.e., lock a portion of the file via a static controlling Map, say, in the Data class, and then write to the locked region. I mean, using RandomAccessFile it can be accomplished concurrently.

What I now think I should do is just specify in the static Map structure that a certain record is being used so that another Data object would not proceed to modify it at the same time. All the locking is in that Map structure. In that way, all I have to do is synchronize on the Map, try to see if a record is already locked and wait until it is no longer locked, put the record number back in, and proceed to modify it.

Here's a couple of questions I asked myself:

1) Do I need to use a synchronized Collection, or can I get away with unsynchronized one?

I think I will get a synchronized one. Even though when writing to the Map I will have a lock, I don't want anyone who later changes my program to accidentally make it thread-unsafe.

2) Do I need to make the Data class thread-safe?

I think I do, since I found out that each RMI thread may access whichever Data object it pleases.

3) How do I avoid a deadlock?

I will avoid a deadlock by making sure I never reverse-nest locking. I will nest locking in the same order, though:

i) thread has to acquire lock on Data object.
ii) thread has to acquire lock on Map from locked Data object.

However, all threads will undergo this procedure in the same order so there should not be a problem.


I will:

a) Synchronize the Map. (threads may wait on that.)
b) Check for record being locked in it. If yes, wait() (lock on Map released.)
c) When the record is unlocked, lock it again, this time by different Data object using the calling thread (i.e. client) lockCookie value. When checking, Data object acquires lock on Map automatically. Release lock on Map.
d) Modify the database file. Here I must guarantee that a record is unlocked only after database file is modified.
e) Synchronize the Map. Unlock the record I locked.

So in order to accomplish database access safely, three locks are in play:

lock 1 is the lock on the Data object.
lock 2 is the lock on the Map.
lock 3 is the lock put in the Map. This lock is purely logical: no lock is granted in the sense the synchronized keyword grants it. But this lock is accomplished by a Record object in the Map (or Set, perhaps, to avoid checking for duplicates) as the key (and lockCookie as the value) so that other threads, checking upon the presence of this record number, would abstain from modifying the record in the database.

That's basically what I came up with in regard to locking. I'd like also to keep an eye on records with the Record class. Each instance would have the beginning and end of the record byte region and static info on the header and such; and also instance methods to strip or add whitespace, etc. Each record would not have to be synchronized. This way, the functionality in Record could be modified to handle other database formats easily.

But here's the question I have not answered yet. When I need to search the entire file, what is the most efficient way to handle it? I could put a null value for the key and have threads check on it, and if it exists, wait, so that all logical locks clear before a thread proceeds to search the whole file. In this case, the find feature is going to be pretty time-consuming, and I am not sure it's a good choice. But I don't really see any other way.

The only other way, though, is to have the File object locked by a thread before modyfying it. That way, a find request will not need to lock anything. It needs only to acquire a lock on the File object before searching. Other threads, even those which have locked a Record, will simply wait before accessing and modifying the file. However, again, there is the issue that the file will be searched before being immediately modified, and the client may be getting outdated information. So, I think I am left to the time-consuming null lock-clearing scheme.
[ July 27, 2004: Message edited by: Anton Golovin ]
 
Jon Entwistle
Ranch Hand
Posts: 118
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Anton,



that each Data object has to be able to access the file concurrently, i.e., lock a portion of the file via a static controlling Map, say, in the Data class, and then write to the locked region. I mean, using RandomAccessFile it can be accomplished concurrently.


It is your call, but is seems a pretty complex solution to me. Have you considered having just one Data class instance which is accessed by the server to service all client requests?


What I now think I should do is just specify in the static Map structure that a certain record is being used so that another Data object would not proceed to modify it at the same time. All the locking is in that Map structure. In that way, all I have to do is synchronize on the Map,


If I see this right more than one thread could be writing to the file at the same time here (even though on different records) which would corrupt the file.


1) Do I need to use a synchronized Collection, or can I get away with unsynchronized one?


If you are going to be accessing the collection with more than one thread concurrenly then yes.


2) Do I need to make the Data class thread-safe?


You need to make all objects accessed by more than one thread concurrently thread-safe.


Regards,

Jon
 
With a little knowledge, a cast iron skillet is non-stick and lasts a lifetime.
reply
    Bookmark Topic Watch Topic
  • New Topic