aspose file tools*
The moose likes Developer Certification (SCJD/OCMJD) and the fly likes Locking and File IO Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Certification » Developer Certification (SCJD/OCMJD)
Bookmark "Locking and File IO" Watch "Locking and File IO" New topic
Author

Locking and File IO

William Ni
Greenhorn

Joined: Aug 26, 2004
Posts: 8
Hv been struggling on this issue for weeks. I really appreciate if anyone here can share his/her ideas with me.
I am going to implement a thin client, that is the locking will be done at the server side. What I can't decide is to implement a simple but less-efficient or a complex but efficient locking and file io mechanism.
Now I prefer the former, as Sun nv specifies the efficiency issue.

My design is like this:
I hv a singleton IO_Manager lying between the Data class and the actual file. I do record-level locking at Data class, but the IO_Manager will do the actual IO jobs sequentially.
For example, there might be several clients, each hving a Data instance. Each of them may wanna do read, update or create(implement it but not expose it to clients), etc. So they will call the relevant methods from their own Data instance. Record-level Locking is done here. i.e. no 2 threads are allowed to manipulate the same record.
Then, there would probably be many operations coming to IO_Manager at the same time, i.e. serveral read operations, several update operating on different records, etc. The IO_Manager keeps a FIFO task pool, pick one job and do it each time. In this way, the file IO will always be thread safe.

I understand this is an inefficent way, and it looks to me the record-level locking is somehow unnecessary, as wat I am actually doing is file-level locking.
I am seeking ur comments, and willing to explain more due to my English. thanks.

[ September 07, 2004: Message edited by: William Ni ]
[ September 07, 2004: Message edited by: William Ni ]

I am learning Java and English.
Andrew Monkhouse
author and jackaroo
Marshal Commander

Joined: Mar 28, 2003
Posts: 11509
    
  95

Hi William,

Welcome to JavaRanch and this forum.

One suggestion - try to avoid 'l33t'-speek or removing vowels or other "shortcuts". Your English is very good, but it does slow me down trying to decipher what you mean when you write 'nv' or 'hv' - sure I can work it out, but why would I spend time on just deciphering your message, when I could be spending time responding to other messages? Those other contributors to this forum who are also learning English may not even bother trying to decipher it, which reduces your chances of getting a response.

it looks to me the record-level locking is somehow unnecessary, as wat I am actually doing is file-level locking


Not at all. Booking a record is actually a little more involved than just updating the record. You also need to ensure that only one client books the record. This requires your booking process to check that the record is still available to be booked, and then book it. Since there are two steps involved here, you need some way of making sure that no other thread can book your record in between those two steps.

To make this a bit clearer, we are talking about two threads (A and B) both trying to book the same record:

  • Thread A checks that the record is still available
  • Thread B checks that the record is still available
  • Thread A books the record
  • Thread B books the record



  • Logical record locking can help you avoid this problem - can you see how? (I don't want to just give the answer away ).

    Regards, Andrew


    The Sun Certified Java Developer Exam with J2SE 5: paper version from Amazon, PDF from Apress, Online reference: Books 24x7 Personal blog
    peter wooster
    Ranch Hand

    Joined: Jun 13, 2004
    Posts: 1033
    I agree with Andrew on this, both the english and the locking design. Your english is quite good, if you lose the net-speak abbreviations. Here is your post redone in proper english, with the changes in bold, as you can see most of the changes are very minor.

    "I have been struggling on this issue for weeks. I would really appreciate it if anyone here can share their ideas with me.
    I am going to implement a thin client, that is the locking will be done at the server side. What I can't decide is whether to implement a simple but less-efficient or a complex but efficient locking and file io mechanism.
    Now I prefer the former, as Sun never specifies the efficiency issue.

    My design is like this:
    I have a singleton IO_Manager lying between the Data class and the actual file. I do record-level locking in the Data class, but the IO_Manager will do the actual IO jobs sequentially.
    For example, there might be several clients, each having a Data instance. Each of them may want to do read, update or create(implement it but not expose it to clients), etc. So they will call the relevant methods from their own Data instance. Record-level Locking is done here. i.e. no 2 threads are allowed to manipulate the same record.
    Then, there would probably be many operations coming to IO_Manager at the same time, i.e. serveral read operations, several updates operating on different records, etc. The IO_Manager keeps a FIFO task pool, picks one job and does it each time. In this way, the file IO will always be thread safe.

    I understand this is an inefficent way to do it, and it looks to me that the record-level locking is somehow unnecessary, as what I am actually doing is file-level locking.
    I am seeking your comments, and am willing to explain more due to my English. thanks."

    On the topic of locking, here are my thoughts:

    - I'm doing the rich client, I started with a thin client, but the requirements of a traditional client server and an easily enhanced client drive me in the direction of a rich client. If you go with the thin client, the system is a bit easier to implement, but enhancements require server changes that would impact all clients.

    - I lock arbitrary numeric resources, this is easy to test and seperates the locking activity from the I/O activity. I use a Map of numbers and Locks, a Lock contains a cookie and a Data instance. In the thin client you could just use a Long containing the cookie.

    - I synchronize all access to the lock Map, the locking code uses notifyAll.

    - I synchronize all I/O at the lowest point possible on the RandomAccessFile to provide thread safe data access.

    - like you I have a Singleton I/O manager between the Data instances and the file I/O routines. The file level locking is done in the readRaw and writeRaw methods in this class.

    - I don't use a queue of database requests, although I have used that technique in real world applications using Oracle.

    I hope this helps with both your English studies and your certification effort.

    /peter
    [ September 08, 2004: Message edited by: peter wooster ]
    William Ni
    Greenhorn

    Joined: Aug 26, 2004
    Posts: 8
    Hi, guys, thanks for the reply.
    I am sorry about these abbreviations and I will try posting in proper English.


    Logical record locking can help you avoid this problem - can you see how?


    Yes, Andrew, I can see logical record locking is necessary. What I missed is the keyword "logical". I have been thinking about how to manage record-level locking on the actual file, i.e. let two threads modify the same file at two different locations at the same time. In fact, I am wondering whether it's possible to do that, and of course, in a safe way.



    - I'm doing the rich client, I started with a thin client, but the requirements of a traditional client server and an easily enhanced client drive me in the direction of a rich client. If you go with the thin client, the system is a bit easier to implement, but enhancements require server changes that would impact all clients.

    - I lock arbitrary numeric resources, this is easy to test and seperates the locking activity from the I/O activity. I use a Map of numbers and Locks, a Lock contains a cookie and a Data instance. In the thin client you could just use a Long containing the cookie.

    - I synchronize all access to the lock Map, the locking code uses notifyAll.

    - I synchronize all I/O at the lowest point possible on the RandomAccessFile to provide thread safe data access.

    - like you I have a Singleton I/O manager between the Data instances and the file I/O routines. The file level locking is done in the readRaw and writeRaw methods in this class.

    - I don't use a queue of database requests, although I have used that technique in real world applications using Oracle.


    Hi, Peter, thank you for your reply. I am glad to hear that you also use similar IO_Manager to manage the actual file IO. Below are some of my thoughts:

    1. I am not going to argue with you whether a thin client or a rich client is better, but I think if there is a need for enhancement, the server-side change will always impact the client side. IMO, it usually requires a rich client to change more.

    2. Are you suggesting that you cache all the locked records? So when a client requests a lock on a record, it will first check in the cache whether that record is already locked. If the record is locked by another thread, this thread will go to wait status. Please correct me if I am wrong.

    3. May I know how you schedule your file IO operation instead of the Task Pool pattern? Of course it's ok if you feel it inconvenient.


    Lastly, is there any problems regarding my Locking and File IO design? I am going to start my coding phase next week.
    peter wooster
    Ranch Hand

    Joined: Jun 13, 2004
    Posts: 1033
    1. I am not going to argue with you whether a thin client or a rich client is better, but I think if there is a need for enhancement, the server-side change will always impact the client side. IMO, it usually requires a rich client to change more.


    If you implement the server as a pure database server and put all the business and presentation in the client, there is no need to modify the server to add client enhancements. This way a new client program can be rolled out to a subset of the users without impacting the others.

    If you put the business rules in the server any change to these impacts all clients. the best choice would be a true 3 tier system with a thin browser based client, a business layer on a web server and a database. But that's not permitted.

    2. Are you suggesting that you cache all the locked records? So when a client requests a lock on a record, it will first check in the cache whether that record is already locked. If the record is locked by another thread, this thread will go to wait status. Please correct me if I am wrong.


    I don't cache any records, but I do keep a hashMap of locked record numbers and the cookie and owner. The thread attempting to lock a record uses wait, eventually it will run when the user with it locked calls notifyAll.

    3. May I know how you schedule your file IO operation instead of the Task Pool pattern? Of course it's ok if you feel it inconvenient.


    I'm not familiar with the "Task Pool Pattern". Do you mean a set of "serial processors" running worker threads that execute commands taken off a queue? If so I don't do anything that complicated, but it is a very good strategy for high performance file systems.

    I schedule the I/O by starting a thread when a network connection is made, I use one connection per business activity. That thread then attempts to do the I/O synchronizing on the RandomAccessFile. This means that each user has zero or more threads performing the I/O for the active business operations.

    Lastly, is there any problems regarding my Locking and File IO design? I am going to start my coding phase next week.


    If you have taken Andrew's comments into your design and are locking records at the business layer so that sequences such as {lock, read, update, unlock} are race free and you are synchronizing blocks around your file I/O, or using a queue to make the file I/O single threaded you should be fine.

    You'll probably get further insights when you start to code, that's why most modern project management uses iterative processes such as RUP, UP or XP. I'm currently on my 4th iteration, each iteration produced a working system. Here's my iteration history so far

    1) single threaded, simple GUI, able to read and write data file.

    2) thin client, complete MVC GUI, locking restricted to the server, socket I/O for networking using Command Pattern, sessionless.

    3) rich client, session management, only one business command and server thread per client, locks owned by thread.

    4) rich client, sessionless, new connection for each business command, DTO at client-network layer interface. Locks owned by Data instance. Multiple business commands per client possible. Still under construction.
    William Ni
    Greenhorn

    Joined: Aug 26, 2004
    Posts: 8
    Thank you so much for the detailed explanation, Peter. I benefit a lot from your post.
    I think it's a good practice to manage a project using RUP. I will come out with some simple deliverable first.
    Once again, thanks, Peter, and all ranchers.
     
    I agree. Here's the link: http://aspose.com/file-tools
     
    subject: Locking and File IO