Win a copy of Mesos in Action this week in the Cloud/Virtualizaton forum!
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

Locking files to prevent concurrent modification

 
Eitan Levi
Greenhorn
Posts: 14
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I'm writing a library that uses an XML-based database-style storage format. What I'd like to do, is prevent two separate processes (that will both be using the library) from modifying the files simultaneously. More specifically, I'd like the first process that starts to obtain a (write) lock on the file, and not release it until that process is over -- to prevent inconsistent reads and lost updates from both processes.

(Yes, that does mean that the second process will not be able to do anything useful, and that's exactly what I want.)

The trouble I'm having is that I can't seem to parse the XML file if there is a lock on it, even if I'm reading it from the same process that obtained the lock.

For example, this code yield an "java.io.IOException: The process cannot access the file because another process has locked a portion of the file":



The exception is thrown at the db.parse(file) part. Because it even *reaches* this point, after a call to channel.lock(), it means that this process has obtained the lock. Why then can't it read the file?

I would hazard a guess and say that since (according to JavaDoc) the FileLock class operates on the file's channel, that in order to read from a file for which I have the lock I would need to use the channel's read methods, correct?

Unfortunately, DocumentBuilder#parse can only accept a java.io.File, InputSource, InputStream or a URI. I can't for the life me figure out how to get an Input{Stream,Source} from a FileChannel object -- though it would seem like this is a fairly logical thing to do.

If anyone has any ideas or solutions, or maybe can point me to some resource or documentation that I've overlooked, the help would be greatly appreciated! Thanks!
 
Edward Harned
Ranch Hand
Posts: 291
Java
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
IBM tried using file locks back in the System/360. That was in the mid 1960's. It was a disaster.

When you use the word "process" I can only assume you mean two or more JVM's. I can't begin to list the problems associated with file locking with multiple processes.

IBM and all the rest solved this problem by routing all requests for file "locks" through a single address space (or in our case JVM.)

If you set up a common JVM with all the logic to control a file access without actually marking the physical file then all your other processes can go through this single JVM to read or one-at-a-time, write.
 
Eitan Levi
Greenhorn
Posts: 14
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks for the advice, but I don't have control over how many JVMs may be running and using the same library with the same files.

If I have one common JVM with all the file control logic in it, then I'd need to get any other process that wants to access the data to communicate with it -- which means IPC, which I certainly could do, but I thought that simply checking for a file lock would be the "simplest thing that could possibly work."
 
Manuel Palacio
Ranch Hand
Posts: 45
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I haven't tried this but couldn't you just read() the file channel into a ByteBuffer and then create a ByteArrayInputStream from the byte [] backing it?

Then you pass the stream to DocumentBuilder.
[ September 05, 2006: Message edited by: Manuel Palacio ]
 
Edward Harned
Ranch Hand
Posts: 291
Java
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
This is what DBMS are all about. The DBMS owns the data. Applications arn't even aware of the underlying file structure.

Use an existing DBMS or create your own.
 
Stan James
(instanceof Sidekick)
Ranch Hand
Posts: 8791
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
If the data must be in the file system you can still use the database to manage a token. Lock the token (row) in the database, do your file IO, release the token. I don't know if you'd be bets off to use real database locks or use a particular value in a column as a lock or what. Could be some interesting experiments there.
 
Eitan Levi
Greenhorn
Posts: 14
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks a lot for the replies, so far!

Unfortunately I don't define the constraints, I just work within them.

Using a real, true-to-form database would be ideal, but it's not guaranteed. My architecture is flexible enough to work with a database as a data source, but right now (and for the indefinite future) I'm stuck using the file system. So I'm trying to find a way to prevent accidental concurrent modification of the files.

A voluntary lock is fine; I'm trying to prevent two instances of the same code from clashing. If another program tries to access and change the file, I know there's not much I can do.

When I started this thread I had in mind some method that's analogous to Unix's flock( fd, FLOCK_EX ) function. (But of course, being Unix-specific I doubt Java could possibly guarantee that kind of behaviour across platforms).

Originally posted by Manuel Palacio:
I haven't tried this but couldn't you just read() the file channel into a ByteBuffer and then create a ByteArrayInputStream from the byte [] backing it?

Then you pass the stream to DocumentBuilder.

[ September 05, 2006: Message edited by: Manuel Palacio ]


That's a clever idea so I tried it... It actually works! Thanks.
[ September 05, 2006: Message edited by: Eitan Levi ]
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic