• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

File I/O problem

 
Ranch Hand
Posts: 85
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I have a live location where a file is written. I don't know by what process, could be manual aur programmed.

My process runs and checks the file to modified time changes. and if it changes, i transfer a copy of the file to another location for further processing.
Currently, while the live file is being written, my application picks up and copies the file (this copy is partly, as the file wasnt written completely).

How do i work around this such a lock or something? I dont have any control over the process which creates the live file.

Please advise.
 
Wanderer
Posts: 18671
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Well, you could try using FileChannel's tryLock() to obtain a lock on the file before you read it. If it returns null, that would mean another process has a lock on the file. You can use lock() to wait until the lock is available, or you can sleep or do something else. This isn't guaranteed to work, but it's worth a try. It's possible that the other process ignores or circumvents locking; it's also possible that it just continues to hold on to the lock it has.

Another option might be to use File's renameTo() to attempt to move the file to another directory. This is generally a nice atomic operation. If it fails, that may indicate that the other process is currently writing to the file. If it succeeds, you can be pretty sure that the process is no longer writing to that file, as it's now relocated. The process should now create a new file at the old location, to add its new content. This only works if it's acceptable for the file to be moved and broken up into parts; I have no idea if that applies to your situation.

Otherwise, you may simply have to write your Java code to allow for the possibility that the file is currently being written to. The last line of the file should always be regarded as potentially incomplete. Perhaps you should always delay processing the last line until the next update, when additional lines arrive.
 
fahad siddiqui
Ranch Hand
Posts: 85
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
FileChannel's lock is not possible, as the operation which provides the live file does not create any lock over the file and so, FileChannel does get a lock over the file which in turn disrupts the writing process.

renameTo, i cant use as the live file cannot be tinkered with. I just need to get a local copy of it for further processing.

Isn't there some unix funda of managing this kind of locking on its own? The live file comes on a unix box.
 
Ranch Hand
Posts: 291
Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
This problem has been around since the first 3rd generation computers in the mid 1960's.

The simple answer is NO.

The way to protect files from different processes is to use a file/database management system. Simply put -- all I/O goes through a single process.
 
Jim Yingst
Wanderer
Posts: 18671
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Well, you could do something like

tail -f filename | java MyLineProcessor

to read from the file as it's updated, and pass those updates on to your Java program as System.in. Using tail -f allows you to skip addressing several complications in your java code, at a cost of less platform independence since you're relying on a unix tool. But there you've still got the issue that the last line you've read may not be complete. Perhaps the simplest way to handle that is something like this:

This assumes that a single line is always enough to process. If, for example, you need to process one record at a time, and a record consists of ten lines, then you could write something that keeps on appending to a StringBuilder until ten lines have been read, then calls toString() on the StringBuilder, sends the ten lines off to some handleRecord() method, and then starts a new StringBuilder for the next ten lines.
 
With a little knowledge, a cast iron skillet is non-stick and lasts a lifetime.
reply
    Bookmark Topic Watch Topic
  • New Topic