This week's giveaway is in the Android forum.
We're giving away four copies of Android Security Essentials Live Lessons and have Godfrey Nolan on-line!
See this thread for details.
The moose likes Threads and Synchronization and the fly likes How to: synchronizing a file access with other programming languages. Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Android Security Essentials Live Lessons this week in the Android forum!
JavaRanch » Java Forums » Java » Threads and Synchronization
Bookmark "How to: synchronizing a file access with other programming languages." Watch "How to: synchronizing a file access with other programming languages." New topic
Author

How to: synchronizing a file access with other programming languages.

Aji Prasetyo
Ranch Hand

Joined: Jun 13, 2007
Posts: 65
Dear guys,

I am not sure whether I should post it here or in the JNI forum. but I am not asking about native code, so I guess I am posting it here.

I have a stream of data, I write it into a file using FileWriter(). The problem is, the data in the file is suppose to be read by C++ application. How do you synchronize a file access with something which is not java (C++)?

If anybody familiar with the case, please help me.

Thank you so much beforehand.
Edward Harned
Ranch Hand

Joined: Sep 19, 2005
Posts: 291

Obviously you're new to the world of computing. This problem has been around since the 1940's.

The solution to multiple processes (or processors) accessing the same file at the same time is to use a file/data base management system. The "system" owns the file/table etc. and controls all access to it.


Ed's latest article: A Java Parallel Calamity http://coopsoft.com/ar/Calamity2Article.html
Peter Chase
Ranch Hand

Joined: Oct 30, 2001
Posts: 1970
Originally posted by Edward Harned:
Obviously you're new to the world of computing. This problem has been around since the 1940's.


Multiple processes in the 1940's? I'd be interested to see the evidence of that. File systems in the 1940's? Ditto.

Back to the original poster's question. Do you really need to use an actual file? What about a pipe or socket? Presumably, your reading process can successfully open the file for read, while the writing process is writing it. But I guess your difficulty is that arbitrary chunks of data get written to it, and sometimes these chunks constitute a part-complete update. Can you just add some End-Of-Record marker to the end of each complete update? Can you then just poll the file (or, better, pipe or socket)?

If it's more complicated than that, then maybe you need a transaction service. You'd get one if you used an RDBMS, but you can also get ones outside of that.
[ October 16, 2007: Message edited by: Peter Chase ]

Betty Rubble? Well, I would go with Betty... but I'd be thinking of Wilma.
Edward Harned
Ranch Hand

Joined: Sep 19, 2005
Posts: 291

Multiple processes in the 1940's? I'd be interested to see the evidence of that. File systems in the 1940's? Ditto.

Yes, 1940's. The 1950's machines were designed in the late 1940's. I said the problem has been around that long, not a particular computer.

Every approach to this problem from hardware locking to read/write/read to software has been tried to no avail. Passing all requests through one "system" works best.
Stan James
(instanceof Sidekick)
Ranch Hand

Joined: Jan 29, 2003
Posts: 8791
I deleted my prior intercalation (word of the day e-mail today!) but I still wonder if "synchronize" in the original post meant timing or just writing Java and reading C++.

If you really need to write from one process and read from another at the same time, asynchronous message queues might be a better choice than files. JMS or native WebSphere MQ (formerly MQ-Series) on the Java end, native MQ APIs on the C++ end. In-memory pipes might be faster, but a lot less portable.

If you can let the writing process finish before the reading process opens the file, the writer can write to a temp name and rename it to the real name after close. The reader can poll the shared file area for new files with the real name format. It will only find complete files.

In my company at least, the solution up through the 50s was to write to a 3/4 tape, dismount it from the writing system, carry it to the reading system and mount it up for read. I suppose they could have done asynchronous queues with punch cards - carry a hand full over at a time. Anybody actually encountering this problem in those days was pretty sophisticated.


A good question is never answered. It is not a bolt to be tightened into place but a seed to be planted and to bear more seed toward the hope of greening the landscape of the idea. John Ciardi
Edward Harned
Ranch Hand

Joined: Sep 19, 2005
Posts: 291

Well Stan, you're obviously a Geezer Geek. Any modern programmer knows that you don't use magnetic tape for a job that can be done faster and easier with paper tape.
Pat Farrell
Rancher

Joined: Aug 11, 2007
Posts: 4646
    
    5

we geezer geeks used to swap to high speed paper tape

To the OP, let the OS do it. Or on good operating systems, map the file into memory and use a semaphore. Dijkstra's paper was published about 1974, but multi-programming predated that by a long time.

Now I'll have to go search for the origin and ancient runes
Aji Prasetyo
Ranch Hand

Joined: Jun 13, 2007
Posts: 65
Thank you so much. But I am afraid I only understand about 10-20 percents of the solution you guys offered. Looks like I have a lot to study.

My problem about synchronizing is not about timing, but plainly about reading and writing of data queue. In my program there will be user commands that need to be executed by the C++. So the commands will be put into some kind of queue and C++ will executed it one by one accordingly.

This time, could somebody provide me with a sample code? Because it helps me a lot to understand.

I am sorry for my ignorance.
Pat Farrell
Rancher

Joined: Aug 11, 2007
Posts: 4646
    
    5

I'm not sure I follow you here. Java programs can use 'exec' for start up any program, written in C++ or perl or shell or whatever you like.

Its trivial for one program to write out a specific file name, and for another to read it. No matter what languages you write either part.

If you start one, write the file, start another and read the file, I don't see any synchronization needed.

If both programs are running all the time, the usual pattern is to agree on a file name, one program writes to a temp file, when it is complete, rename it to a agreed upon name, the second loops napping and looking for the file. When it finds it, it also renames it to another temp name, so that it can read it; and if needed, the first can create a second file.

If you really need a queue, you can use things like pipes, or records in a database, or many other patterns.
Stan James
(instanceof Sidekick)
Ranch Hand

Joined: Jan 29, 2003
Posts: 8791
For dead simple, if the programs run at the same time you could skip files and use sockets. One program writes commands, the other reads and puts items in an internal queue. The receiver has one thread that can keep up with reading and putting into the queue and another thread (or ten) that gets them out of queue before the queue takes all of memory. Java has suitable queues and threading in the library; don't know about C++.

The message queuing I mentioned above does much the same thing with seriously robust middleware.

I worked with a language called EASEL (now ESL) that integrated new modules by starting them much like runtime exec() and communicating through the new process's stdout and stdin. As a plus you could test the whole module from the command line. It sounds fragile but I don't think it ever really broke.
 
 
subject: How to: synchronizing a file access with other programming languages.
 
Similar Threads
servlet 4b checking password in VideoLoginServlet
How can i read data from file
about odbc connection
Toolbar Question
Including files in a jsp from different machine