I am not sure whether I should post it here or in the JNI forum. but I am not asking about native code, so I guess I am posting it here.
I have a stream of data, I write it into a file using FileWriter(). The problem is, the data in the file is suppose to be read by C++ application. How do you synchronize a file access with something which is not java (C++)?
If anybody familiar with the case, please help me.
Obviously you're new to the world of computing. This problem has been around since the 1940's.
The solution to multiple processes (or processors) accessing the same file at the same time is to use a file/data base management system. The "system" owns the file/table etc. and controls all access to it.
Originally posted by Edward Harned: Obviously you're new to the world of computing. This problem has been around since the 1940's.
Multiple processes in the 1940's? I'd be interested to see the evidence of that. File systems in the 1940's? Ditto.
Back to the original poster's question. Do you really need to use an actual file? What about a pipe or socket? Presumably, your reading process can successfully open the file for read, while the writing process is writing it. But I guess your difficulty is that arbitrary chunks of data get written to it, and sometimes these chunks constitute a part-complete update. Can you just add some End-Of-Record marker to the end of each complete update? Can you then just poll the file (or, better, pipe or socket)?
If it's more complicated than that, then maybe you need a transaction service. You'd get one if you used an RDBMS, but you can also get ones outside of that. [ October 16, 2007: Message edited by: Peter Chase ]
Betty Rubble? Well, I would go with Betty... but I'd be thinking of Wilma.
I deleted my prior intercalation (word of the day e-mail today!) but I still wonder if "synchronize" in the original post meant timing or just writing Java and reading C++.
If you really need to write from one process and read from another at the same time, asynchronous message queues might be a better choice than files. JMS or native WebSphere MQ (formerly MQ-Series) on the Java end, native MQ APIs on the C++ end. In-memory pipes might be faster, but a lot less portable.
If you can let the writing process finish before the reading process opens the file, the writer can write to a temp name and rename it to the real name after close. The reader can poll the shared file area for new files with the real name format. It will only find complete files.
In my company at least, the solution up through the 50s was to write to a 3/4 tape, dismount it from the writing system, carry it to the reading system and mount it up for read. I suppose they could have done asynchronous queues with punch cards - carry a hand full over at a time. Anybody actually encountering this problem in those days was pretty sophisticated.
A good question is never answered. It is not a bolt to be tightened into place but a seed to be planted and to bear more seed toward the hope of greening the landscape of the idea. John Ciardi
we geezer geeks used to swap to high speed paper tape
To the OP, let the OS do it. Or on good operating systems, map the file into memory and use a semaphore. Dijkstra's paper was published about 1974, but multi-programming predated that by a long time.
Now I'll have to go search for the origin and ancient runes
Joined: Jun 13, 2007
Thank you so much. But I am afraid I only understand about 10-20 percents of the solution you guys offered. Looks like I have a lot to study.
My problem about synchronizing is not about timing, but plainly about reading and writing of data queue. In my program there will be user commands that need to be executed by the C++. So the commands will be put into some kind of queue and C++ will executed it one by one accordingly.
This time, could somebody provide me with a sample code? Because it helps me a lot to understand.
I'm not sure I follow you here. Java programs can use 'exec' for start up any program, written in C++ or perl or shell or whatever you like.
Its trivial for one program to write out a specific file name, and for another to read it. No matter what languages you write either part.
If you start one, write the file, start another and read the file, I don't see any synchronization needed.
If both programs are running all the time, the usual pattern is to agree on a file name, one program writes to a temp file, when it is complete, rename it to a agreed upon name, the second loops napping and looking for the file. When it finds it, it also renames it to another temp name, so that it can read it; and if needed, the first can create a second file.
If you really need a queue, you can use things like pipes, or records in a database, or many other patterns.
Joined: Jan 29, 2003
For dead simple, if the programs run at the same time you could skip files and use sockets. One program writes commands, the other reads and puts items in an internal queue. The receiver has one thread that can keep up with reading and putting into the queue and another thread (or ten) that gets them out of queue before the queue takes all of memory. Java has suitable queues and threading in the library; don't know about C++.
The message queuing I mentioned above does much the same thing with seriously robust middleware.
I worked with a language called EASEL (now ESL) that integrated new modules by starting them much like runtime exec() and communicating through the new process's stdout and stdin. As a plus you could test the whole module from the command line. It sounds fragile but I don't think it ever really broke.