I/O binary piping ? Please Help !

[jay vas]

Hi Everybody :

Im trying to extract the binary output of a process, and then
feed that as input into another process using runtime.exec().

Is this possible ? I know that you can send a String as input to runtime.exec(), but what happens when you want to send a String of binary
data as input to a process ?

Heres the code ...

//Blob b is obtained from the database.
//sysprog is an arbitrary system program which accepts binary data...

Process p = runtime.exec(b.getBinaryStream() + " | " + sysproc )
OutputStream s = p.getOutputStream();
runtime.exec(s.getOutputStream() + " | " + sysprog )

Anyways...Im totally lost about how to do this so any suggestions would really be appreciated !!!

J

[Joe Ess]

what happens when you want to send a String of binary data as input to a process ?

b.getBinaryStream() just gets you a stream, it doesn't get you the data. You could either read in the stream, convert it to a String then invoke runtime.exec with the results (which would cause havoc with binary data), or if your programs accept input through system input, you could start sysproc with runtime.exec(), then get the output stream of that process and write your blob to it

[jay vas]

Thanks!!!

One last question : Can you give me an example of
how you would use those functions ...just so I can run it
and see how they would be used in an actual program ? Im a
little bit unfamiliar with I/O streams and pipes, so the concept
of an InputStream and an OutputStream is a bit abstract to me.

[Jim Yingst]

From the posting time I believe this is a response to something I had briefly posted and then deleted. Sorry, I deleted my earlier post because I realized I'd misunderstood what you were asking for, making it more complex than necessary. Joe's earlier reply addressed your question better.

Here's a simple way to copy from the Blob to the Process though:

        InputStream in = b.getBinaryStream();
        OutputStream out = p.getOutputStream();
        byte[] buffer = new byte[8096];
        for (int n; (n = in.read(buffer)) != -1;) 
            out.write(buffer, 0, n);
        out.close();
        in.close();

Note - no Strings are used here. At all. For binary data, it's usually a mistake to try to convert to a String unless you know that it's supposed to represent text. And for this problem, we don't really care whether it's a String or not.

I recommend studying the Java tutorial if you want to get more background on the IO classes.

I've also pretty much omitted error handling for simplicity. Normally I'd prefer something like

        InputStream in = b.getBinaryStream();
        try {
            OutputStream out = p.getOutputStream();
            try {
                byte[] buffer = new byte[8096];
                for (int n; (n = in.read(buffer)) != -1;)
                    out.write(buffer, 0, n);
            } finally { out.close(); }
        } finally { in.close(); }

...and you still need to catch exceptions at a higher level, but the details of that will depend on the context.
[ January 31, 2006: Message edited by: Jim Yingst ]

[jay vas]

Hi and thanks again :

I guess Im still a little confused. Shouldnt you be writing to the inputStream?

The general process Im trying to implement is :

data_0 -> Process_1 -> data_1 -> Process_2 -> output_2 -> ...

Now, if the inputStream to a process is the stream of data which will be used as input to that process, wouldnt it be logical to do something like :

//assume p1 has succesfully excuted :
p2.writeToInputStream(p1.getOutputStream());
p2.execute();
p3.writeToInputStream(p2.getOutputStream());
p3.execute();
//etc..


?

[Layne Lund]

What operating system are you running this on? If it is a *nix platform, then you can easily do this with a shell script in one line:

process1 | process2 | process3 ...

The most difficult part is to figure out how process1 will input the first set of data. It can possibly come from a command-line argument (like a file name) or from stdin. In fact, most *nix utilities allow for both.

I think you can do this with a Windows batch file in a single line as well. As much as I enjoy programming in Java, sometimes it is not the right tool for the job. In this case, I think shell/batch scripts might be more useful.

Layne
[ January 31, 2006: Message edited by: Layne Lund ]

[jay vas]

Thanks Again :


First off, I REALLY am appreciating youre help on this matter.

Now --- here is a better explanation of what Im trying to do :

I have a binary data file, which represents spectral data obtained by mathematical software, which is enormous, and stored in a mysql database.

Im using a program called NmrPipe. NMRPipe is usually utilized with unix pipes, and users typically write scripts.

nmrpipe -in a.fid | nmrpipe -fn FFT | nmrpipe -out a2.fft

Now, why dont i just run a script?

Here is the scenario : a.fid is a binary file stored in a database, so the option of using a file path is not available (writing it to a temp file is not a good idea).

So I want to stream the binary file from the database, into
a java Binary stream object, and then directly into a runtime.exec() process command. Here is what I would like to do :

Step 1 : First, I execute a query (using runtime.exec()) for a raw blob.

mysql --user=root -N -e "SELECT Blob FROM Blob_Table WHERE Blob_ID=0;"

Step 2 : Acquire the binary result as a binary stream...

Step 3 : set the acquired stream (step 2) as input to a new process.

Step 4 : runtime.exec that new process.

So basically, since I want to execute the query as a system process, I believe it should be possible to stream its output directly into another system process using java.

I dont want to use a shell script, because Im trying to build these scripts dynamically in java, so that users of the program do not have
to maintain text logs of what scripts they ran.

Does this make sense ?

Thanks !
-j

PS My boss said he'd send you a bottle of youre favorite wine when this is all over.

[Joe Ess]

Originally posted by JAY asasd:

I guess Im still a little confused. Shouldnt you be writing to the inputStream?

One writes to OutputStreams. One reads from InputStreams. I admit that it seems backwards in certain situations like the Process class.

Now, if the inputStream to a process is the stream of data which will be used as input to that process, wouldnt it be logical to do something like:

Again, getOutputStream() returns a stream. You still have to move the data in the manner Jim demonstrated. As Jim suggested, spend a little time going over the tutorial. Knowing the basics will help you a great deal in this task.

And as for the bottle of wine, I don't know when Jim, Layne and I will be able to sit down together to enjoy it. Consider making a small donation to the JavaRanch.