Strange behavior of Runtime.getRuntime().exec(command)

[Nam Ha Minh]

I am calling FFmpeg from within Java like this:

		String command = "ffmpeg -i Screen%03d.jpeg -r 24 video.mp4";

		Process process = Runtime.getRuntime().exec(command);


            //BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream()));
            BufferedReader reader = new BufferedReader(new InputStreamReader(process.getErrorStream()));


            String line = null;
            while ((line = reader.readLine()) != null) {
                System.out.println(line);
            }

            process.waitFor();

I experienced these issues which I don't really understand why:

- If I try to read the process' output from process.getInputStream(), then the program seems to be blocked at the while statement. If I read from the error stream (process.getErrorStream()) then I can read the process' output perfectly. My question is, why don't some program send its output to the input stream but error stream, like the FFmpeg program? How can we know which stream to which the program sends output?

- If I call process.waitFor(), then the program seems to be waiting forever. Why?

Thanks for your insights on these issues.

[Paul Clapham]

Nam Ha Minh wrote:My question is, why don't some program send its output to the input stream but error stream, like the FFmpeg program? How can we know which stream to which the program sends output?

Because the programmers of that program decided to send the output to stderr. But do you need to know why? That's what they did, and you have to deal with it.

You can find out whether output goes to stderr or stdout by experimentally running the program and redirecting one or the other stream and seeing what happens, but again you don't need to know this either. Read on to find out why.

If I call process.waitFor(), then the program seems to be waiting forever. Why?

Because it's waiting for you to consume the output which it sent to stdout or stderr. Those streams don't have an infinitely large buffer, so when the buffer fills and isn't emptied by the receiver (your code), the writer of the stream (the other application) will naturally wait. You should read the classic article When Runtime.exec() won't and implement all of its recommendations.

[Nam Ha Minh]

Thanks Paul for your insight,

The Java doc of waitFor() method does not tell that I have to consume the process' output stream when calling waitFor(). So I thought the waitFor() method will block the program until the process finished without writing any additional code. So is my thought wrong?

Regards.

[Jeff Verdegan]

Nam Ha Minh wrote:Thanks Paul for your insight,

The Java doc of waitFor() method does not tell that I have to consume the process' output stream when calling waitFor(). So I thought the waitFor() method will block the program until the process finished without writing any additional code.

It doesn't say you have to consume it, but it also doesn't say you don't have to. It just says it waits for the process to finish, and that is true; that's what it does. However, if a process is trying to write to a buffer, and that buffer is full, the process can't proceed beyond that point. The process you're executing is probably written something like this:

main() {
  read input
  process it
  write output
}

It has no way of knowing that you are not interested in its output so it's going to patiently sit there at the write line waiting for the buffer to empty. Until that happens, that process is not finished.

So is my thought wrong?

I would think that the behavior you've seen, coupled with the explanation you received, would have answered that question before you asked it.

[Nam Ha Minh]

Thanks Jeff. I got it now.

[Mike Simmons]

Note that in terms of the docs, while the docs for waitFor() didn't explain your problem, the docs for the Process class did. You need to look at more than just the individual method doc. The overall class documentation is also important.

Also, the classic article "When Runtime.exec() won't" is an excellent resource, but also rather old at this point. It was written before the ProcessBuilder class was introduced. This is now the recommended replacement for Runtime.exec(). It offers a few improvements. Most of the issues discussed in the original article are still entirely relevant, however - so time spent reading that article is not wasted. But it may be worth your while to replace Runtime.exec() with ProcessBuilder.

[Nam Ha Minh]

Thanks Mike for your insight also

[Paul Witten]

Paul Clapham wrote: You should read the classic article When Runtime.exec() won't and implement all of its recommendations.

Hohoho, those little buggers naming the param to exec() "command" and then MS making "dir" not a command (not an executable.) Talk about Devil in the Details. That was a great article.

[Tony Docherty]

Yes it is a great article. It saved my skin many a year ago and I've also been recommending it ever since.

Not that this will effect the operation of the code but I believe the current recommended way to create a Process is to use ProcessBuilder rather than Runtime.

[Mike Simmons]

Um, yes. Good idea.

[Richard Tookey]

Tony Docherty wrote:Yes it is a great article. It saved my skin many a year ago and I've also been recommending it ever since.

Not that this will effect the operation of the code but I believe the current recommended way to create a Process is to use ProcessBuilder rather than Runtime.

When ProcessBuilder was introduced in 1.5 Runtime.exec() was re-written to use ProcessBuilder behind the scenes. Since ProcessBuilder provides a richer API, unless one needs compatibility with 1.4 or earlier, it makes sense to use ProcessBuilder rather than Runtime.exec().

[Mike Simmons]

Also, one could use ProcessBuilder rather than Runtime.exec(). Just a thought.

[Richard Tookey]

Mike Simmons wrote:Also, one could use ProcessBuilder rather than Runtime.exec(). Just a thought.

Sorry Mike - went straight over my head I'm afraid. What am I missing?

[Mike Simmons]

I was feeling like we're in a loop. I recommended using ProcessBuilder here a week ago, above.

[Richard Tookey]

Mike Simmons wrote:I was feeling like we're in a loop. I recommended using ProcessBuilder here a week ago, above.

And I was pointing out that using Runtime.exec() is to actually use ProcessBuilder (though with a more restrictive API).