Is there any way to execute a Linux command from Java without fork-ing?

[Teddy Wang]

We had code which basically looks like this:

Process proc = Runtime.getRuntime().exec(command);

Unfortunately this runs on an app server which has min/max heap set very high (2.2G) and when this code is hit, we get an out of memory condition:

java.io.IOException: java.io.IOException: Cannot allocate memory
at java.lang.UNIXProcess.<init>(UNIXProcess.java:148)
at java.lang.ProcessImpl.start(ProcessImpl.java:65)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:451)
at java.lang.Runtime.exec(Runtime.java:591)
at java.lang.Runtime.exec(Runtime.java:464)

This is obviously due to the fact that Java is forking the Linux (app server) process and we don't have enough memory (both physical and swap) to support such activity. Is there an alternative method to call external commands without incurring the cost of the fork?

Thanks!

[Ernest Friedman-Hill]

You could have a separate program act as a "fork server" -- a small program that listened on a socket and executed whatever strings it received as shell commands. Then your big code could talk to the little one whenever it needed to run something. With a little bit of stream gymnastics, you should even be able to connect up to the stdio of the child.

[Rob Brueckmann]

Hey Teddy...

Did you come up with a solution to this? We just recently started testing Weblogic 10 in a 64-bit environment...the same code has been running beautifully for 4 years on our 32-bit Linux systems in Weblogic 8 but the code run in 64-bit Linux on Weblogic 10 is encountering the same exception you mention below after a few hours of usage and then just dies.

I don't really understand the guy who replied to your issue's comments about a fork server...so I was curious if you came up with a solution and if so, what you ended up doing and also what type of environment you're running this code in, just for comparison's sake?

Thanks so much!
Rob

Originally posted by Teddy Wang:
We had code which basically looks like this:

Process proc = Runtime.getRuntime().exec(command);

Unfortunately this runs on an app server which has min/max heap set very high (2.2G) and when this code is hit, we get an out of memory condition:

java.io.IOException: java.io.IOException: Cannot allocate memory
at java.lang.UNIXProcess.<init>(UNIXProcess.java:148)
at java.lang.ProcessImpl.start(ProcessImpl.java:65)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:451)
at java.lang.Runtime.exec(Runtime.java:591)
at java.lang.Runtime.exec(Runtime.java:464)

This is obviously due to the fact that Java is forking the Linux (app server) process and we don't have enough memory (both physical and swap) to support such activity. Is there an alternative method to call external commands without incurring the cost of the fork?

Thanks!

[Ernest Friedman-Hill]

Originally posted by Rob Brueckmann:

I don't really understand the guy who replied to your issue's comments about a fork server.

Just run another server with a small heap; a trivial 10-line program that sits in loop, listens on a socket for UNIX command lines, and executes them. When you need to run a command, send it to this second server to execute by connecting to the socket and printing the command line.

[vipin jos]

I am sorry, but I did not understand that. As per the first post - it is said that "we don't have enough memory (both physical and swap) to support such activity".

So how do we get over the physical/ swap memory restriction through java - by writing a new server or otherwise?

My understanding is that OS is not able to start the new process due to lack of swap. Correct me if I am wrong.
We have faced issues while starting a new Tomcat server on a heavily loaded server. It would say something like "Not enough memory to allocate heap" in the log file and fail.

One solution could be to increase your swap (which obviously you might have tried and failed.

[Ernest Friedman-Hill]

Ahhh. I think everybody in this thread (aside from Teddy and me) doesn't understand what fork() does -- i.e., how processes are launched on UNIX. Here's a quote from an article at Sun's web site that explains it and explains the problem:

Historical Background and Problem Description

Traditionally, Unix has had only one way to create a new process: using a fork() system call, often followed by an exec() system call. The fork() call makes a copy of the entire parent process' address space, and exec() turns that copy into a new process.

(Note: In the Solaris OS, the term swap space is used to describe a combination of physical memory and disk swap space configured for the system. However, with other Unix systems this term may mean swap space on disk, also known as backing store. To avoid any confusion, I'll use the term Virtual Memory (VM) to mean physical memory plus disk swap space.)

Generally, the fork/exec method has worked quite well. However, it has disadvantages in some cases, such as running out of memory without a good reason and poor fork performance.

Out of Memory: For a large-memory process, the fork() system call can fail due to an inadequate amount of VM, because fork() requires twice the amount of the parent memory. This can happen even when fork() is immediately followed by an exec() call that would release most of that extra memory. When this happens, the application will usually terminate.

For example, suppose a 64-bit application is consuming 6 gigabytes (Gbytes) of VM at the moment, and it needs to create a subprocess to run the ls(1) command. The parent process issues a fork() call that will succeed only if there is another 6 Gbytes of VM available at the moment. If the system doesn't have that much VM available (which is a frequent situation), fork() will fail with ENOMEM. Obviously, the ls(1) command doesn't need anywhere near 6 Gbytes of memory to run, but fork() doesn't know that.

Not only applications, but also Sun's own tools can suffer from the same problem. For example, the following Sun RFE (request for enhancement) has been filed for dbx: "4748951 dbx shell should use posix_spawn() for non-builtin commands rather than fork(2)".

RFE 4748951 came about when a customer's utility invoked dbx to read a huge core file using a script that also needed to run a cut(1) command from within dbx. They got a cannot fork - try again error message causing dbx to abort. An investigation revealed that dbx used fork/exec to execute that tiny cut(1) command and ran out of VM during the fork() call.

The Solaris Java Virtual Machine (JVM) is also suffering from the same problem currently, as described in this Sun RFE: "5049299 Use posix_spawn, not fork, on S10 to avoid swap exhaustion".

The article goes on to talk about a special Solaris function that can be used to work around the problem. But maybe you understand my proposed solution now: run a second process with a tiny memory footprint, and let it call fork() for you. The child program thus created will have a small memory footprint too, and will (hopefully) fit into memory, even though allocating a copy of the large server process would fail.

[vipin jos]

Thanks Ernest. This is a very new information for me.