Process waits forever when using Stringbuffer with more than 2000 strings

[Vivek Jain]

Hi,
I am using ProcessBuilder to invoke an external command which will provide me the output. And I'm supplying input through String Buffer. My program works fine for 2934 strings within StringBuffer. However, the moment my stringbuffer size goes beyond 2934, the program hangs & waits forever.

Below is my code.

    
import java.io.*;
import java.util.Properties;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

class check
{
	public static void main(String[] args) 
	{

		Process process = null;
		ProcessBuilder processBuilder = null;		
						
		String line = null;

		InputStream inStream = null;
				
		boolean result = false;
				

		StringBuffer lst = new StringBuffer();

		lst.append("string1\n");
		lst.append("string2\n");
		lst.append("string3\n");
		lst.append("string4\n");
		lst.append("string5\n");
		lst.append("string6\n");
		// ....
		// Like this add more than 2000 strings in string buffer.

		try {		

			processBuilder = new ProcessBuilder("/bin/sh", "-c", "Externalutitilty");
			

			process = processBuilder.start();
			
			OutputStream outStream = process.getOutputStream();         
			Writer writer = new OutputStreamWriter(outStream);     
			writer.write(lst.toString());
			
			writer.flush();
						
			writer.close();
						
	                int exitVal = process.waitFor();

			if (exitVal == 0) {		    	
				inStream = process.getInputStream();	
				
				BufferedReader reader = new BufferedReader(new InputStreamReader(inStream));
				
				while ((line = reader.readLine()) != null) {
					if (line != null && !line.equalsIgnoreCase("")) {
						System.out.println("line="+line);					
					} 
				}
			} 

		} catch (IOException e) {			
			System.out.println(e);
		} catch (InterruptedException e) {
			//log.error(METHOD_NAME + ": InterruptedException: ", e);
			System.out.println(e);
		} finally {
			try {
			    if (process != null) {
			    	process.destroy();		
			    }			    
			} catch (Exception e) {
				//log.error(METHOD_NAME, e);
			}
		}			
	}
}

Can anyone tell me what could be the issue? Is there any limit in String Buffer? If so, how to resolve it?

PS: I also tried specifying large capacity such as 99999 in constructor of String buffer but still no luck....

Thanks!

[Campbell Ritchie]

Are you really writing of .append("string"); 2000 times? That might take the method size over the maximum allowed.

And this is too difficult a question for beginners. Moving.

[Campbell Ritchie]

And what are you doing with the input and error streams from the Process? You know you have to "drain" them both, otherwise the Process may suffer deadlock.

[Vivek Jain]

Yes....I'm in fact giving input of 9000 strings within String Buffer.

Using

inStream = process.getInputStream(); 

, I'm actually capturing the output of the external utility which I am calling using ProcessBuilder.

Can you please help?

It looks like the buffer cannot accommodate much data. I just tried replacing String Buffer with Array List but the problem stays the same. So, it could be related to the output stream being not able to handle much data.

ok....I just noticed that the program works fine for small chunk of strings say 500 strings. However, if I use more than this limit, it just hangs at line process.waitFor().

What could be the resolution?

Thanks!

[Vivek Jain]

Hi,
After doing more analysis, I noticed that the process is hanging because the I/O buffer is full...And it should be released by reading it.

If I try the below code, it works fine...But again it also has the limitation of processing around 8000 strings.....and thats not the right approach since I'm reading it before even knowing whether the process has finished it or not....So is there any other better soln???

    
 import java.io.*;  
 import java.util.Properties;  
 import java.util.regex.Matcher;  
 import java.util.regex.Pattern;  
   
 class check  
 {  
     public static void main(String[] args)   
     {  
   
         Process process = null;  
         ProcessBuilder processBuilder = null;         
                           
         String line = null;  
   
         InputStream inStream = null;  
                   
         boolean result = false;  
                   
   
         StringBuffer lst = new StringBuffer();  
   
         lst.append("string1\n");  
         lst.append("string2\n");  
         lst.append("string3\n");  
         lst.append("string4\n");  
         lst.append("string5\n");  
         lst.append("string6\n");  
         // ....  
         // Like this add more than 2000 strings in string buffer.  
   
         try {         
   
             processBuilder = new ProcessBuilder("/bin/sh", "-c", "Externalutitilty");  
               
   
             process = processBuilder.start();  
               
             OutputStream outStream = process.getOutputStream();           
             Writer writer = new OutputStreamWriter(outStream);       
             writer.write(lst.toString());  
               
             writer.flush();  
                           
             writer.close();  

			   
		inStream = process.getInputStream();      

		BufferedReader reader = new BufferedReader(new InputStreamReader(inStream));  

		while ((line = reader.readLine()) != null) {  
		if (line != null && !line.equalsIgnoreCase("")) {  
		 System.out.println("line="+line);                     
		}   
		}  

             int exitVal = process.waitFor();  

   
             if (exitVal == 0) {               
		// Everything is already read...
             }   
   
         } catch (IOException e) {             
             System.out.println(e);  
         } catch (InterruptedException e) {  
             //log.error(METHOD_NAME + ": InterruptedException: ", e);  
             System.out.println(e);  
         } finally {  
             try {  
                 if (process != null) {  
                     process.destroy();        
                 }                 
             } catch (Exception e) {  
                 //log.error(METHOD_NAME, e);  
             }  
         }             
     }  
 }  

thanks!

[Rob Spoor]

Someone here hasn't read the Javaworld article "When Runtime.exec won't"

[Vivek Jain]

Rob,
I've read the article at Javaworld. Appreciate if you can tell me the solution for my case?

Thanks!

[Rob Spoor]

Try reading the error stream as well, in a different thread.

[Vivek Jain]

Hi Rob,
Still no luck....Tried reading both error & input streams before waiFor....it hangs....When I do the Ctrl-C, it gives below error:

-bash-3.2$ /usr/java/jdk1.6.0_10/bin/java myExec
java.io.IOException: Broken pipe
        at java.io.FileOutputStream.writeBytes(Native Method)
        at java.io.FileOutputStream.write(FileOutputStream.java:260)
        at java.io.BufferedOutputStream.write(BufferedOutputStream.java:105)
        at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:202)
        at sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:263)
        at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:106)
        at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:116)
        at java.io.OutputStreamWriter.write(OutputStreamWriter.java:203)
        at java.io.Writer.write(Writer.java:140)
        at myExec.main(myExec.java:2990)

And if I look at the line number 2990 from above error, it seems like the issue is somewhere else.....Basically, when I'm throwing the string as an input to this external utility, it is hanging there itself....

			process = processBuilder.start();
			
			// Write the data from StringBuffer to process's output stream
			OutputStream outStream = process.getOutputStream();         
			Writer writer = new OutputStreamWriter(outStream);     
			writer.write(lst);

Code hangs at Line 2990 which is writer.write(lst);

Can you please help fixing the issue?

Appreciate your help!!!