Determining Termination Condition for threads

have written a multi-threaded Java application which reads a bunch of .jar files from a directory. This application spawns multiple threads and each threads reads bunch of jar files. I'm having trouble identifying the stopping condition for this application. How can i identify that all the files have been read?

The following is my code:

import java.io.*;
import java.util.Enumeration;
import java.util.jar.*;
import java.util.zip.ZipEntry;
import java.util.zip.ZipFile;
import java.util.zip.ZipException;
import java.io.FilenameFilter;

public class ArchiveFileTest implements Runnable {
	 private static boolean stopAll = false;
	 private static int threadNumber = 0;
	 private int myNumber = 0;

public ArchiveFileTest () {
			myNumber = threadNumber;
			threadNumber++;
	 }

public static boolean setStopAll () {
			return setStopAll(true);
	 }

public static boolean setStopAll (boolean b) {
			stopAll = b;
			return stopAll;
	 }

public static String[] listFiles (File parentDir,final String ext1,final String ext2,final String ext3,final String ext4) {
			String allFiles[] = parentDir.list(new FilenameFilter() { 
				 public boolean accept(File pDir, String fName) {
						if (fName.endsWith("."+ext1) || fName.endsWith("."+ext2) || fName.endsWith("."+ext3) || fName.endsWith("."+ext4)) return true;
						else return false;
				 }
			});
			for (int i=0; i<allFiles.length; i++)
				 allFiles[i] = parentDir.getAbsolutePath() + File.separator +  allFiles[i];
			return allFiles;
	 }

public ZipFile getMyZipFile (File parentDir) {
			String fn[] = listFiles(parentDir, "jar", "zip", "war", "rar");
			int fileNum = myNumber % fn.length;
			ZipFile zFile = null;
			for (int i=0; i><fn.length; i++) {
				 String jFile = fn[(fileNum + i)%fn.length];
				 try {
						zFile = new ZipFile(jFile);
						break;
				 } catch  (IOException e) {
					 setStopAll();
				 }
			}
			return zFile;
	 }

public void doStuff() throws Exception {
			File dName = new File("/home/sqatest/chander/sample-files");
		  final int N_TIMES = 15;
			final int N_FILES = 500;
		  int counter = 0;
			int fCount  = 0;
			if (!dName.isDirectory() || !dName.exists()) {
				 System.err.println("The parent directory given should point to an existing directory...");
				 setStopAll();
				 return;
			}
		  while (counter >< N_TIMES) { 
				 ZipFile zipFile = getMyZipFile(dName);
				 if (zipFile == null) {
						System.err.println("No zip file entry for the Thread-" + myNumber);
						break;
				 }
				 try {
						Enumeration <? extends ZipEntry> zipEntries = zipFile.entries(); 
						fCount = 0;
						ZipEntry ze = null;
						while (zipEntries.hasMoreElements()) {
							 ze = zipEntries.nextElement();

if (ze.isDirectory()) continue; // if it is a directory go to next entry

InputStream is = zipFile.getInputStream(ze); 
							 fCount++;
							 int readCount = 0;
							 try {
									while(is.read((new byte[50])) != -1 && readCount != 200) readCount++;
									System.out.println("Successfully Read " + zipFile.toString());
									
									//is.close();
							 } catch (IOException e) {
									e.printStackTrace();
							 }
							 if (fCount == N_FILES) break;  // read maximum of N_FILES
						}
						if (stopAll) break;
				 } catch (Exception e) {
					 e.printStackTrace();
				 } finally {
						counter++; 
				 }
		  } 
	 }

public void run () {
			try {
				 doStuff();
			} catch (IOException e) {
				 e.printStackTrace();
				 setStopAll();
			} catch (Exception e) {
				 e.printStackTrace();
			}
	 }

public static void main (String[] args) throws Exception { 
		  final int MAX_THREADS = 500;
		  final int MAX_HOLDING_THREADS = 5;
			int loopCount = 0;
			Thread mainThread = Thread.currentThread();
			for (int m=0; ; m++) {
				 Thread t[] = new Thread[MAX_HOLDING_THREADS];
				 for (int n=0; n<t.length; n++) {
						t[n] = new Thread(new ArchiveFileTest());
						t[n].start();
						if ((m+1)*(n+1)==MAX_THREADS) {
							 System.out.println("\n" + MAX_THREADS + " reached... \nMain Sleeping for some mins..."); 
							 loopCount++;
							 try {
									t[n].join();
									System.out.println("\nMain is back... (" + loopCount + ")");
							 } catch (InterruptedException e) {
									e.printStackTrace();
									setStopAll();
							 }
							 m = 0;
						}
				 }
			}
	 }
}

>

Hi

Do you read all jar files of the given directory?
Why do you use multiple threads for this?

Don't use Thread and Runnable but an ExecutorService and Callable. You can use Executors.newFixedThreadPool(int) to create an ExecutorService, then submit the Callable objects. This gives you back Future objects that have get methods that a) have a return value, and b) can throw exceptions. The ExecutionException wraps the exception thrown from the Callable's call() method; you can retrieve it with the getClause() method. Instead of joining on the threads you get the Futures.

Also, your loop is written in such a way that you have no concurrency. You create a thread, then join on it within the same loop iteration. You should instead have two loops after each other - one to start the work, one to join / get.

I have simplified my code. In this code, i create multiple threads and each thread will read all the jars in the directory. I created this to stress test an application. I wanted feedback on the following code, if it can be done in a better way.

Also, one question i had was that my main thread exits before other threads have completed. Is there a way i can make my main thread to wait until all the threads have completed?

import java.io.*;
import java.util.Enumeration;
import java.util.jar.*;
import java.util.zip.ZipEntry;
import java.util.zip.ZipFile;
import java.util.zip.ZipException;
import java.io.FilenameFilter;

public class ArchiveFileTest implements Runnable {
	private static boolean stopAll = false;
	private static int threadNumber = 0;
	private int myNumber = 0;

public ArchiveFileTest () {
		myNumber = threadNumber;
		threadNumber++;
	}

public static boolean setStopAll () {
		return setStopAll(true);
	}

public static boolean setStopAll (boolean b) {
		stopAll = b;
		return stopAll;
	}

public static String[] listFiles (File parentDir,final String ext1,final String ext2,final String ext3,final String ext4) {
		String allFiles[] = parentDir.list(new FilenameFilter() { 
			public boolean accept(File pDir, String fName) {
				if (fName.endsWith("."+ext1) || fName.endsWith("."+ext2) || fName.endsWith("."+ext3) || fName.endsWith("."+ext4)) return true;
				else return false;
			}
		});
		for (int i=0; i<allFiles.length; i++)
			allFiles[i] = parentDir.getAbsolutePath() + File.separator +  allFiles[i];
		return allFiles;
	}

public ZipFile getMyZipFile (File parentDir) {
		String fn[] = listFiles(parentDir, "jar", "zip", "war", "rar");
		int fileNum = myNumber % fn.length;
		ZipFile zFile = null;
		for (int i=0; i><fn.length; i++) {
			String jFile = fn[(fileNum + i)%fn.length];
			try {
				zFile = new ZipFile(jFile);
				break;
			} catch  (IOException e) {
				setStopAll();
			}
		}
		return zFile;
	}

public void doStuff() throws Exception {
		File dName = new File("C:\\data-files");
		final int N_TIMES = 1;
		final int N_FILES = 500;
		int counter = 0;
		int fCount  = 0;
		if (!dName.isDirectory() || !dName.exists()) {
			System.err.println("The parent directory given should point to an existing directory...");
			setStopAll();
			return;
		}
		while (counter >< N_TIMES) { 
			//ZipFile zipFile = getMyZipFile(dName);
			String[] files = listFiles(new File("C:\\data-files"), "jar", "zip", "war", "rar");

for (String file : files) {
				ZipFile zipFile = new ZipFile(file);
				if (zipFile == null) {
					System.err.println("No zip file entry for the Thread-" + myNumber);
					break;
				}
				try {
					Enumeration <? extends ZipEntry> zipEntries = zipFile.entries(); 
					fCount = 0;
					ZipEntry ze = null;
					while (zipEntries.hasMoreElements()) {
						ze = zipEntries.nextElement();

if (ze.isDirectory()) continue; // if it is a directory go to next entry

InputStream is = zipFile.getInputStream(ze); 
						fCount++;
						int readCount = 0;
						try {
							while(is.read((new byte[50])) != -1 && readCount != 200) 
								readCount++;

System.out.println("Successfully Read " + ze.toString());

//is.close();
						} catch (IOException e) {
							e.printStackTrace();
						}
						if (fCount == N_FILES) break;  // read maximum of N_FILES
					}
					if (stopAll) break;
				} catch (Exception e) {
					e.printStackTrace();
				} finally {
					counter++; 
				}

}

} 
	}

public void run () {
		try {
			doStuff();
		} catch (IOException e) {
			e.printStackTrace();
			setStopAll();
		} catch (Exception e) {
			e.printStackTrace();
		}
	}

public static void main (String[] args) throws Exception { 
		final int MAX_THREADS = 1;
				
		for(int i=0; i < MAX_THREADS; i++)
		{
			Thread t = new Thread(new ArchiveFileTest());
			t.start();
		}
				
	}
}

chander shivdasani wrote:Also, one question i had was that my main thread exits before other threads have completed. Is there a way i can make my main thread to wait until all the threads have completed?

Join on them. But you don't need to wait - since the threads are not daemon threads the JVM will not exit before all of them have ended.

So you mean something like this?

Yes.