How to return execution to main program only when all scheduled jobs are done?

[kuhu vora]

Hi,

I am trying to schedule some future jobs using ExecutorService. The Executor runs fine but I want next statement to process only when all the job in the scheduler are done. Can someone please help me out here?

My simple scheduler code:

public class Scheduler{
	public void createScheduler(int taskIntervalMins, int taskDurationHours,
			Runnable doTask) {
		final ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(1);

		final ScheduledFuture<?> scheduleTask = scheduler.scheduleAtFixedRate(
				doTask, 2, taskIntervalMins, MINUTES);
		scheduler.schedule(new Runnable() {
			public void run() {
				scheduleTask.cancel(true);
			}
		}, taskDurationHours, MINUTES);
	}
}

Calling the scheduler:

public class Test[
	public static void main(String[] args){
		final Runnable doTask = new Runnable() {
				public void run() {
					try {
						System.out.println("do something");
					} catch (Exception e) {
						e.printStackTrace();
					}
				}
			};
	}
	
	Scheduler s = new Scheduler();
	s.createScheduler(1,3,doTask);

        //this statement should only process after all tasks are done
        System.out.println("Output:");
}

*Edited:

This program runs fine for small tasks such as print statement but when I try to do some big Task the program prints the result before the execution of completed scheduled tasks. I think the issues is with Thread Synchronization. Can someone suggest how to make this work?

[Salil Wadnerkar]

The ExecutorService is shutdown using .shutdown() and .awaitTermination() calls.
Have a look at https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ExecutorService.html

[kuhu vora]

How does closing help in making the rest of the program to execute? I want the program to run instead.

[Salil Wadnerkar]

I don't understand what you mean by closing. You want to print something when all the jobs have finished executing. Since these are periodic jobs, you need to "shutdown" the executor, which then will not accept any new periodic task. Then "awaitForTermination" will wait for the current jobs to finish. So, after that you can print something.

[kuhu vora]

Yes, I want to print the line after all the scheduled tasks are executed. Here what does "Disable new tasks from being submitted" mean? Will it not allow the existing service to run the tasks or will any new service will not be allowed. I am trying to test what this code is doing.

[kuhu vora]

I called the shutdown method in the help on my code. It stops the scheduler even before it runs.

[Salil Wadnerkar]

How long you want to run these tasks depends on your application logic. In the simplest case, if you want to run these tasks for 10 minutes and then run your print statement, just add Thread.sleep statement. Refer: http://www.journaldev.com/2340/java-scheduledthreadpoolexecutor-example-to-schedule-tasks-after-delay-and-execute-periodically

[kuhu vora]

I was trying to stop the method using the followig code:

Helper class:

	/**
	 * Stops a scheduler
	 */
	public void stopScheduler(ExecutorService pool) {
		pool.shutdown(); // Disable new tasks from being submitted

	}

	public ScheduledExecutorService createScheduler(int taskIntervalMins, int taskDurationHours,
			Runnable doTask) {
		final ScheduledExecutorService scheduler = Executors
				.newScheduledThreadPool(1);

		final ScheduledFuture<?> scheduleTask = scheduler.scheduleAtFixedRate(
				doTask, 2, taskIntervalMins, MINUTES);
		scheduler.schedule(new Runnable() {
			public void run() {
				scheduleTask.cancel(true);
			}
		}, taskDurationHours, MINUTES);
		
		return scheduler;
	}

main class:

//scheduling tasks
			final Runnable doTask = new Runnable() {
				public void run() {
					try {
						System.out.println("test");
					} catch (Exception e) {
						logger.error("command did not execute");
					}
				}
			};

			stopScheduler(createScheduler(dataInterval, testDuration, doTask));
			
			System.out.println("test completed");
			

my "test completed" is getting printed before any test is printed. I don't want to use sleep if the time becomes hours to execute. Can I do this using the stop thing?

[Salil Wadnerkar]

You are executing a buncvh of tasks periodically. They will execute forever if you don't stop them. So, you need to decide when you want to stop this periodic execution.

[kuhu vora]

scheduler.schedule(new Runnable() {
        public void run() {
            scheduleTask.cancel(true);
        }
    }, taskDurationHours, MINUTES);

doesn't this piece of code specify that the task should run periodically(suppose 1 min) till taskDurationHours(suppose 5 min )? I thought this means that task will run after 1 min after a initial delay of 2 mins till a total time of 5 mins.

[Salil Wadnerkar]

You can refer: https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ScheduledExecutorService.html

These time parameters indicate "initial delay" and "periodic delay" - nothing about the termination.

[kuhu vora]

Here is a class with a method that sets up a ScheduledExecutorService to beep every ten seconds for an hour:

 import static java.util.concurrent.TimeUnit.*;
 class BeeperControl {
   private final ScheduledExecutorService scheduler =
     Executors.newScheduledThreadPool(1);

   public void beepForAnHour() {
     final Runnable beeper = new Runnable() {
       public void run() { System.out.println("beep"); }
     };
     final ScheduledFuture<?> beeperHandle =
       scheduler.scheduleAtFixedRate(beeper, 10, 10, SECONDS);
     scheduler.schedule(new Runnable() {
       public void run() { beeperHandle.cancel(true); }
     }, 60 * 60, SECONDS);
   }
 }

The example does say that this will run till an hour due to the 60*60 thing..

[Salil Wadnerkar]

Oh, you are right. I missed the last line. So, beeperHandle.cancel(true) cancels the periodic executions. You can just add scheduler.shutdown() after that statement and then print whatever you wanted to print.

[Mike Simmons]

kuhu vora wrote:

scheduler.schedule(new Runnable() {
        public void run() {
            scheduleTask.cancel(true);
        }
    }, taskDurationHours, MINUTES);

First, a minor point, I think you need to decide if you're using hours or minutes, and be consistent. At least, consistent within a given method call.

Second, I think this "cancel" task is preventing you from shutting down as promptly as you'd like. I assume it's set to stop everything sometime in the future. If you try to shutdown() before that, well, this cancellation task is already scheduled, so the service will wait until it executes. To shutdown more promptly, use shutdownNow() rather than shutdown(). The shutdownNow() method will not only prevent any new tasks from being schedules, but it will also remove any already-scheduled tasks that have not run yet, and it will attempt to interrupt any currently running tasks. That's probably what you want.

Last, even shutdownNow() merely starts the shutdown process. It doesn't wait until it's over. If you want to print "tests completed" after everything is shut down, you need to also call awaitTermination() after the shutdownNow().

[kuhu vora]

Mike Simmons wrote:

kuhu vora wrote:

scheduler.schedule(new Runnable() {
        public void run() {
            scheduleTask.cancel(true);
        }
    }, taskDurationHours, MINUTES);

First, a minor point, I think you need to decide if you're using hours or minutes, and be consistent. At least, consistent within a given method call.

Second, I think this "cancel" task is preventing you from shutting down as promptly as you'd like. I assume it's set to stop everything sometime in the future. If you try to shutdown() before that, well, this cancellation task is already scheduled, so the service will wait until it executes. To shutdown more promptly, use shutdownNow() rather than shutdown(). The shutdownNow() method will not only prevent any new tasks from being schedules, but it will also remove any already-scheduled tasks that have not run yet, and it will attempt to interrupt any currently running tasks. That's probably what you want.

Last, even shutdownNow() merely starts the shutdown process. It doesn't wait until it's over. If you want to print "tests completed" after everything is shut down, you need to also call awaitTermination() after the shutdownNow().

Great points you made there! I will definitely follow point one. For point two, I was initially not shutting it down at all, I did after the post by Salil though. I thought it was the right thing to do from good coding practices POV. I don't want to user shutDownNown because the user has the option to run the task till 24 hrs. Awaiting 24 hours doesn't seem right to me. Excellent third point! From a application POV, I am trying to do a lot of tasks after this scheduledtask is done but they should be done only after all the tasks complete. So, I would mostly say that this is a helper thread. How to do suggest I do it?? I am not very good with threads, if you can some way to achieve synchronization there, I would be very grateful.

[kuhu vora]

Salil Wadnerkar wrote:Oh, you are right. I missed the last line. So, beeperHandle.cancel(true) cancels the periodic executions. You can just add scheduler.shutdown() after that statement and then print whatever you wanted to print.

I should have maybe better explained the question.I am able to achieve the desired results when using simple tasks such as println statements but when I run heavy tasks in the scheduler, there are problems. The line to print output runs first before scheduler tasks. I fear it is a thread synchronization problem. Unfortunately, I am extremely bad at thread synchronization. So, was hoping there was some alternate way.

Secondly, I tried the shutDown thing you recommended. I think it should definitely be a part of code as good coding practice. But this doesn't terminate the program and the program keeps to running after all the execution is done.


Kindly suggest.

[Salil Wadnerkar]

The call that cancels the user scheduled tasks is not .shutdown() or .shutdownNow(), but, beeperHandle.reset(). From what you said, you don't want to cancel user's task, but execute some logic when their task is complete. For that, you can check it with beeperHandle.isDone() and if it is true, execute some logic like this:

        scheduler.schedule(new Runnable() {
            public void run() {
                if (scheduleTask.isDone()) {
                  // Things that I want to do after user scheduled tasks are complete
                  System.out.println("Tasks completed!");
                }
            }
        }, taskDurationHours, MINUTES);

[Mike Simmons]

Salil Wadnerkar wrote:The call that cancels the user scheduled tasks is not .shutdown() or .shutdownNow(), but, beeperHandle.reset().

Um, no. there is no such method on ScheduledFuture, the interface implented by the beeperHandle. Perhaps you meant the cancel() method, already discussed?

Regardless, scheduler.shutdownNow() does cancel the user-scheduled tasks. If you don't believe me, try testing it, as I have.

Both shutdown() and shutdownNow() remove all future executions of the recurring task, the one that keeps printing "beep". However shutdown() does nothing about the previously-scheduled one-time task to cancel() the beeper. Whereas shutdownNow() removes that one as well, and also issues an interrupt to any currently-executing tasks.

So, if you want to cancel a specific task, use the cancel method on that task, sure. But if you want to cancel *all* tasks in a ScheduledExecutorService, shutdownNow() does that as well as possible. The only limitiation being, a currently-executing task may or may not react to an interrupt, depending on how it's written.

Of course, that was assuming the original poster did want to cancel everything. Now we know he doesn't, so...

Salil Wadnerkar wrote:From what you said, you don't want to cancel user's task, but execute some logic when their task is complete. For that, you can check it with beeperHandle.isDone() and if it is true, execute some logic like this:

        scheduler.schedule(new Runnable() {
            public void run() {
                if (scheduleTask.isDone()) {
                  // Things that I want to do after user scheduled tasks are complete
                  System.out.println("Tasks completed!");
                }
            }
        }, taskDurationHours, MINUTES);

This looks highly unreliable. It would check once, and only once, to see if the scheduleTask is done. If it runs before that, we're out of luck, we never check again.

This could be modified to check periodically instead (e.g. using scheduleAtFixedRate() rather than schedule()). In this case, you'd want to modify this task to cancel itself as well, once completed.

On the other hand... if the original requirement is to run certain code only after the cancel() has been called... why not just put that code in the same task as the cancel(), right after the cancel()? Well, I guess we need to make sure that the task is actually completed first...

    scheduler.schedule(() -> {
      beeperTask.cancel(true);
      try {
        beeperTask.get();
      } catch (CancellationException e) {
        // expected; ignore
      } catch (InterruptedException | ExecutionException e) {
        e.printStackTrace();
      }
      System.out.println("Now execute whatever post-cancellation tasks you want");
    }, taskDurationMinutes, MINUTES);

[Salil Wadnerkar]

Thanks Mike for the explanation. Yes, you are right that I meant cancel() on the beeperTask.
The original code called cancel() on the Future first and then shutdown() on the Executor. So, that's why I said that cancel() has already cancelled the tasks. (If cancel was not called, shutdown() would have cancelled these tasks).
After some googing, my proposed isDone() will not work (as a ScheduledFuture never gets to complete "normally") and there does not seem to any other way than forcefully canceling user tasks using cancel().

[kuhu vora]

@Mike : Thanks for your thoughtful reply. As rightly pointed out by you, I am not looking to cancel any future tasks. I did previously try to put the code to run after the scheduler after the cancel thing, it works as I desire. But the problem is that I need to put rest of the entire code there which I don't think is such a good idea.

[kuhu vora]

Salil Wadnerkar wrote:Thanks Mike for the explanation. Yes, you are right that I meant cancel() on the beeperTask.
The original code called cancel() on the Future first and then shutdown() on the Executor. So, that's why I said that cancel() has already cancelled the tasks. (If cancel was not called, shutdown() would have cancelled these tasks).
After some googing, my proposed isDone() will not work (as a ScheduledFuture never gets to complete "normally") and there does not seem to any other way than forcefully canceling user tasks using cancel().

Salil,

Thanks for your thoughts on this. My problem though is that I don't want to cancel any tasks. On the contrary, I want to make sure that these run compulsory because the rest of the code is dependent on that. Also, once the scheduler starts, it obviously will not terminate till some sort of shutdown mechanism is called. I am not sure what is the right way to call shut so that any future tasks are not affected. My scenario here is that the tasks needs to be run periodically after some minutes while the shutdown has to be done after the some hours(user specified) which has a max of 24 hours. Ideally I have to make sure that the periodic tasks are all completed before the execution gets back to the main program.

Thanks a lot for taking the time for this. I learnt a lot of helpful tips from this discussion.

[Salil Wadnerkar]

Kuhu,

Since you want to wait for the user tasks to complete, I think you can call beeperTask.cancel(false) which will not interrupt already running user tasks, but cancel their future runs (You need this because user does not want to execute these tasks after the specified limit < 24 hours). Apart from that, what Mike's solution should work. Note that, Mike added beeperTask.get() which will wait for the currently executing tasks to finish, if there are any.

scheduler.schedule(() -> {
  beeperTask.cancel(false);
  try {
    beeperTask.get();
  } catch (CancellationException e) {
    // expected; ignore
  } catch (InterruptedException | ExecutionException e) {
    e.printStackTrace();
  }
  System.out.println("Now execute whatever post-cancellation tasks you want");
}, taskDurationMinutes, MINUTES);

[kuhu vora]

This code does solve the synchronization problems but still the program does not terminate

[Salil Wadnerkar]

To terminate program, you need to call scheduler.shutdown() or shutdownNow() and then .awaitTermination() as discussed previously. I did not suggest this because it looked like you have jobs from multiple users and you should not shutdown the executor unless you don't want to schedule anything anymore.

[kuhu vora]

@Salil, This struck me just after posting my reply. I tried and was back to reply when saw your reply. It works! Thanks a lot for your help:)