I've been working with ScheduledExecutorService.scheduleAtFixedRate() and noticed is has some odd behaviors related to incomplete executions. For example, if an Exception is thrown that is not caught within the Runnable method, all future executions of the schedule will be cancelled. I also noticed if a thread takes deadlocks or "just takes a really long time", all future executions will delayed indefinitely.
Is there a way to set a time to live when using the ScheduledExecutorService service? My goal: Have a process that runs every 60 seconds regardless of whether the previous process failed or is still running. Alternatively, it could kill the previous process if it is still running after some specified time to live.
I can fix the exception case by wrapping the content of the run method with a catch block for Throwable, but I'm not sure how to set a thread time to live and/or allow the thread to start while a previous instance is still running.
Rather than using the scheduleAtFixedRate method , could you use the scheduleWithFixedDelay method to trigger the first one and then call it again as the first thing you do in your Runnable's run method ?
There's really no difference between on how these two function other than the time the subsequent thread starts. If one thread deadlocks or dies, all subsequent passes will be cancelled regardless of the method used.
Could you use the scheduler to kick off yet another thread each time to do the processing in? That way the ScheduledExecutorService would be (I would expect) completely unaware of the success or failure of the processing itself.
(Yes, I realise that's ugly, and there must be a better way, but it might just be a way of making a small change to get it working.)
You can probably have a reaper thread that checks whether your task is still running, if yes, then call a cancel on the ScheduledFuture you got out of the scheduledAtFixedRate() call. On successful cancel you can re-schedule another task with zero initial delay.
You can track whether a task is done or not (ScheduledFuture#isDone() will not give you status of intermediate executions) by tracking execution using beforeExecute() and afterExecute(). There will be some race conditions here that you will have to handle though.
Of course, this solution mandates that your task responds to interrupts, if not, then anyways there is very little you can do safely.