• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Batch job with multiple threads

 
Ranch Hand
Posts: 167
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I have a batch job which has 2 steps, Step A and Step B. Step B should always execute after A is complete. Step A involves processing many files, the names of which we get as a list. Now I have been asked to divide step A into multiple threads, each of which will execute a finite number of files. If I start multiple threads, how do I ensure that Step B starts only after all threads processing step A have completed?

Thanks

Jhakda
 
Marshal
Posts: 28193
95
Eclipse IDE Firefox Browser MySQL Database
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Either as part of step A, or immediately after it, you wait for all those threads to be completed. Then you start on step B.
 
Jhakda Velu
Ranch Hand
Posts: 167
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thank you for the response.
Based on your suggestion, here is what I pan to do:

Suppose there are 10n files to be processed, and I decide to allocate 10 files per thread, so I will create 'n' threads and add to a collection.
Then I keep checking each thread's state to see if the execution is complete( using getState()), if so, start Process B. Else wait.

Does the above seem good enough? You do you have a better approach, about which you can give some hints?

Thanks once again

Jhakda
 
Sheriff
Posts: 3837
66
Netbeans IDE Oracle Firefox Browser
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
You would just invoke the join() method on all threads in the collections. Join waits until the "target" thread completes, so you'd know that after invoking join on every thread, process A is done. And, unlike periodic checking of individual thread states, this waiting is done efficiently.

I'd strongly suggest using some of the library functions (Executors) for this task, though. Have a look at this tutorial.
 
Bartender
Posts: 4179
22
IntelliJ IDE Python Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Also, since you will undoubtedly be sending those file from one thread to another, you should look into either a SynchronousQueue (one directional information dump) or Exchanger (two directional information exchange), which could help both the sequencing and the exchange of data.
 
Jhakda Velu
Ranch Hand
Posts: 167
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks for the information. I am currently going through the tutorial. Also, agree that join works better than my approach.

@Steve: The directive given to me was not to make the threads I create for Step A to be synchronous. Step A and Step B should be synched, with Step A being done by multiple threads. However, I will surely keep your input in mind if I need to build a synchronous activity.

Thanks to all
Jhakda
 
With a little knowledge, a cast iron skillet is non-stick and lasts a lifetime.
reply
    Bookmark Topic Watch Topic
  • New Topic