• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Paul Clapham
  • Tim Cooke
  • Jeanne Boyarsky
  • Liutauras Vilda
Sheriffs:
  • Frank Carver
  • Henry Wong
  • Ron McLeod
Saloon Keepers:
  • Tim Moores
  • Frits Walraven
  • Tim Holloway
  • Stephan van Hulst
  • Carey Brown
Bartenders:
  • Al Hobbs
  • Piet Souris
  • Himai Minh

Multithreading in Quartz Scheduler

 
Ranch Hand
Posts: 275
jQuery Eclipse IDE Spring
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi All,

I am implementing org.quartz.Job interface from the Quartz API version 1.4.
I have an execute method which follows the following steps:

1) Create arraylist of names of the unread files in an FTP folder
2) for the list
a) process 50 files at a time , each creating a response .. save the responses in a map .. rename file as read
b) save all these responses as files on FTP

The problem is, when the number of files is large (in the order of 1000-1500) step 2a) and 2b) get intermingled... i observe when i write loggers and console messages.
Here .. the number of response files generated decreases, as the order of processing is twisted.
I suspect there is some multithreading involved... i observe some at eclipse debugger as well.

I tried keeping step 1 and 2 in a synchronized block.. and the correct number of responses were found.
But I still doubt the order of execution ( on observing the logger ).

Is this scheduler multithreaded .. if yes, how does it affect the code in execute method ? and is my apporach correct ??

Please help... I am also looking for some documentation.. will update if I get something useful.
 
author
Posts: 3281
8
Mac OS X Eclipse IDE Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hmm, I'm not really clear on how you've set up your scheduled jobs. Do you have a job for retrieving files and a separate job for putting back responses?
 
Aniruddh Joshi
Ranch Hand
Posts: 275
jQuery Eclipse IDE Spring
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
It is in the same job
 
Ranch Hand
Posts: 2908
1
Spring Java Ubuntu
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
As you said, you have as single job for processing files and saving the response, then check out if you mistakenly created multiple Jobs for the same or not.






 
Martijn Verburg
author
Posts: 3281
8
Mac OS X Eclipse IDE Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
If step 2.b can't start until 2.a is completed then I'm not sure how this problem can occur for you. are steps 2.a and 2.b asynchronous?
 
Aniruddh Joshi
Ranch Hand
Posts: 275
jQuery Eclipse IDE Spring
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
First of all apoligies for the delay in response..

Martijn , The probelm is 2a gets processed but 2b is skipped.

i have declared the execute method synchronized to synchronize the process.
And i have chenged the processing method to one request file at a time. Now the process goes like this :

1. Read a request file
2. Process this file ( this consumes some time ) : response is generated here
3. Put response on FTP

The magnitude of the problem has reduced , but it still exists.


The code follows:
 
Martijn Verburg
author
Posts: 3281
8
Mac OS X Eclipse IDE Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I think I'm still unclear. From reading your code it looks like:

1.) The scheduler will call execute on the Job (how often are you calling this BTW?)
2.) It goes a gets a list of files from the FTP server.
3.) For each valid file it generates a response and puts that back on the FTP server.

You say that the putting back of the response is 'skipped', how so? What logging etc are you seeing? Is the method not being called at all?
 
Aniruddh Joshi
Ranch Hand
Posts: 275
jQuery Eclipse IDE Spring
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
hi All,
sorry for the delayed response again
i was triggering jobs each 2 minutes and 1 job was taking longer than that to complete.

i made the job stateful by implementing StatefulJob interface and its working fine.

thanks again
 
Martijn Verburg
author
Posts: 3281
8
Mac OS X Eclipse IDE Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Glad you solved it and you're welcome!
 
It looks like it's time for me to write you a reality check! Or maybe a tiny ad!
Garden Master Course kickstarter
https://coderanch.com/t/754577/Garden-Master-kickstarter
reply
    Bookmark Topic Watch Topic
  • New Topic