• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

need help on implementing taskexecutor framework(concurrency) using spring framework

 
Ranch Hand
Posts: 61
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Dear All,

My requirement is to implement concurrency-taskexecutor(theading concept) framework in spring application.

Till now I have done the below .


My Application class (where I need to implement threading concept)
------------------------------------------------------------------
From the below class, i need to call rule_SelectInLoop and rule_CursorInLoop methods . currently the excecution is happening sequencially. But my requirement is to execute rule_SelectInLoop and rule_CursorInLoop concurrently. For this i need to use taskexecutor framework.



Configuration done in spring-servlet.xml
-------------------------------------------


TaskExecutorExample class
----------------------------


Please Help here how to proceed to implement concurrency using spring application .
 
Rancher
Posts: 43081
77
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

But my requirement is to execute rule_SelectInLoop and rule_CursorInLoop concurrently.



Where does that requirement come from? Please don't say "it has been given to me" - I'm asking what the purpose of it is.

And since both methods work on all the same data structures, I suspect there would have to be a good deal of synchronization.
 
rajareddya reddy
Ranch Hand
Posts: 61
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Sir,

we have more methods like execute rule_SelectInLoop and rule_CursorInLoop .
we are uploading multiple files by checking rules(SelectInLoop,CursorInLoop etc..) check boxes in the UI. so here it is calling validateWarnings method and calling rule_SelectInLoop and rule_CursorInLoop methods inside for each file. problem here for each file it is calling the methods declared in the validateWarnings sequentially. it is taking lot of time. So we need to implement concurrency concept here.
 
Ulf Dittmer
Rancher
Posts: 43081
77
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
In that case I would start by analyzing what is taking such a long time. It might be something that is not amenable to being sped up easily. Or it might be something that is easily sped up. But jumping into concurrent execution before you know why it is taking a long time could lead you down a path where you spend a lot of time for very little results.

And, as I said, since both routines work with the same data structures, proper synchronization might negate any speed gains.
 
Rancher
Posts: 2759
32
Eclipse IDE Spring Tomcat Server
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Well, apparently, you have looked at the example provided by Spring here That page has enough information to get you started. I'm having a hard time understanding what exactly you are having a problem with. Are you having a problem understanding how the task executor works? Do you want someone to do the design for you? Have you spent time thinking about it?

As far as I can see, you can declare 2 task classes; both should implemen tRunnable. One task class should do rule_SelectInLoop and the other should do rule_CursorInLoop. Then you just create 2 instances, one for each task class, and give it to the executor. This is the approach I would take given what you are describing here.

To achieve true concurrency, you will have to make sure your threads do not modify the same state. Like Ulf said, if they are both operating on the same Java object, and if both try to modify the Java object, the Java object will need to be synchronized, and this will reduce the amount of concurrency in the syste, The same thing applies to other things. If they are both trying to write to the same file, then you will need to protect the file, and you will reduce concurrency. If they try to modify a row in the database, then they will put locks on the database, and block each other
 
rajareddya reddy
Ranch Hand
Posts: 61
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Dear Ulf Dittmer/Jayesh,

Thank you very much for your kind reply.



There are more than 40 methods(like rule_SelectInLoop and rule_CursorInLoop method calls) i need to call from validateWarnings method. That means i need to write 40 classes that implements Runnable class. Is there any way that i can implement the task executor functionality with less number of classes.

One more doubt is in validateWarnings method for the rule_SelectInLoop and rule_CursorInLoop method calls return type is LIST . i need to add all the LISTS that are return by these method calls.For this Taskexecutor functionality enough or i need to use callable functionality that returns future object.
 
Jayesh A Lalwani
Rancher
Posts: 2759
32
Eclipse IDE Spring Tomcat Server
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Yes, you need 40 tasks because you decided that you want to run the 40 validations in parallel. The simplest solution to have a class for each kind of task. If your tasks share code, you might want to do some design around the tasks so you don;t violate DRY

That's not how I would do it, though. I;m working on the premise that the only way to do it is to run the 40 tasks in parallel, based on what you have posted. You are setting yourself up to have a rather rigid structure. Like what happens if the requirement changes and you need the results of one task in another task? Also, the time spent in executing the tasks will be dependent on the "long pole". Let's say you have 10 threads and 10 tasks. The total time taken will be equal to the longest running task; ie; the long pole. If you have 10 threads and 40 tasks, one thread might run the 2 longest tasks, simply because the task executor uses round robin execution. In this case, the total time taken will be equal to the time taken by 2 longest tasks

The ideal way to parallelize something is to roughly divide the work into equal sized tasks. This way you know you are spreading the load around. One way to do that is to divide your input file into equal sized batches, and execute a task for every batch. Within the task you can do all 40 validations on the records in the batch. I am not sure whether you can do it exactly this way. However, it;s worth thinking about how should the work be broken apart. Breaking them apart by validation is one way. But there can be other ways too.
 
Jayesh A Lalwani
Rancher
Posts: 2759
32
Eclipse IDE Spring Tomcat Server
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
BTW, have you looked at something like DROOLS http://en.wikipedia.org/wiki/Drools

It seems to me like you are trying to reinvent Drools
reply
    Bookmark Topic Watch Topic
  • New Topic