aspose file tools*
The moose likes Threads and Synchronization and the fly likes How to make my JMS Subscriber more scalable? Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Java » Threads and Synchronization
Bookmark "How to make my JMS Subscriber more scalable?" Watch "How to make my JMS Subscriber more scalable?" New topic
Author

How to make my JMS Subscriber more scalable?

Pavan Kumar
Ranch Hand

Joined: Jan 23, 2004
Posts: 78
greetings,

I need some advice and a little help.
I have a JMS Subscriber that can subscribe upto 2000 messages/second. I have a worker class to which I hand off the subscribed message for further processing. This method is not really scaling well. When the incoming message rate increases, which is a certain possibility, my subscriber falls behind.
I have searched around the web for some solutions to overcome this problem, fortunately I hit upon this resource and it looks like MessageDispatcher pattern solves the problem that I have at hand. I tried to gain access to some more material to have a bit more understanding to incorporate the same into my design.

The pattern describes that I have to use Thread pools and more worker threads, so that once the message is consumed can be handed off to one of the free worker threads that can process the message further and return immediately. This looks to be a reasonable approach, I am willing to learn of any other ways that can make my subscriber scale better.

I tried to look for some Thread Pools and most of the resources were pointing to Doug Lea's PooledExecutor, since this is part of JDK5 as well, I want to explore it a little further. at this point, I must admit, it looks very difficult. I hope with some help I should be able to get through. I would really appreciate if some of you suggest me some further reading or some examples in this regard.

currently I have a ParserTask like this,

Are there any downsides to this approach? Is there a better/alternative solution?
I'm running my application on a single CPU machine, JDK 1.4.2.

Thanks very much for your help.
Regards,
CNU
[ February 28, 2005: Message edited by: cnu sri ]

formerly known as Cnu
Stan James
(instanceof Sidekick)
Ranch Hand

Joined: Jan 29, 2003
Posts: 8791
I agree with you - the doc on Doug Lea's packages went right over my head the first time. But essentially the same designs in JDK 5 were a piece of cake to pick up. You might try reading the Executor and BlockingQueue doc with JDK 5 just to see if they'r easier to follow.

Here's how simple my HTTP server is - minus exception handling and logging:

You already have a runnable command so you're well on the way to this kind of thing. If you can't get to JDK 5 and Doug Lea's code looks too complex google for other Java thread pools. There's one in Apache Commons that is delightfully simple, too.

BTW: If you're in an EJB container all the rules about threads change. You can control the number of instances of an MDB for the same effect, I think.

Let us know how it works out!


A good question is never answered. It is not a bolt to be tightened into place but a seed to be planted and to bear more seed toward the hope of greening the landscape of the idea. John Ciardi
Stan James
(instanceof Sidekick)
Ranch Hand

Joined: Jan 29, 2003
Posts: 8791
Oops, hit submit twice. Sorry.
[ March 01, 2005: Message edited by: Stan James ]
Pavan Kumar
Ranch Hand

Joined: Jan 23, 2004
Posts: 78
Stan,
Thanks very much for the inputs, I've just downloaded jdk5 documentation and would explore the docs further.

I have a very quick question though, if I do the following I'd be creating a lot more threads than re-using???

This is actually where I'm having problem understanding. When I first had problems I was thinking more on the lines of having a few number of tasks instantiated and stored in a LinkedList, then I would pick one task assign a message, once the processing is done return this to the pool(LinkedList) again. Since this would not shrink and grow as needed apart from being error prone, I wanted to look for some similar implementations already existing.

No, My application is not running in an EJB container, This is a TIBCO subscriber which runs on it's own thred updating a database. Part of this also updates a swing client.

Thanks for your time.
Regards,
[ March 01, 2005: Message edited by: cnu sri ]
Pavan Kumar
Ranch Hand

Joined: Jan 23, 2004
Posts: 78
OK, I think I'm making a little progress with LinkedBlockingQueueue.
I'll come back to discuss more,

Thanks,
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: How to make my JMS Subscriber more scalable?