Win a copy of Learn Spring Security (video course) this week in the Spring forum!
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

Thousands of messages are hanging around for delivery in JMS queue. consumerCount=5?

 
Chandra Bhatt
Ranch Hand
Posts: 1710
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
We use JBoss 5.1 application server. My application server is running into high load. We push thousands of messages into JMS queue.

Its goes into situation where I see huge difference between Message count Vs Delivery count of the messages.

below is what i am seeing under jboss.messaging.destination of my specific queue in jmx-console:

MessageCount R int The number of messages in the queue 113430
DeliveringCount R int The number of messages currently being delivered 232

ConsumerCount is always 5.

The problem I am facing is the huge delay in delivery of individual messages. Basically once MDB is invoked to process one message and we make a HTTP connection to send that message to some external server. Looks like thousands of messages are never delivered, chocking in JMS queue.

When i restart the servers, for sometime it the messageCount and deliveringCount stays same for some time the difference goes on increasing. Looks like production of messages is more than the amount that can be consumed.

It has got performance issue as well, huge increase in CPU and Heap utilization.

Do we need to increase the cosumer count, if yes where is the configuration.

Any help will be very much appreciated.

Thanks!
 
Jayr Motta
Ranch Hand
Posts: 110
Google App Engine Google Web Toolkit Java
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I really haven't experienced such a critical situation, but I'll try to help:

Here on http://docs.jboss.org/jbossas/guides/j2eeguide/r2/en/html/ch6.chapt.html # 5.3.11. Destination MBeans, they show a jbossmq-service.xml and there is some explanations about the usage of the metadata provided there, check this out:

ReceiversCount: The number of receivers currently associated with the queue.


I can't really see a simple solution to it, maybe if your message broker runs separately and dispatches messages to a cluster of message consumers would help. You may look for poisoned messages too, maybe your queue is full of messages that can't be proccessed because everytime it generates an error making impossible to acknowledge the message as delivered and then queueing them forever.

I would bet either that you can't deal with your load running it without a farm of servers or that you have some proccessing issue. If you have problems with the load, you should try to increase the MDB pool size or even add more nodes to your cluster (ohh, I hate vertical scalability). If you have proccessing issues, maybe you're depending on something that doesn't respond on the same rate that you produce messages, or maybe your MDB isn't robust enough to deal with error that rollback the transaction making the message not be acknowledged.

Hope it helps ... let me know if you have some progress on it! Cya
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic