Win a copy of TensorFlow 2.0 in Action this week in the Artificial Intelligence and Machine Learning forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Liutauras Vilda
  • Paul Clapham
  • Bear Bibeault
  • Jeanne Boyarsky
Sheriffs:
  • Ron McLeod
  • Tim Cooke
  • Devaka Cooray
Saloon Keepers:
  • Tim Moores
  • Tim Holloway
  • Jj Roberts
  • Stephan van Hulst
  • Carey Brown
Bartenders:
  • salvin francis
  • Scott Selikoff
  • fred rosenberger

Producer Consumers in Messages Queues

 
Ranch Hand
Posts: 431
2
Java
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Suppose we have a queue and some producer app is producing the messages into this queue at the rate of 100 messages/second. And the consumer is consuming messages at the rate of 50 messages/sec. So, there is a lag of 50 messages/sec which will cause the queue to full at some point in future. So, how to resolve such issues?
 
Saloon Keeper
Posts: 12440
269
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
There are different approaches. One is to block the producer until the queue has space. Another is to drop messages that can't be added to the queue. Yet another is to remove low priority messages from the queue to make space for high priority messages. The correct approach depends on your application.
 
Sheriff
Posts: 22028
108
Eclipse IDE Spring VI Editor Chrome Java Ubuntu Windows
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
There's another approach - start more consumers. If your consumer can process 50 messages per sec, then two consumers can most likely process more than 50 messages per sec (just don't expect 100 messages per sec, there's bound to be some overhead).
 
Vaibhav Gargs
Ranch Hand
Posts: 431
2
Java
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
In case of dropping low priority messages, these need to be passed to consumer first, is my understanding correct? So, there are also we will have queue filled in that case.
 
Saloon Keeper
Posts: 6650
161
Android Mac OS X Firefox Browser VI Editor Tomcat Server Safari
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
No, dropping them means discarding them withput being consumed as soon as the queue would otherwise overflow.
 
Vaibhav Gargs
Ranch Hand
Posts: 431
2
Java
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Do we have such capabilities to identify message priorities at the broker level? Does it follow JMS specs?
 
Stephan van Hulst
Saloon Keeper
Posts: 12440
269
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Messages can easily indicate their own priority by making them comparable to each other.

What do you mean by "JMS specs"?

Maybe it's better if you start explaining to us what your application needs to do, rather than us giving you vague explanations about broad topics.
 
This one time, at bandcamp, I had relations with a tiny ad.
Thread Boost feature
https://coderanch.com/t/674455/Thread-Boost-feature
reply
    Bookmark Topic Watch Topic
  • New Topic