posted 11 years ago
There are entire products designed to do nothing but manage queues. For example, I work in healthcare. We have a central repository that holds medical data from (literally) hundreds of different applications distributed among 15 hospitals and scores of provider locations.
Imagine a scenario without queues. Each application would have to have a dedicated listener. Since most applications don't send 24/7, many of those listeners would be idle a LOT of the time, wasting CPU time and memory. Or, you could have a few listeners that are constantly hopping around, checking first this one, then that one, then the next. Each time a new interface is added or subtracted, you'd have to re-write your list of what to check. Meanwhile, a bunch of applications are blocked from sending while they wait their turn. If you have real-time data (think vital signs during an operation), do you really want that stuck, being held up, while a 200 page discharge report is being generated by some other application?
That sounds horrible.
Instead, we use queue management software. Every application connects to the queue manager, and can drop off new messages as fast as they want. Messages go into a queue. From there, they can be re-routed to other priority queues - so vitals and labs can be put on a 'process now' queue, and discharge reports can be put on a 'process when we have downtime' queue. We then have various worker processes that read from specific queues, and process the data as fast as they can.
It doesn't matter if an application has 1 or 10,000 documents to drop - they drop them as fast as they can on their queues. and the workers process off as fast as they can. No sender has to wait for a worker to be ready to process things.
There are only two hard things in computer science: cache invalidation, naming things, and off-by-one errors
