I have an extremely high performance high scale high available problem. It's sort of combining producer/consumer, broadcast/subscribe in a very high volume basis. Think 10k transactions per second with responses needed in 50msecs in a 24x7x365 up time scenario.
I have a pool of producers spread across multiple servers that get work. Ideally each producer would get a unit of work and broadcast the job out to all consumers of different types.
1. A Producer instance broadcasts job to consumer type A, consumer type B, consumer type C spread across multiple servers.
2. Consumer type A, B, and C work on the problem and respond with their individual results.
3. There will be many instances of consumer type A (or B or C), but only 1 instance should work on any one job.
4. Producer instance waits for results from A, B, and C and then picks the most optimal result from the work done by A, B, and C. If A, B, or C are slow to respond, they lose (expiry maybe?).
5. Producers and consumers will all be in multiple servers (VIPs).
What are the best technologies to consider for this?
MQ Series is the right tool for this. IBM has a talented group of software engineers. They would certainly be able to help you determine which technologies are best suited for the requirements and also how to design a solution that works.
0
