Let me apologise beforehand for the length of this post: I'm sorry
I'm currently working on a project that relies heavily upon Websphere MQ as a JMS provider to send messages from our WebSphere application to mainframe applications and back. We have just about everything working quite nicely, including ASCII<->EBCDIC conversion.
The problems start when the messages send by the mainframe application get over 32k in size, since the PL/1 applications cannot process data over 32k in size. JMS has two possible ways to solve this problem: message segmentation and logical message groups. The first (segmentation) is simply not supported on the IBM z/OS platform (http://www.developer.ibm.com/tech/faq/individual?oid=2:81777), so logical message groups automagically became our solution.
With logical message groups, the sender sends multiple messages, indicating in each message that it is part of a group and what the sequence number of that part is. When the application sends the last message, it uses a special flag while submitting the message to indicate that this was the last message in the group.
Since the software on the mainframe is written in PL/1, it is limited to 32k messages. As we have logical messages that are larger than 32k (using xml-ish syntax), we need a way to 'segment' the messages. As I understand it, there are two options available to realise this: - 1 - MQ controlled segmentation. Apparently (according to IBM documentation on the web ( http://www.developer.ibm.com/tech/faq/individual?oid=2:81777 )) this is not possible on z/OS: "WebSphere MQ for z/OS does not support segmentation, but it supports logical message groups." - 2 - Application controlled 'segmentation' using message groups where each message group consists of a single 'logical' message.
This second option was used succesfully for a short period on AIX. (It still functions correctly on the Windows pre-test environments. (Same ear file)). We have no idea (yet) whether that was mere coincidence or whether something in either the MQ or the WebSphere configuration was changed, causing the current implementation to fail. At the time of writing every attempt to receive grouped messages (this way) failed.
Current implementation on the WebSphere side does the following: Receive a message (using a timeout period of several seconds). - On error, report. On success read the JMSXGroupID message property to determine whether this is a grouped message. - If not, return the message received. If it is, store any and all messages matching the correlation id set on the previously received message (using the same receiver instance by calling the receiveNoWait method until it yields no more results). - Compile the complete message and return the result.
There is an assumption in this way of handling grouped messages that the receive method, used to receive the original message, does not return a message until all the messages in the group are available. Apparently this is either an incorrect assumption, or it can be configured somewhere and its setting has recently been changed.
As I see it, MQ (the transport layer) has all the information it needs to successfully deliver either the entire group of messages, or not a single one, and only signal the receiving queue manager when all messages have arrived. I believe JMS support for message groups is lacking one of two things here:
- 1 - JMSXGroupLast property (or something similar), or - 2 - JMSXMaxGroupSeq property (or something similar), set on each message to indicate the sequence number of the last message in the group.
Either option would allow the receiving application to successfully recompile the original message.
After some deliberations, we set out on the following workaround: When a received message is determined to be one of a grouped message, all subsequent messages are received using a shorter (1 sec) timeout. As soon as we receive the first timeout, we assume that all messages in the group have been received. Of course there is no guarantee that all the messages have been received and the total wait time for the message has been extended by a minimum of 1 second, but at least this seems to get the job done for now.
Another option that does not rely upon a timeout is to check the actual data for the end of the message, but that would mean crossing the transport/application layer boundary, which I would rather not do.
These are the questions I asked IBM: Question A: Am I correct in assuming that the receive method should not return a grouped message until all messages in the group are available? Question A-2: If not, what method (using JMS), could I use to determine that all messages in the group have indeed been received? There appears to be no standard (or IBM specific property on a message to indicate that it is the last in a group). I know that there is in fact a flag in the MQMD to indicate this. Question B: Do you have a better way of receiving the remaining messages? Or Question B-2: Is this the way other customers have implemented receiving of message groups as well?
I won't post the reply I got from IBM. Suffice to say it made me and and gave me the feeling of .
Apparently I asked some extremely difficult questions. Any comments? Ideas? Suggestions? Similar experiences?
Quis custodiet ipsos custodes?
Stop it! You're embarassing me! And you are embarrassing this tiny ad!
free, earth-friendly heat - a kickstarter for putting coin in your pocket while saving the earth