We call out from an EJB to a partner system web service. Sometimes the partner accepts the connection and the request but never sends a response. Our code hangs until the container times out the transaction at the default 120 seconds. The container writes a timeout exception of some kind to the log, but we don't seem to be able to catch the exception and do anything useful. My totally made up theory is that when we have used up our 120 seconds the container stops running our code and doesn't give us even one more clock cycle. Is that something like what really happens? [ November 19, 2005: Message edited by: Stan James ]
A good question is never answered. It is not a bolt to be tightened into place but a seed to be planted and to bear more seed toward the hope of greening the landscape of the idea. John Ciardi
Its not so much clock cycles as it noticing you've gone over the transaction time limit and therefore invalidates the entire transaction. After all, there's no point for an invalid transaction to continue.
Some ways around this are to:
Increase transaction timeout in the Admin console (although that wont help if the other server never sends a response)
Spawn a thread to connect to the web service and terminate the thread after some number small than the transaction time such as 60 seconds.
Set the process that connects to the web service as a separate EJB method with its own Transactional attribute such as "RequiresNew". This way if the child fails, the parent transaction can keep going (although may still time out if waiting on the child).
Add a finally block to your bean that will always be executed even if the transaction fails.
[ November 19, 2005: Message edited by: Scott Selikoff ]
It is the responsibility of the service provider to tell you the timeout value for their web service. Your transaction timeout should be set higher than the timeout of the service. Usually a service goes through some web server and a timeout should be available. Do not expect no for an answer.
Do not spawn threads in an EJB container, this is against the J2EE spec and very dangerous. WAS 6 provides WorkManager's as a work around for thread spinning, but in this case, it is up to the service provider to tell you the timeout value. I do not recommend threads. [ November 20, 2005: Message edited by: Roland Barcia ]
Roland Barcia: IBM Distinguished Engineer, CTO Mobile for Lab Services
Ah your right, I forgot its against the J2EE spec to spawn threads. Better to more carefully manage the connection then, as previously discussed. [ November 19, 2005: Message edited by: Scott Selikoff ]
Joined: Jan 29, 2003
Actually we probably will spawn a thread to manage this. I haven't found anything else with much promise of working for us. I know it's against the spec, and I think I read that future versions of WebSpehere might enforce the spec by refusing to start up new threads, but I also know there is precedent for it in other systems that seem to run just fine.
The 120 second limit is an issue, but something I can live with or adjust as need be. The real problem is when the container times out we don't seem to be able to catch it and respond to it.
Interestingly we just installed Wily Introscope and then saw an incident like this. The partner stopped responding and our EJB concurrency and response time averages went up. When the partner recovered, our stats went back to normal. But because we can't catch and handle this error we did not give good feedback to our users and they started calling in problem reports on our system. We "keep score" on things like this, so we hate to take the blame.
What API are you using to connect to the external service? You should be able to set a timeout value or monitor the connection in some way other than spawning a thread.
I agree with Roland, you should ask the service provider the timeout for the web service and if this is greater than the timeout of your application, you should set the timeout of the connection lower and have a case for what to do when this timeout is reached the service is not complete.
If the timing is off by only a few seconds though, you can always increase your transaction timeout value.
Joined: Jan 29, 2003
We're using HttpURLConnection to post to remote web services and read the response. The biggest problem occurs when the partner's HTTP server is healthy, accepts a connection and a request, but the app server has a problem and never responds. I don't know if they could make their HTTP server throw back a timeout error. That's something I should follow up now that we've talked about it.
You could use java.net.Socket instead of HttpURLConnection since it has a set timeout value. Use lose some of the nice Http connection features, but you may not be using them anyway. Not sure if this would be applicable for your situation but good luck!