Win a copy of Modern JavaScript for the Impatient this week in the Server-Side JavaScript and NodeJS forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Ron McLeod
  • Paul Clapham
  • Bear Bibeault
  • Junilu Lacar
Sheriffs:
  • Jeanne Boyarsky
  • Tim Cooke
  • Henry Wong
Saloon Keepers:
  • Tim Moores
  • Stephan van Hulst
  • Tim Holloway
  • salvin francis
  • Frits Walraven
Bartenders:
  • Scott Selikoff
  • Piet Souris
  • Carey Brown

Trying to avoid the max connection warning

 
Ranch Hand
Posts: 226
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I am trying to figure out a solution for the following error message without increasing the default connection size from 1000 to 2000 or more in the activemq config files.

Recently I came across the following error when around 1000 messages were sent to the broker with a delay of 5 minutes as shown in the code below.

   





The following is the code which is listening to ActiveMQ continously and as soon as it sees `COMPLETE`, it sends out an email to the user after generating a file.Otherwise, it goes inside `else`
block and sends out the message to the broker again.

Inside the else block, I want to test by closing the connection after I am done sending the message. So I have closed the connection inside a finally block as shown below. Is this a correct way to go about it?

Also, I am opening a new connection everytime I the else block is executed using the following piece of code:



How can I use a single connection and avoid using / creating multiple connections multiple times?

 
 
Saloon Keeper
Posts: 12251
259
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
You should use a PooledConnectionFactory, and you should close your connections using try-with-resources, not with try-finally. You're also not using Spring's dependency injection mechanism properly.

Now the trick is to configure Spring that it creates a PooledConnectionFactory for your application once. You should be able to do it like this:
 
Saloon Keeper
Posts: 22479
151
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
In purely abstract terms, the best way to avoid running out of connections is the make the connection-handling code run as quickly as possible. If your connection requires a process that takes a long time - and sending an email probably will - then instead of doing all the work in your message-handling thread, make the message handler create a request and add it to a queue so that backend processes can deal with the slow parts at leisure.

To scale up from there, you could have multiple message listeners running under a load-balancer and/or multiple backend server machines.
 
Jack Tauson
Ranch Hand
Posts: 226
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Stephan van Hulst wrote:You should use a PooledConnectionFactory, and you should close your connections using try-with-resources, not with try-finally. You're also not using Spring's dependency injection mechanism properly.

Now the trick is to configure Spring that it creates a PooledConnectionFactory for your application once. You should be able to do it like this:



Thanks Stephan.

I tested a solution different than yours and didn't use PooledConnectionFactory as shown below and it's working fine:



So basically, the connection is cached  (e.g. in a static variable) and re-used as shown above. It didn't break through until now.

You'll note that in this code there is no finally block to close the connection. That's intentional because the whole point of this code is keep the connection open so that it's not opening & closing the connection to send a single message. The connection is being re-used between invocations. The only time the connection is closed is when a Throwable is caught.

Do you think I would still need to add a finally block, test for open connection and if it's open then close it? I wonder since finally block is always executed, it's going to close the connection after exiting the else block, right?

 
Tim Holloway
Saloon Keeper
Posts: 22479
151
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
As a general rule, a Factory is a service class that's designed to dispense resources efficiently. If the resources are poolable, then often the factory will maintain its own internal pool. For example the DataSource factory that's built into every J(2)EE-compliant web application server.

Also as a general rule, holding resources for indefinite periods of time is stealing reasources from other users. And I won't even begin to consider what could happen if you have a single instance of something and multiple thread users without any synchronization.

Finally, while there are times where storing things in static variables and/or using static methods is essential, more often than not it's going to be trouble sooner or later. I speak from repeated experience, alas. Too often repeated.

Which brings me to my final observation. Just because something "works" isn't good enough. You should be able to confirm with reasonably certainty that it will always work, and not just when you're there watching it. Murphy's Law says that when it doesn't work will be at the worst possible time. If you are wise/experienced/covered with burn scars, you will not only proof your solution, you'll add logging and other de- and anti-bugging code to the solution so that when it does go wrong, you'll know why, what "impossible" data values were involved and how to most expeditiously recover, diagnore and repair.
 
Jack Tauson
Ranch Hand
Posts: 226
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Tim Holloway wrote:As a general rule, a Factory is a service class that's designed to dispense resources efficiently. If the resources are poolable, then often the factory will maintain its own internal pool. For example the DataSource factory that's built into every J(2)EE-compliant web application server.



Thanks for the explanation. I will try to test the pooled connection factory approach that Stephan has mentioned in his code above.

And I won't even begin to consider what could happen if you have a single instance of something and multiple thread users without any synchronization.



Do you mean when multiple users are accessing the application at the same time?


 
Master Rancher
Posts: 4663
49
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Doesn't have to be multiple users, as in people, just multiple threads.

If more than one thread at a time accesses this code then something will eventually break.
All it would take is for one thread to close the connection before another thread tries to use either it or the Session value from lines 70-78.
 
Tim Holloway
Saloon Keeper
Posts: 22479
151
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Dave Tolls wrote:Doesn't have to be multiple users, as in people, just multiple threads.

If more than one thread at a time accesses this code then something will eventually break.
All it would take is for one thread to close the connection before another thread tries to use either it or the Session value from lines 70-78.



Or for two threads to try to use the connection at once and thus corrupt the connection's internal state.
 
Dave Tolls
Master Rancher
Posts: 4663
49
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Indeed.
And that can be quite hard to spot.
 
Jack Tauson
Ranch Hand
Posts: 226
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Tim Holloway wrote:

Dave Tolls wrote:Doesn't have to be multiple users, as in people, just multiple threads.

If more than one thread at a time accesses this code then something will eventually break.
All it would take is for one thread to close the connection before another thread tries to use either it or the Session value from lines 70-78.



Or for two threads to try to use the connection at once and thus corrupt the connection's internal state.



I see. How about using addressing multiple people accessing the application at the same time by adding synchronized block as shown below:

 
Tim Holloway
Saloon Keeper
Posts: 22479
151
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
If performance is a problem, then adding synchronization will only make it worse. Synchronization will bottleneck all requests. Far better, if possible to use pooled connections with one connection per requester.

Also, why invoke the BeanFactory to get the DAO? Why can't you have it injected? I do.
 
Dave Tolls
Master Rancher
Posts: 4663
49
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Not only what Tim says re: creating a bottleneck, but I'm not convinced that the synching you have there will actually work.
Most of the code using the connection is not synched, so you'll still hit the same issues. It's quite possible for the connection to be closed/changed/operated on, while some other thread is using it.

Just use a connection pool.
It's what they were built for.
 
Those who dance are thought mad by those who hear not the music. This tiny ad plays the bagpipes:
Building a Better World in your Backyard by Paul Wheaton and Shawn Klassen-Koop
https://coderanch.com/wiki/718759/books/Building-World-Backyard-Paul-Wheaton
    Bookmark Topic Watch Topic
  • New Topic