One Global Remote Connection or Multiple Connections

My web requires many connections to remote data sources. When I say many I mean I get data from many files located on our AS400 server. As of right now every time data is needed I create a connection using AS400 Java Tool Kit library, get my data then close the connection. My boss wants me to find out if it would be more effeicent to open one connection as a user starts a session and close it when they end the session versus the many opening and closeing connection. Can someone help answer this. And how would I set up a connection that is available to all my servlets and classes? I have tried to set a connection to a session attribute but can not get this to work when I need to use it as a connction.

Not convinced that is a bona fide beginner's question. I think I ought to move the thread.

Tying connections to sessions is not always a good idea.

Are these JDBC connections?
If so, have you considered connection pooling?