Cached JNDI lookup in AS Cluster

[N Jain]

Hello Forum,

I have 2 JBoss AS configured in a cluster (HA-JNDI incorporated). I have a service locator class that implements JNDI caching.

Code Snippet

  final private static String CACHE_TYPE_FOR_SERVICE_LOCATOR = "ejb.servicelocator.cache.type";
    /**
     * get instance of cache from factory given cache type
     */
    private static Cache cache = getCache(CACHE_TYPE_FOR_SERVICE_LOCATOR);
    public static Object lookupEJB3(Context context, String jndiName) throws CustomException {
        try {
            Object obj = cache.getFromCache(jndiName);
            if (obj == null) {
                synchronized (ServiceLocator.class) {
                    obj = context.lookup(jndiName);
                }
                cache.putInCache(jndiName, obj, 0);
            }
            return obj;
        } catch(NamingException namingException) {
            throw new CustomException(NAMING_EXCEPTION_ERROR_NUMBER, new String[] { jndiName });
        }        
    }

As the application is accessed, n number of lookup's are cached that refer to either AS1 or AS2. Now when I shutdown AS1, and the application try to use any cached lookup that is referencing AS1, I get the following error.

org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://AS1:3873/]
	at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:776)
	at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:165)
	at org.jboss.remoting.Client.invoke(Client.java:1724)
	at org.jboss.remoting.Client.invoke(Client.java:629)
	at org.jboss.aspects.remoting.InvokeRemoteInterceptor.invoke(InvokeRemoteInterceptor.java:60)
	................
	at org.jboss.ejb3.remoting.IsLocalInterceptor.invoke(IsLocalInterceptor.java:74)
	at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
	at org.jboss.aspects.remoting.PojiProxy.invoke(PojiProxy.java:62)
	at $Proxy4.invoke(Unknown Source)
	at org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:207)
	at org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:164)
	................
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
	............
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
	at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
	at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
	at java.lang.Thread.run(Unknown Source)
Caused by: java.net.ConnectException: Connection refused: connect
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.PlainSocketImpl.doConnect(Unknown Source)
	at java.net.PlainSocketImpl.connectToAddress(Unknown Source)
	at java.net.PlainSocketImpl.connect(Unknown Source)
	at java.net.SocksSocketImpl.connect(Unknown Source)
	at java.net.Socket.connect(Unknown Source)
	at org.jboss.remoting.transport.socket.SocketClientInvoker.connect(SocketClientInvoker.java:289)
	at org.jboss.remoting.transport.socket.SocketClientInvoker.createSocket(SocketClientInvoker.java:206)
	at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.getConnection(MicroSocketClientInvoker.java:1089)
	at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:762)
	at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:165)
	at org.jboss.remoting.Client.invoke(Client.java:1724)
	at org.jboss.remoting.Client.invoke(Client.java:629)
	at org.jboss.aspects.remoting.InvokeRemoteInterceptor.invoke(InvokeRemoteInterceptor.java:60)
	.............
	at org.jboss.ejb3.remoting.IsLocalInterceptor.invoke(IsLocalInterceptor.java:74)
	at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
	at org.jboss.aspects.remoting.PojiProxy.invoke(PojiProxy.java:62)
	at $Proxy4.invoke(Unknown Source)
	at org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:207)
	at org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:164)
	..............
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
	..............
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
	at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
	at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
	at java.lang.Thread.run(Unknown Source)
	at org.jboss.aspects.remoting.InvokeRemoteInterceptor.invoke(InvokeRemoteInterceptor.java:72)
	... 27 more

I thought the lookup would be successful as AS2 is available.

Please advice where I am going wrong.

Thank you,
Nitin