java.net.SocketException: Connection reset while reading a large file

[Deepthi Musunuru]

We have built java based REST Webservices for the mobile app to interact with Sharepoint server. We are hosting webservices on weblogic server. Textual data is transferred in JSON format and file assets are transferred to iPad application as a binary stream. We are using HTTP GET to retrieve the file asset from the sharepoint. We have been noticing issues while trying to retrieve file assets which are greater than 20MB and only in production environment.

For files greater than 20MB, we have been noticing java.net.SocketException:Connection reset or java.net.SocketException: Socket closed depending on when the socket is closed.

We are using Apache HTTPClient 4.2 as http client and apache commons IO library for IO copy.

Below is the code -

  public org.apache.http.HttpEntity getAssetEntity(final String uri) {


    DefaultHttpClient client = new DefaultHttpClient();
    client.getParams().setParameter("http.connection.stalecheck", new Boolean(true));
    authProvider.addAuthentication(client);

    // Encode only special chars and not the whole URI
    String repURI = "";
    try {
        URL url = new URL(uri);
        URI httpURI = new URI(url.getProtocol(), url.getUserInfo(),
                url.getHost(), url.getPort(), url.getPath(),
                url.getQuery(), url.getRef());
        repURI = httpURI.toString();
    } catch (Exception e) {
        throw new SharePointClientException(e);
    }

    LOGGER.debug("Requesting OutputStream from URL:" + repURI);
    HttpGet httpget = new HttpGet(repURI);
    HttpResponse response = null;
    try {
        response = client.execute(httpget);
        org.apache.http.HttpEntity ent = response.getEntity();
        return ent;
    } catch (IOException e) {
        throw new SharePointClientException(e);
    }

}

protected StreamingOutputDetails getStreamingOutputForChapterAsset(final String assetURL) throws AuthorizationException {
    final HttpEntity assetEntity = getClient().getAssetEntity(assetURL);
    final StreamingOutputDetails streamingOutputDetails = new StreamingOutputDetails();
    streamingOutputDetails.setOutputSize((int) assetEntity
            .getContentLength());
    streamingOutputDetails.setStreamingOutput(new StreamingOutput() {
        @Override
        public void write(OutputStream output) throws IOException {
            try {
                getClient().streamFileAsset(assetEntity, output);
            } catch (SharePointClientException e) {
                // since write() throws IOException, we need to throw a
                // checked exception,
                // so wrap the current exception in an IOException
                throw new IOException(e);
            } catch (SharePointResourceException e) {
                // since write() throws IOException, we need to throw a
                // checked exception,
                // so wrap the current exception in an IOException
                throw new IOException(e);
            } catch (AuthorizationException e) {
                throw new IOException(e);
            }
        }
    });
    return streamingOutputDetails;  
}

   @Override
public void streamFileAsset(org.apache.http.HttpEntity assetEntity,
        OutputStream output) {

    InputStream contentStream = null;
    CloseShieldInputStream closeShieldInputStream = null;
    int bytes = 0;
    try {

        contentStream = assetEntity.getContent();
        closeShieldInputStream = new CloseShieldInputStream(contentStream);
        bytes = IOUtils.copy(closeShieldInputStream, output);
        EntityUtils.consume(assetEntity);

    } catch (IOException e) {
        throw new SharePointClientException(e);
    } finally {

        LOGGER.debug("bytes copied to output stream:" + bytes);
        if(null != closeShieldInputStream) {
            IOUtils.closeQuietly(closeShieldInputStream);
        }
        if (null != contentStream) {
            IOUtils.closeQuietly(contentStream);
        }
    }

}

This is only happening in production and uat environments where we cannot have wireshark installed to debug this further. Have validated the sharepoint setting and weblogic settings and they are same as other environments.

Below is the stack trace of the error -

Caused by: java.net.SocketException: Connection reset at 
java.net.SocketInputStream.read(SocketInputStream.java:168) at 
org.apache.http.impl.io.AbstractSessionInputBuffer.read(AbstractSessionInputBuffer.java:204) at 
org.apache.http.impl.io.ContentLengthInputStream.read(ContentLengthInputStream.java:182) at 
org.apache.http.impl.io.ContentLengthInputStream.read(ContentLengthInputStream.java:204) at 
org.apache.http.conn.EofSensorInputStream.read(EofSensorInputStream.java:155) at 
org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1792) at 
org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1769) at 
org.apache.commons.io.IOUtils.copy(IOUtils.java:1744) at 
com.test.client.SharePointServerListClient.streamFileAsset(SharePointServerListClient.java:217)

Thanks!

[Luan Cestari]

The connection reset comes from the client side (not the server you are getting the date), which means a stale object/connection. I saw that the CloseShieldInputStream ( http://grepcode.com/file/repo1.maven.org/maven2/commons-io/commons-io/2.0.1/org/apache/commons/io/input/CloseShieldInputStream.java ) doesn't close the connection, so you need to close the contentStream in the finally block.

Best regards,
Luan