File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
http://aspose.com/file-tools
The moose likes Other JSE/JEE APIs and the fly likes OutOfMemoryError in SFTP Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Spring in Action this week in the Spring forum!
JavaRanch » Java Forums » Java » Other JSE/JEE APIs
Bookmark "OutOfMemoryError in SFTP" Watch "OutOfMemoryError in SFTP" New topic
Author

OutOfMemoryError in SFTP

Jacob Reilly
Greenhorn

Joined: Jan 16, 2012
Posts: 10
Just wondering if anyone has encountered this strange error. I am using jscape SFTP to access a client's FTP server. I am getting this error:



This is happening on the connect so it has nothing to do with the size of the file (it's only 4 bytes anyway). It has nothing to do with the directory file listing (the directory is empty). I used FileZilla to connect to the server and had no problem. If I change the IP address to a different FTP server the code works perfectly. So as near as I can guess, there is something about the combination of the client's server and jscape that is causing this error. I did a google search and found nothing that helps. So, has anyone encountered this strange problem? Has anyone an idea of what might be causing the error?

I should add that the code was written using jscape 7.0.2 but I did test with the current version and had the same problem.

Thanks for any help.
Jacob Reilly
Greenhorn

Joined: Jan 16, 2012
Posts: 10
I figured out the problem. The jscape code has a serious bug in it. When you connect to the client SFTP server you start a window size negotiation. You pass a preferred window size to the server and it passes a preferred size back. If the client's preferred window size is larger then jscape uses that size. The problem is that jscape does not verify that the size is reasonable. If the client's size is too large, larger than the maximum heap size for example, jscape still tries to allocate a byte array of that size causing an OutOfMemoryError.

So, if you get this error you need to ask your client to check their maximum window and packet size on their server and adjust them to a reasonable maximum size.
Bryan Meyer
Greenhorn

Joined: Nov 18, 2008
Posts: 6
I also encountered the same issue while using JScape SFTP v8.5 in order to make a secure connection to a server.

This post was on the JScape website and talks about configuring your connection so that it is not adaptive. I actually tried this solution, but it did not work for me. I thought I'd pass it along to see if you or someone else may have success with it.

http://help.jscape.com/Knowledgebase/Article/View/156/0/ssh-channel-not-closing--sftp-connection-dropped-prematurely
Jacob Reilly
Greenhorn

Joined: Jan 16, 2012
Posts: 10
I tried the same change and it didn't work for me either.
Jacob Reilly
Greenhorn

Joined: Jan 16, 2012
Posts: 10
Here is part of the debug log

01/19/2012 14:25:50 INFO [SSH] message sent: SSH_MSG_CHANNEL_OPEN, channel type: session, senderChannel channel: 0, initial window size: 32368, maximum packet size: 32368
01/19/2012 14:25:51 INFO [SSH] message received: SSH_MSG_CHANNEL_OPEN_CONFIRMATION, recipient channel: 0, sender channnel: 1120, initial window size: 2147483647, maximum packet size: 32368


As you can see, the return window size is Integer.MAX_SIZE. jscape doesn't check to make sure the return size isn't too big.
Muhammad Jawwad
Greenhorn

Joined: May 29, 2010
Posts: 3
Please try the latest version from this location :

http://www.jscape.com/download-secure-inet-factory/

Best Regards,
Mohammad Jawwad
JSCAPE
Managed File Transfer and Security Solutions
http://www.jscape.com

JSCAPE - Managed File Transfer and Security Solutions
http://www.jscape.com
Jacob Reilly
Greenhorn

Joined: Jan 16, 2012
Posts: 10
I don't need iNet. I need sftp. In any case, I tried the new version and it still doesn't work. Failing to validate the size of an int that is passed to you before using it to create a byte array is a pretty serious mistake. I think you owe a free upgrade to all your clients.
Jacob Reilly
Greenhorn

Joined: Jan 16, 2012
Posts: 10
A little further research... 10 years ago the maximum window size was limited to 16 bits which meant a top size of 65,535 bytes. But that is ancient history. The limit now is 4 gigabytes which is way beyond what you should be allocating byte arrays. Apparently the folks at jscape didn't take this change into account in their coding.
Muhammad Jawwad
Greenhorn

Joined: May 29, 2010
Posts: 3
Hi Jacob,
Use setDebug(true) before the connect() call and re-run the code, post all of the output here for review. All output is
necessary to see the full communication including the server banner string in the first 5-6 lines.

Best Regards,
Mohammad Jawwad
JSCAPE
Managed File Transfer and Security Solutions
http://www.jscape.com
Jacob Reilly
Greenhorn

Joined: Jan 16, 2012
Posts: 10
The full log was posted on your web site.
Jacob Reilly
Greenhorn

Joined: Jan 16, 2012
Posts: 10
They gave me the fix. The setStartSubsystemReplyNeeded handles the problem. It is only available in version 8.2 and above:


Paul Clapham
Bartender

Joined: Oct 14, 2005
Posts: 18656
    
    8

Thanks for following up with the solution, Jacob.
Jacob Reilly
Greenhorn

Joined: Jan 16, 2012
Posts: 10
I should add that jscape did go out of their way to provide help even though we did not have a support contract with them. Thanks Mohammad.
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: OutOfMemoryError in SFTP