File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
http://aspose.com/file-tools
The moose likes Spring and the fly likes Spring Integration | FTP - FILE Adapter Issue Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of The Java EE 7 Tutorial Volume 1 or Volume 2 this week in the Java EE forum
or jQuery UI in Action in the JavaScript forum!
JavaRanch » Java Forums » Frameworks » Spring
Bookmark "Spring Integration | FTP - FILE Adapter Issue" Watch "Spring Integration | FTP - FILE Adapter Issue" New topic
Author

Spring Integration | FTP - FILE Adapter Issue

Shriram Gopalakrishnan
Ranch Hand

Joined: Feb 20, 2007
Posts: 44
The application which i am working on has an Integration Layer build using Spring Integration 2.5, It tries to pull a CSV file from FTP server using FTP adapter and drops it on to local server which is polled by
File adapter. So as soon as file is downloaded by ftp the file adapter passes it on to Splitter wherein the CSV file is split into separate records and send to downstream for further processing.

The issue occurs when splitter some times does not see the file tha is downloaded by the FTP adapter. Below is the config details and that piece of code from splitter which tries to check if the file exists and throws user defined exception if the file does not exists.

The problem is that it always runs into the exception.





In the above code it throws under defined expection ResourceLoaderException. Any suggestion on how to resolve this issue would be appreciated, Its Production issue and early response would really.
Shriram Gopalakrishnan
Ranch Hand

Joined: Feb 20, 2007
Posts: 44
Anybody please reply since this is an urgent issue.
Mark Spritzler
ranger
Sheriff

Joined: Feb 05, 2001
Posts: 17250
    
    6

A first comment before I read some code.

The whole ftp part just rings bells about the whole "Is the file completely downloaded yet"

Basically, when you ftp it, it might take a while for the full file to transfer, and the FileAdapter might run before the file has completely downloaded. This might be what is happening.

Mark


Perfect World Programming, LLC - Two Laptop Bag - Tube Organizer
How to Ask Questions the Smart Way FAQ
Mark Spritzler
ranger
Sheriff

Joined: Feb 05, 2001
Posts: 17250
    
    6

Yeah, after looking at the code, you are throwing an exception which will throw if the file hasn't been completely downloaded yet, so I would expect that to happen almost immediately. I wouldn't throw that exception. I would try a different approach.

Read this here. Scroll down till you get to the section called "More on File Filtering and Large Files"
http://static.springsource.org/spring-integration/reference/html/ftp.html

Mark
Shriram Gopalakrishnan
Ranch Hand

Joined: Feb 20, 2007
Posts: 44
Thanks Mark for reply,

So with large files in mind and in order to avoid any incomplete file processing 'filename-pattern="${ftp.server.localfilename}' attribute was used in file adapter which would match exact file name only process if that file exists. So once FTP starts downloading the file it will rename with some temp suffix until it has fully downloaded the file. Now this makes the file adapter pick up only the completed files.

Also we have code which also clears the local folder once the file has been sent to down stream for processing. Is there anything else which would have gone wrong here?
Mark Spritzler
ranger
Sheriff

Joined: Feb 05, 2001
Posts: 17250
    
    6

Not that I notice off the top of my head. What I would try is to do remote debugging and set a breakpoint at different points in the flow. But definitely one at that if (file.exists) statement and see what values have been set in the debugger for those objects.

Good Luck

Mark
 
jQuery in Action, 2nd edition
 
subject: Spring Integration | FTP - FILE Adapter Issue