• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Spring Integration | FTP - FILE Adapter Issue

 
Ranch Hand
Posts: 44
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
The application which i am working on has an Integration Layer build using Spring Integration 2.5, It tries to pull a CSV file from FTP server using FTP adapter and drops it on to local server which is polled by
File adapter. So as soon as file is downloaded by ftp the file adapter passes it on to Splitter wherein the CSV file is split into separate records and send to downstream for further processing.

The issue occurs when splitter some times does not see the file tha is downloaded by the FTP adapter. Below is the config details and that piece of code from splitter which tries to check if the file exists and throws user defined exception if the file does not exists.

The problem is that it always runs into the exception.





In the above code it throws under defined expection ResourceLoaderException. Any suggestion on how to resolve this issue would be appreciated, Its Production issue and early response would really.
 
Shriram Gopalakrishnan
Ranch Hand
Posts: 44
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Anybody please reply since this is an urgent issue.
 
ranger
Posts: 17347
11
Mac IntelliJ IDE Spring
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
A first comment before I read some code.

The whole ftp part just rings bells about the whole "Is the file completely downloaded yet"

Basically, when you ftp it, it might take a while for the full file to transfer, and the FileAdapter might run before the file has completely downloaded. This might be what is happening.

Mark
 
Mark Spritzler
ranger
Posts: 17347
11
Mac IntelliJ IDE Spring
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Yeah, after looking at the code, you are throwing an exception which will throw if the file hasn't been completely downloaded yet, so I would expect that to happen almost immediately. I wouldn't throw that exception. I would try a different approach.

Read this here. Scroll down till you get to the section called "More on File Filtering and Large Files"
http://static.springsource.org/spring-integration/reference/html/ftp.html

Mark
 
Shriram Gopalakrishnan
Ranch Hand
Posts: 44
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks Mark for reply,

So with large files in mind and in order to avoid any incomplete file processing 'filename-pattern="${ftp.server.localfilename}' attribute was used in file adapter which would match exact file name only process if that file exists. So once FTP starts downloading the file it will rename with some temp suffix until it has fully downloaded the file. Now this makes the file adapter pick up only the completed files.

Also we have code which also clears the local folder once the file has been sent to down stream for processing. Is there anything else which would have gone wrong here?
 
Mark Spritzler
ranger
Posts: 17347
11
Mac IntelliJ IDE Spring
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Not that I notice off the top of my head. What I would try is to do remote debugging and set a breakpoint at different points in the flow. But definitely one at that if (file.exists) statement and see what values have been set in the debugger for those objects.

Good Luck

Mark
 
Don't get me started about those stupid light bulbs.
reply
    Bookmark Topic Watch Topic
  • New Topic