I am building a program to ftp some files which has a few parts. The download can happen at anytime. How can I detect whether I have finished download the few files besides from using threading? Any api that is suitable for this? Thanks
I am assuming that you are referring to the receiving end, as the sending side can easily tell when it is finished...
IMHO, you have a couple of ways that it could work...
- You could monitor the modification times. If the file hasn't been modified in the last minute or so, then you can assume that the transfer completed.
- You could modify the sender to send a small marker file. For example, it transfers a file call "ABC.TXT", when it is done, it transfers another file called, ".MYPROG.MARKER.FILE.ABC.TXT". You receiver should then just check for the appearance of ".MYPROG.MARKER.FILE.*" and assume the file it is referring to, is completed.
Hi Thanks for the reply but actually I need to determine that all 10 files has been downloaded successfully. Meaning I have to detect the point where all the 10 files has been ftp in sucessfully. Telling the sender to send in confirmation files when finish downloading is not impossible but troublesome as this may involve an additional 10 files to send from the other side. Thanks
Doesn't the File-Transfer-Protocoll define a way to tell when finished? When I download a very small file, I don't have to wait a minute or so. Firefox immediately knows when ready. It also knows appromaxitive how long the download will take and exactly, how big the file will be - that might be a starting point.
And I don't receive MARKER.ABC files all the time (and how to know I finished receiving them? get a MARKER.ABC.MARKER.ABC file )
Originally posted by Stefan Wagner: Doesn't the File-Transfer-Protocoll define a way to tell when finished? When I download a very small file, I don't have to wait a minute or so.
The FTP client knows when a file transfer has finished. The FTP server knows when a file transfer has finished. Unfortunately, in this case, it is neither the ftp client or server -- it is a java program sitting on the server side, waiting for files to appear.
Actually the way I read the original post, the question is whether the client can tell when the download is complete. But it's obvious that an FTP client can tell when it has finished downloading a file, so there must be more to the question than that. I don't see why it would be difficult for this program to have a few boolean flags somewhere, and set one for each download. When all the flags are set, all the downloads are done.
Joined: Nov 20, 2003
Sorry for the vague question. I need to do a client program to check whether the 10 files is completely obtained so that I can trigger another program to make a report based on the data on those 10 files. So I need a program that can check from time to time.
I was under the impression (incorrectly) that this was a third program that was not involved in the transfer. If this is the ftp client, then detecting completion should be part of the API. (of the ftp library that you are using)
If you can't get the people sending the file to cooperate, here's what I would do: when you first see the file on your system, wait one hour and then process it. If they don't like the one-hour delay, suggest to them that a more reliable method (like sending a control file) would enable you to speed the process up.
Joined: Jan 01, 2006
come one everyone! I am sure this is a common problem and there must be smart and fail-free mechnism to detect. Please suggest a solution other than the control file solution.
Samir, if you believe there is a simple and reliable solution then by all means go out and find it and post it here. I have never found any such solution. And I have participated in quite a few discussions of the topic (including the one you linked to) and none of them ever produced anything practical. But a simple solution must be easy to find, so please do.
What we usually agree with our business partners is that we will do an FTP upload to a file whose name they are not looking for, and when the upload is complete then we will rename the file to something they are looking for. And likewise for their uploads to us. They all agree to do that because they've had the problem too.
I too had this problem - on an AIX platform. I managed to solve it (so far..?) like this..
When the sender is sending the file to your machine and your Java app needs to check this inbound file to see if it's "finished", the best way i found was to try and "read" the file. Basically if it's still transferring, it will throw a FileNotFoundException which you catch and handle and essentially "loop" until the error doesn't happen.
Trying the built in File.canRead() or File.canWrite() functions are useless -- they always seem to return a "true" which isn't what you want for an FTP transfer check. The reason being on UNIX is the swap file -- yuo can still rename and move a file around while it is in mid-FTP transfer so the built in "java.io.File" commands that seem obvious don't work.
I figured out that if you try to physically "read" the first line of data in the file while it is in FTP transfer, it throws the FileNotFoundException. When the transfer is done, it doesnt throw the error, meaning it's cool for you to do what you need to the file. here's a snippet of what I did:
The problem with this now is what happens if you get a file that has 0 bytes in it - well, it loops forever I need to put another check into this. BTW, it *was* working flawlessly for the past 3 years until a 3rd party sent us an unexpected file with 0 bytes. It shouldnt have happened but it highlighted the flaw, so I am correcting it now.
But the main issue is, that if you try to do a br.readLine() on a file that is in mid-transfer, it fails. So just handle the exception (ie, do nothing) and keep looping until the exception ISNT thrown.
Hope maybe this helps - it's worked wonders for me as I couldnt find any solution to this issue on AIX.