aspose file tools*
The moose likes Sockets and Internet Protocols and the fly likes Please HELP Socket Amateur! Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Spring in Action this week in the Spring forum!
JavaRanch » Java Forums » Java » Sockets and Internet Protocols
Bookmark "Please HELP Socket Amateur!" Watch "Please HELP Socket Amateur!" New topic
Author

Please HELP Socket Amateur!

Sam Moran
Ranch Hand

Joined: Sep 28, 2002
Posts: 86
I am trying to transfer a simple text file to a server using a Socket and ftp commands. I get connected and authenticated to the server, and I see the bytes going out. I just don't understand what I'm missing here. The file never makes it to the server. Can anyone please help? The code is below, thank you in advance:

[ October 27, 2002: Message edited by: Sam Moran ]
[ October 27, 2002: Message edited by: Sam Moran ]
[ October 27, 2002: Message edited by: Sam Moran ]
[ October 27, 2002: Message edited by: Sam Moran ]
[ October 27, 2002: Message edited by: Sam Moran ]
[ October 27, 2002: Message edited by: Sam Moran ]

We make a living by what we get, we make a life by what we give!
Lewin Chan
Ranch Hand

Joined: Oct 10, 2001
Posts: 214
Hi,
Port 21 is only used as the control socket for FTP... Data is actually transferred via a different socket connection, either initiated by the remote FTP server or by yourself (passive FTP mode).
I'm not personally sure how all this works, I just use the library from http://www.enterprisedt.com myself. If you really want to re-invent the wheel , you could try looking at RFC959 which details the core FTP spec.


I have no java certifications. This makes me a bad programmer. Ignore my post.
Sam Moran
Ranch Hand

Joined: Sep 28, 2002
Posts: 86
I thought that once you created a socket that you could send data over the same socket. It is bi-directional, no?
Lewin Chan
Ranch Hand

Joined: Oct 10, 2001
Posts: 214
Sam,
Yes a tcpp/ip socket is bi-directional in nature, however this doesn't mean that the FTP protocol uses it like that, which is what you're trying to emulate using raw sockets.
If you were using the HTTP protocol, then the same socket would be used to send and receive data, subject to the protocol format/specification.
Network gurus may, at this point, optionally, start going on about the OSI 7 layer model or some other terminology that I never really pay much attention to
brgds
L
Sam Moran
Ranch Hand

Joined: Sep 28, 2002
Posts: 86
I concede that I have been given bad information about send data over the same socket from a Java Instructor. Go figure! I have a Server socket and when I wait for the server.accept(); I wait for ever! I know for a fact that the ftp server is an NT BOX, and that the ftp service is Microsoft's , but does anyone know whether or not there is a way to prevent the NT server from sending the accept? I wonder if that is my problem. Could the NT admin have shut this off? Am I making any sense, or do I have my head hidden where the sun doesn't shine? Any thoughts? Thank you.
Lewin Chan
Ranch Hand

Joined: Oct 10, 2001
Posts: 214
Sam,
I'd suggest doing it in passive mode rather than having to write a server. This can be activated by using PASV command, what you get back from the session is something like this

Now, 192,168,0,1 is the IP address (replacing those , with . )
And 16,204 represents the port such that

16 bitshifted 8 times to the left, and then add 204 (is that the right description?)
Then you can just do a STOR command on mySocket, and then transfer the data by

You should perform a PASV command each time you do a actual data transfer, I wouldn't expect the passive socket to live on after you've sent the data.
Is this too much information for you yet?
brgds
L
Sam Moran
Ranch Hand

Joined: Sep 28, 2002
Posts: 86
Thank you for the info! I am trying to work out pulling the responses from the server, and have not found the correct combination. I'll let you know how I make out with the PASV.
Sam Moran
Ranch Hand

Joined: Sep 28, 2002
Posts: 86
Okay, so I add the following lines and either the server doesn't send the proper EOL or I'm using the wrong reader?!?! The code stops working where ever I place it, no error, just stops. Should I be using a different reader? Thanks!

Lewin Chan
Ranch Hand

Joined: Oct 10, 2001
Posts: 214
Sam,
Given your code excerpt, and a correct command being issued, I would expect to see something like

being returned by the server (from a raw telnet session).
Now, in your code, and I could very easily be wrong here, bufferedReader.readLine() will block if there's no data forthcoming from the server (for SO_TIMEOUT ms before throwing InterruptedIOException? ). So, I would actually see a print out of
S: 200
And then nothing....
Now, an FTP server response is only ever 1 line long, and only the first 3 digits are relevant, the rest is just informational.

This is what I would expect to work (apologies for the alarming lack of actual java correctness

brgds
Lewin
Sam Moran
Ranch Hand

Joined: Sep 28, 2002
Posts: 86
Thank you again for the response! You must be hoping that either die or go away! That worked for reading the line, but the problem is, is that when I issue the PASV command, nothing comes back from the server. I am including the small snipet of code:

Lewin Chan
Ranch Hand

Joined: Oct 10, 2001
Posts: 214
FTP is a fairly interesting protocol, and reading RFC's makes me look like I'm actually doing something at work
The "problem" with your code is that you aren't issuing *another* command after your PASV. If you changed your code so that you did a "CWD .." after the PASV you'd see the PASV response.
I suppose some might argue that technically it's not a problem with your code, it's more to do with the socket being flushed correctly.
L
Sam Moran
Ranch Hand

Joined: Sep 28, 2002
Posts: 86
I did just as you suggested and issued the CWD after the PASV like so:

I never get a response from the server. I have a couple of questions:
1) I am a little confused about the use of the writer as opposed to what I had originally a PrintWriter, and issued a println. Aren't they the same?
2) I read somewhere that the flush() is what actually send the data to the server. Is this true?
3) I keep wondering if the Microsoft ftp server doesn't follow the RFC959 standard? I had a similar problem pulling files from another Microsoft ftp server and had to abandon the socket approach and use the URLConnection.
Again, thank you very much for being so helpful with my understanding of this socket implementation.
Lewin Chan
Ranch Hand

Joined: Oct 10, 2001
Posts: 214
Sam,

1) I am a little confused about the use of the writer as opposed to what I had originally a PrintWriter, and issued a println. Aren't they the same?

I suppose they are the same, however you should remember that println() will print the platform specific line terminator. This may or may not be suitable to terminate a FTP command. For instance, HTTP specifies that a header line should terminate with \r\n, however, on Unix, the default line terminator is simply a \n. If you used println(), there would be no guarantee that a very strict Web server wouldn't just completely ignore you.

2) I read somewhere that the flush() is what actually send the data to the server. Is this true?

Not strictly true, flush() causes the buffer to be flushed. If the amount of data that you were sending was over Socket.getSendBufferSize() then data would automatically be written to the socket (not all of it, just enough so that getSendBufferSize() wasn't exceeded). The send/receive buffersize is generally defaulted to 8k or so, which is why you use the flush a lot.

3) I keep wondering if the Microsoft ftp server doesn't follow the RFC959 standard? I had a similar problem pulling files from another Microsoft ftp server and had to abandon the socket approach and use the URLConnection.

M$ not adhering to standards, whatever gave you that idea :roll: . I have to admit that my last suggestion was made having run the test against a linux ftp server.
brgds
Lewin
Sam Moran
Ranch Hand

Joined: Sep 28, 2002
Posts: 86
I have code that works with UNIX like boxes! Go figure right! I suppose that the problem is with the NT ftp box!
I will have to get some help from the NT admin who set this server up. I keep wondering if he is blocking portions of the service?!?!
I can't thank you enough for all of the help explain the socket stuff.
Sam Moran
Ranch Hand

Joined: Sep 28, 2002
Posts: 86
Lewin, if you are still out there, or if someone else can help, I just got the PASV command to work! I had to add 6 of the following lines after I pass my Password:

I believe the problem is with reading the data from the server! The following doesn't work because the server never sends NULL or the client never gets NULL?!?

So what is the best way to process an unknown amout of data coming in? I need to detect the PASV reply, and discard the stuff that is held in the buffer, with:

Can anyone point me to the FORCE! Thank you.
Peter den Haan
author
Ranch Hand

Joined: Apr 20, 2000
Posts: 3252
The null value is never really sent over the wire. It is returned by readLine() when the end of the stream is reached -- i.e., when the remote server has close()d the connection.
Otherwise, readLine() will simply block until the remote server sends you some data.
I've never studied the FTP protocol but am confident that if you carefully check the specification, you will find that there is some way to know what to expect next, either through a file length count, magic terminator sequence, whatever.
- Peter
Lewin Chan
Ranch Hand

Joined: Oct 10, 2001
Posts: 214
Sam,
I'm still here. There's a couple of things you could try.
a) Don't use BufferedReader, write your own FilterInputStream that can provide a "readLine()" style functionality.
b) or maybe, seeing as we know that we expect a reply from the server...

might work for you.
c) Use of cmdin.ready() might help as well.

L
Sam Moran
Ranch Hand

Joined: Sep 28, 2002
Posts: 86
Here is my current thinking:
I am going to see if I can get the following to work:

If this doesn't work and so far it is not! I am going to go to the M$ website and see what their interpretation is of the RFC959.
Currently, I am able to read all of the lines properly, but I still experience a block. I did go back to the standard and it says that the terminator is <crlf>. Thanks again for the help, if you see anything wrong with this logic please let me know.
Lewin Chan
Ranch Hand

Joined: Oct 10, 2001
Posts: 214
Sam,
It won't work if using BufferedReader
From jdk1.4 BufferedReader docs

public String readLine() throws IOException
Read a line of text. A line is considered to be terminated by any one of a line feed ('\n'), a carriage return ('\r'), or a carriage return followed immediately by a linefeed.
Returns:
A String containing the contents of the line, not including any line-termination characters, or null if the end of the stream has been reached Throws:
IOException - If an I/O error occurs

brgds
L
Sam Moran
Ranch Hand

Joined: Sep 28, 2002
Posts: 86
I am using JDK 1.3.1, and I just read what your previous post explained from O'Rielly's book. The BufferReader ReadLine() reads a line at a time but removes or ignores the line terminator! So, based on that do you suggest that I try the FilterInputStream? I did a first attempt at this but it yacked because it is Private. My implementation was off but if you think that this is the way to go, I will correct. I could, I suppose do a character by character parse, but this seems over kill!
Lewin Chan
Ranch Hand

Joined: Oct 10, 2001
Posts: 214
Using your own FilterInputStream and parsing char by char will will be highly inefficient and wont' scale you realise... but sometimes, you gotta do what you gotta do.

L
Sam Moran
Ranch Hand

Joined: Sep 28, 2002
Posts: 86
Lewin, after getting the PASV command sent and receiving and pulling the pieces of the IP and port information and attempting to open a Data Socket with that information, the F@*KING server won't let me connect to it!!!
So, I am of the opinion now that I should go with you original suggestion and use the FTP Client classes. The thing is I have used some of the same principals that they used and they have failed. So, my question to you is have you used them to upload a file successfully to a M$ server before? Because I have found a temporary solution, albeit one that I am not happy with, and that is to have a mapped drive on the PC that I am running my program from.
What really infuriates me is that I have got my simple ftp stuff to work on UNIX boxes, but not NT!
Thank you again for the help. If you have any any other ideas I'm all ears.
Lewin Chan
Ranch Hand

Joined: Oct 10, 2001
Posts: 214
Sam,
I haven't used those classes to test a M$ server, but I would be inclined to trust the classes from enterprisedt as a lot of people here use em everyday for their FTP'ing.
Here's what I did to test the PASV stuff.

And then in another window

Now, of course, it's entirely possible that ftp.microsoft.com isn't actually running a windows ftp server, but that would be stupid right?
The connection to the data socket was automatically closed when I disconnected from the control socket.
Perhaps it's something to do with the server setup that your network admin can assist with. Maybe you should install IIS on your local machine, and have a play with the FTP classes locally.
L
-- I now know about FTP, something I really didn't want to learn.
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Please HELP Socket Amateur!