This week's giveaway is in the Android forum.
We're giving away four copies of Android Security Essentials Live Lessons and have Godfrey Nolan on-line!
See this thread for details.
The moose likes Sockets and Internet Protocols and the fly likes Multi Files Transfer from client machines to server machines using socket communication in java Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Android Security Essentials Live Lessons this week in the Android forum!
JavaRanch » Java Forums » Java » Sockets and Internet Protocols
Bookmark "Multi Files Transfer from client machines to server machines using socket communication in java " Watch "Multi Files Transfer from client machines to server machines using socket communication in java " New topic
Author

Multi Files Transfer from client machines to server machines using socket communication in java

Riyas Hameed
Ranch Hand

Joined: Jan 07, 2012
Posts: 33
Hi All,

if any one have the code to transfer files from client to server , Could you please [share] the code for multi file transfering from client to server.
Expecting your valuable response ..

My requirement is transfering all files from particular folder in client machine to target folder in server machine, I have the code but its transferring only one file to server machine after that connection disconnect. plese guide me to complete my requirement. (Java Socket Communication)

Thanks in Advance..
Henry Wong
author
Sheriff

Joined: Sep 28, 2004
Posts: 18546
    
  40

http://www.coderanch.com/how-to/java/UseTheForumNotEmail


Books: Java Threads, 3rd Edition, Jini in a Nutshell, and Java Gems (contributor)
Riyas Hameed
Ranch Hand

Joined: Jan 07, 2012
Posts: 33
Henry Wong wrote:http://www.coderanch.com/how-to/java/UseTheForumNotEmail
Sorry for that. I was looking in thread but i didnt get a solution for Multi file transfer using socket communication in java, could you please guide , My code is transferring only one file at a time..


I am new to this concept. for file transfering which is the best mechanism in java , either Socket communication or FTP?
please advice me the best concept to do the file transfer mechanism .

Expecting your valuable reply.
Thanks in Advance .
Jeff Verdegan
Bartender

Joined: Jan 03, 2004
Posts: 6109
    
    6

Riyas Hameed wrote:
Henry Wong wrote:http://www.coderanch.com/how-to/java/UseTheForumNotEmail
Sorry for that. I was looking in thread but i didnt get a solution for Multi file transfer using socket communication in java, could you please guide , My code is transferring only one file at a time..


If you know how to do one file, then you know how to do multiple files.

I am new to this concept. for file transfering which is the best mechanism in java , either Socket communication or FTP?


There is no single "best" mechanism. The best one for you in this particular situation is the one that best meets your requirements in this particular situation.

Riyas Hameed
Ranch Hand

Joined: Jan 07, 2012
Posts: 33
Jeff Verdegan wrote:
Riyas Hameed wrote:
Henry Wong wrote:http://www.coderanch.com/how-to/java/UseTheForumNotEmail
Sorry for that. I was looking in thread but i didnt get a solution for Multi file transfer using socket communication in java, could you please guide , My code is transferring only one file at a time..


If you know how to do one file, then you know how to do multiple files.

I am new to this concept. for file transfering which is the best mechanism in java , either Socket communication or FTP?


There is no single "best" mechanism. The best one for you in this particular situation is the one that best meets your requirements in this particular situation.



Thanks for your reply, but I am getting connection refused after one file transferred to server, I am looking on that.

Here My code is :

Client part :

Server part:

kindly advise me , what mistake i did in my code ..
Jeff Verdegan
Bartender

Joined: Jan 03, 2004
Posts: 6109
    
    6

In the future, please UseCodeTags so your code will be readable.


You're closing your socket's OutputStream after each file. You need to either:

1) Keep the socket and its streams open throughout the entire loop over all files.

OR

2) Create a new socket connection for each file, and close it in a finally block at the end of transferring that file.

I'd recommend #2. If you go with #1, you'll have to develop an additional protocol to delineate the files. That's not necessarily difficult, but since you're having these difficulties so far, it's probably best to start simple.
Rob Spoor
Sheriff

Joined: Oct 27, 2005
Posts: 19656
    
  18

Riyas Hameed wrote:

0 is a valid return code for InputStream.read(byte[]). It means that currently no data is available, but in the future there still may be. -1 is the only indicator that there will not be any data anymore, so change the > into >=.


SCJP 1.4 - SCJP 6 - SCWCD 5 - OCEEJBD 6
How To Ask Questions How To Answer Questions
Riyas Hameed
Ranch Hand

Joined: Jan 07, 2012
Posts: 33
Thanks a lot to all, I will do the same , Iam new to this forum going forward i will ad my code in code tag. I will do what you have mentioned and come back to you if i am facing any issue.
Riyas Hameed
Ranch Hand

Joined: Jan 07, 2012
Posts: 33
Sir ,Please see the below code .
Riyas Hameed
Ranch Hand

Joined: Jan 07, 2012
Posts: 33
Thanks a lot , As per your words, I did the changes , Its working fine..I jus added one more while loop in server part







Now its working fine, This client application will run more than 500 Machines.So how can i make the best perofrmance .please suggest me to do this part. Thanks in Advacne.
Riyas Hameed
Ranch Hand

Joined: Jan 07, 2012
Posts: 33
Riyas Hameed wrote:Thanks a lot , As per your words, I did the changes , Its working fine..I jus added one more while loop in server part.

Now its working fine, This client application will run more than 500 Machines.So how can i make the best perofrmance .please suggest me to do this part. Thanks in Advacne.
Jeff Verdegan
Bartender

Joined: Jan 03, 2004
Posts: 6109
    
    6

Riyas Hameed wrote:Now its working fine, This client application will run more than 500 Machines.So how can i make the best perofrmance .please suggest me to do this part. Thanks in Advacne.


For parallel I/O such as this, the two main general approaches to optimize performance are 1) buffering, and 2) multithreading. You're already doing #1, although you may find that tweaking your buffer size up or down helps somewhat, so the main thing now is to use mulitple threads.

If I understand your scenario correctly you have many clients receiving files from a server and storing them on their local file systems. So you'll want multithreading in two places. 1) On the server, you'll want multiple threads to handle clients' requests. You'll have to experiment to find the optimal number. It could be anywhere between a few threads and a few hundred. 2) On the client, you might want two threads--one for reading from the server and one for writing to the file. Or that might not help at all. Impossible to say without trying it in the specific situation in which it will be used.

Other than that, the only thing you can do is see if the app is meeting your performance requirements, and if not, use a profiler to find bottlenecks.
Riyas Hameed
Ranch Hand

Joined: Jan 07, 2012
Posts: 33
Jeff Verdegan wrote:
Riyas Hameed wrote:Now its working fine, This client application will run more than 500 Machines.So how can i make the best perofrmance .please suggest me to do this part. Thanks in Advacne.


For parallel I/O such as this, the two main general approaches to optimize performance are 1) buffering, and 2) multithreading. You're already doing #1, although you may find that tweaking your buffer size up or down helps somewhat, so the main thing now is to use mulitple threads.

If I understand your scenario correctly you have many clients receiving files from a server and storing them on their local file systems. So you'll want multithreading in two places. 1) On the server, you'll want multiple threads to handle clients' requests. You'll have to experiment to find the optimal number. It could be anywhere between a few threads and a few hundred. 2) On the client, you might want two threads--one for reading from the server and one for writing to the file. Or that might not help at all. Impossible to say without trying it in the specific situation in which it will be used.

Other than that, the only thing you can do is see if the app is meeting your performance requirements, and if not, use a profiler to find bottlenecks.



Thank a lot for your support , In my requirement , All clients will transfer the files to the centralized server and save in server location.

So I have to create two threads in server side one for reading and one for writing ? please correct me If i am wrong. I have implemented thread concept in server side , some time some of the bytes are missing while writing in server side , now i am facing the issue . can i share my code here ??
Jeff Verdegan
Bartender

Joined: Jan 03, 2004
Posts: 6109
    
    6

Riyas Hameed wrote:
Thank a lot for your support , In my requirement , All clients will transfer the files to the centralized server and save in server location.

So I have to create two threads in server side one for reading and one for writing ? please correct me If i am wrong. I have implemented thread concept in server side


If all 500 clients are sending their files at the same time, I would start with something like 10-20 threads reading from clients, putting into queues (1 queue per destination file) and 1-5 threads reading from the queues and writing to files. I'd probably use ThreadPoolExecutors and ArrayBlockingQueues or LinkedBlockingQueues.

Since I don't know your system or your requirements at all, this is just starting point. You'll have to do some research into the relevant classes, and do some testing and tweaking on your setup to see what works best.

, some time some of the bytes are missing while writing in server side , now i am facing the issue


Then there's a bug in your code.

. can i share my code here ??


Probably the whole thing will be too much for anybody to want to read. You'll have better luck if you can create an SSCCE that shows the relevant parts of your code and ideally demonstrates the problem, without any extra stuff that's not directly related to the problem.
Riyas Hameed
Ranch Hand

Joined: Jan 07, 2012
Posts: 33
Jeff Verdegan wrote:
Riyas Hameed wrote:
Thank a lot for your support , In my requirement , All clients will transfer the files to the centralized server and save in server location.

So I have to create two threads in server side one for reading and one for writing ? please correct me If i am wrong. I have implemented thread concept in server side


If all 500 clients are sending their files at the same time, I would start with something like 10-20 threads reading from clients, putting into queues (1 queue per destination file) and 1-5 threads reading from the queues and writing to files. I'd probably use ThreadPoolExecutors and ArrayBlockingQueues or LinkedBlockingQueues.

Since I don't know your system or your requirements at all, this is just starting point. You'll have to do some research into the relevant classes, and do some testing and tweaking on your setup to see what works best.

, some time some of the bytes are missing while writing in server side , now i am facing the issue


Then there's a bug in your code.

. can i share my code here ??


Probably the whole thing will be too much for anybody to want to read. You'll have better luck if you can create an SSCCE that shows the relevant parts of your code and ideally demonstrates the problem, without any extra stuff that's not directly related to the problem.



Thank you very much, yes I have to go through all other thing which you have mentioned in the previous message . I am not sure but atleast minimum 50 to 100 Machine will send a file to server at a time . i will discuss with them and come back to you ..
Riyas Hameed
Ranch Hand

Joined: Jan 07, 2012
Posts: 33
Riyas Hameed wrote:
Jeff Verdegan wrote:
Riyas Hameed wrote: Thank a lot for your support , In my requirement , All clients will transfer the files to the centralized server and save in server location. So I have to create two threads in server side one for reading and one for writing ? please correct me If i am wrong. I have implemented thread concept in server side
If all 500 clients are sending their files at the same time, I would start with something like 10-20 threads reading from clients, putting into queues (1 queue per destination file) and 1-5 threads reading from the queues and writing to files. I'd probably use ThreadPoolExecutors and ArrayBlockingQueues or LinkedBlockingQueues. Since I don't know your system or your requirements at all, this is just starting point. You'll have to do some research into the relevant classes, and do some testing and tweaking on your setup to see what works best.
, some time some of the bytes are missing while writing in server side , now i am facing the issue
Then there's a bug in your code.
. can i share my code here ??
Probably the whole thing will be too much for anybody to want to read. You'll have better luck if you can create an SSCCE that shows the relevant parts of your code and ideally demonstrates the problem, without any extra stuff that's not directly related to the problem.
Thank you very much, yes I have to go through all other thing which you have mentioned in the previous message . I am not sure but atleast minimum 50 to 100 Machine will send a file to server at a time . i will discuss with them and come back to you ..



Here is my code for read and write in file :

Server side : read the data from the file which have sent from client and writ in a file .


Client code :




Some of the file bytes are missing while writing to server . please advise me to solve this . Thanks in Advance
Paul Clapham
Bartender

Joined: Oct 14, 2005
Posts: 18541
    
    8

If you read the documentation for the "read" method you're using, you will see that it returns -1 when it gets to the end of the stream. But you aren't checking for that. You are also considering 0 as a reason to stop reading.

But when the "read" method returns 0, that just means it has no bytes for you just at the moment. It doesn't mean that there will never be any more bytes. So fix your end-of-stream tests.
Jeff Verdegan
Bartender

Joined: Jan 03, 2004
Posts: 6109
    
    6

Paul Clapham wrote:If you read the documentation for the "read" method you're using, you will see that it returns -1 when it gets to the end of the stream. But you aren't checking for that. You are also considering 0 as a reason to stop reading.

But when the "read" method returns 0, that just means it has no bytes for you just at the moment. It doesn't mean that there will never be any more bytes. So fix your end-of-stream tests.


This was already mentioned by Rob Spoor many replies and 4 days ago. Perhaps this time it will sink in. :-)
Paul Clapham
Bartender

Joined: Oct 14, 2005
Posts: 18541
    
    8

You're right. And not only that, Rob described exactly what code changes should be done. And that was the only thing in his post. So yeah, there was really no excuse for
not noticing his post.
Rob Spoor
Sheriff

Joined: Oct 27, 2005
Posts: 19656
    
  18

Sometimes people are focusing so much on what one person said that what others say doesn't sink in. This wouldn't be the first time when a post of mine had to be brought under the attention again because it was overlooked at first.
Jeff Verdegan
Bartender

Joined: Jan 03, 2004
Posts: 6109
    
    6

I think it's happened to us all a number of times. You get in a thread with multiple respondents and multiple issues to be addressed and the OP gets overwhelmed.
Riyas Hameed
Ranch Hand

Joined: Jan 07, 2012
Posts: 33
Rob Spoor wrote:
Riyas Hameed wrote:

0 is a valid return code for InputStream.read(byte[]). It means that currently no data is available, but in the future there still may be. -1 is the only indicator that there will not be any data anymore, so change the > into >=.





Rob, I tried already what you said in the above conversation, even though i got the same error only, extremely sorry, forget to noticed that one.
Jeff Verdegan
Bartender

Joined: Jan 03, 2004
Posts: 6109
    
    6

Riyas Hameed wrote:
Rob Spoor wrote:
Riyas Hameed wrote:

0 is a valid return code for InputStream.read(byte[]). It means that currently no data is available, but in the future there still may be. -1 is the only indicator that there will not be any data anymore, so change the > into >=.



Rob, I tried already what you said in the above conversation, even though i got the same error only, extremely sorry, forget to noticed that one.


The last code you posted shows:


Which is still the same mistake you were making in the beginning, and does not incorporate Rob's (and Paul's) advice. Fix that, and then if you're still having trouble, post your latest code, or an SSCCE that accurately represents it.
Riyas Hameed
Ranch Hand

Joined: Jan 07, 2012
Posts: 33
Riyas Hameed wrote:
Rob Spoor wrote:
Riyas Hameed wrote:

0 is a valid return code for InputStream.read(byte[]). It means that currently no data is available, but in the future there still may be. -1 is the only indicator that there will not be any data anymore, so change the > into >=.





Rob, I tried already what you said in the above conversation, even though i got the same error only, extremely sorry, forget to noticed that one.


Riyas Hameed
Ranch Hand

Joined: Jan 07, 2012
Posts: 33
Riyas Hameed wrote:
Riyas Hameed wrote:
Rob Spoor wrote:
Riyas Hameed wrote:

0 is a valid return code for InputStream.read(byte[]). It means that currently no data is available, but in the future there still may be. -1 is the only indicator that there will not be any data anymore, so change the > into >=.





Rob, I tried already what you said in the above conversation, even though i got the same error only, extremely sorry, forget to noticed that one.




Thanks a lot for your support, Now it is working fine . I have changed the condition to till -1. Thanks for your time ..
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Multi Files Transfer from client machines to server machines using socket communication in java
 
Similar Threads
problem in RunTime.exec
Help Please- on design approach for data collection across n/w
Transfer multiple files from Server to Client using Sockets
transfer files between 2 pc's
help me out