Win a copy of Re-engineering Legacy Software this week in the Refactoring forum
or Docker in Action in the Cloud/Virtualization forum!
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

HTTPS File Uploader

 
Kumar Ala
Ranch Hand
Posts: 53
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi,

I am working with File upload using commons file upload. Is there anyway we can accelerate the upload process.
Any help is appreciated.


Thanks
 
James Ward
Ranch Hand
Posts: 263
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
How about zipping the file and uploading it?
 
Kumar Ala
Ranch Hand
Posts: 53
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
This is actually a web application so it will be difficult to ask clients to zip all the files and upload.
I am looking for some mechanism that will make the upload process much faster. Expecting almost up to 1GB files also from clients.

 
Paul Clapham
Sheriff
Pie
Posts: 20771
30
Eclipse IDE Firefox Browser MySQL Database
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Your server code doesn't have any control over the upload process. If the client uploads a 1-GB file then the network will transfer that gigabyte of data to your server code in whatever way it sees fit.

But look at what your server code is doing. Hopefully it isn't writing that gigabyte of data to a temporary file and then copying it to some other location; that would make the upload even longer.

You could also scrap the code you are using now and write an applet to assist with the uploading. This applet could upload the data to your server, gzipping it as it goes (as per James Ward's suggestion).
 
Kumar Ala
Ranch Hand
Posts: 53
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
paul,
I do understand why you are suggesting to go with Applet. I will try to give you my upload process overview
Constrains
1) Should use HTTPS protocol.
2) Each file should be scanned with UV scan in web server before moving into the right unix box.

With the above constraints i have to upload the files from my web application and move these files to right unix box after they are properly scanned for virus.

a) Presently i am using struts2 which inturn uses commons fileupload to upload the file from browser.
b) This is actually coping file in to the tmp locaiton in server and i am scanning files from that location and sending the files to right unix box using the JSch API.

Possible improvements i am looking in the following areas
p) Is there any mechanism that will improve the file upload part in the browser [i know that it is totally client side and the server does't have any control]
q) As you guys suggested gzipping might help me some extent.[I am yet to work on this]
r) Making seperate process for each file scanning might improve[using multiple threads]

Let me know if anyone have any suggestions to imporve the above solution or better idea that will improve performance for the above problem with the given contraints. No restrictions on the technologies as long as they are open sources[java].


Thanks
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic